User Testing for Player Eviction Menu

Background

The role of the player eviction menu is to allow the player to choose the traitor (correct) to achieve the game ending based on the information/clues they get from the game's storyline. In sprint 1, our team only tested and evaluated the UI design and basic interaction flow of the eviction menu as the studio had not yet implemented the usage of the storyline. Feedback on this visual and functional effect is necessary for the next phase of the work.

Introduction

The purpose of this user test is to understand the satisfaction and opinion of players (respondents) with the current player eviction menu. These tests mainly focus on the design image and interaction process of the eviction menu. (Note: Since it is not suitable to use unit tests to test code functions at this stage, the data of user testing will also be fed back to developers synchronously for further functional optimization.)

Test Plan

Users (respondents): Members of other Teams in the studio, other students interested in the game, and potential users.

1. Task:

• Enter the game interface, find the Eviction Menu icon and click to enter

(Purpose: Used to know whether the entrance to the Eviction Menu is easy to find.)

• Select the card corresponding to the NPC you think is most likely to be a traitor and expel him

(Purpose: To evaluate whether the interaction flow is clear and understandable.)

• Test whether the display effect of each operation is normal

(Purpose: Different devices will have different screen sizes and resolutions. Test that the interface works with most devices.)

2. Get feedback: Respondents feedback were collected through questionnaires and further interviews were conducted on important or low-scoring questions to supplement data and analysis

Process

A total of 15 users were selected for evaluation in the test process, they were from:

• Members of other teams in the studio
• Classmates and friends of team members

1.Test tasks:

Respondents (Users) need to complete the three basic tasks in Test plan that mentioned above, the process and results will be recorded.

(Note: 12 of them were tested online via Zoom screen sharing. The remaining 3 users were tested in-person.)

2.Questionnaire:

After completing the task, all respondents (users) needs to complete a short questionnaire, which contains 4 satisfaction questions about the task and design.

3.Online interviews:

After collecting and analyzing all questionnaires, as a supplement, we selected 3 respondents with low satisfaction and another 2 with high satisfaction and strong interest to conduct online interviews to obtain more evaluation details.

---

Questionnaire

Distribute satisfaction questionnaires to all respondents who participated in the test： The user testing questionnaire

Online Testing and Interviews

Due to the limitations of the scenario, most of the respondents completed the task online (via zoom). And the purpose of the interview was to get more detailed feedback based on the specific question.

Important Questions

Questionnaire

Analysis and Summary

Questionnaire analysis

Q1 is mainly for the interaction problems encountered in the test tasks. Although the majority of respondents expressed satisfaction (73.3%), the proportion of neutral and dissatisfied reached 26.6%. In the field of interaction, this data is sufficient to indicate that there are certain problems in the interaction. This issue will be highlighted in online interviews for more details.

Q2Q3 are mainly for the level of satisfaction with the color scheme and design. According to the data, the satisfaction levels reached 80% and 73% respectively, which means that there is a low probability of needing significant optimization. However, in the following online interviews, a dissatisfied respondent gave valuable opinions to us, which made us realize that the reason for dissatisfaction may not be entirely on the issue of design and preferences.

Q4 is a vote on the two future design ideas. The idea of 'pop-up a new page' has been approved by the vast majority of respondents, reaching 78.6%.

Interview analysis

As a supplement to questionnaire, interview obtained more details for the results of the data.

In terms of the interaction, the following feedbacks were obtained:

1. When the mouse hovers over the button, there is no interactive feedback to confirm whether the mouse is in the button area and whether it is a button (select|ok|cancel button).

2. There is a cross icon at the top right of the background image, which is not a button. The real exit button is at the bottom of the screen. As a result, the respondent subconsciously clicked on the ineffective cross icon instead of the real exit button. Additionally, repeated elements usually confuse the respondent.

3. Both of the Ok and cancel buttons in the confirmation dialog will close the dialog only without any other actions.

In terms of design and color matching:

4. A dissatisfied respondent has high requirements for color and design due to her major. She thinks that we should use higher pixel resolution to reflect the mysterious feeling of the ocean (adding more refined color matching).

5. In addition to personal aesthetic factors, another reason for giving low satisfaction scores on design issues is that during the interaction process, the eviction menu does not give users actual feedback on the operation, which is not attractive enough (for example, after clicking "confirm eviction", this is also mentioned in the interaction questions).

Summary

In this sprint, the design and interaction process of most functions are as expected, but there are still many small problems that need to be optimized. However, at the end of sprint 1, the above problems were also resolved:

1. The developer has added properties for all buttons, when the mouse hovers over the button, the button changes color to give feedback on the current state.
2. Ui designers and developers collaborated to remove the exit button in the lower middle and change the cross icon in the upper right corner into an exit button with the mouse-over color change attribute.
3. The designer has optimized the design style and color scheme to a lesser extent.(It needs to be considered consistent with the style of the entire studio, and major changes are not recommended)

A small number of users who were dissatisfied with the interaction flow and design style were told that the problem lies in the design and function implementation at this stage only for the UI interface. At the same time the design style and the implementation of the code depend on the design style and progress of the studio. However, over the next few sprints, the team will discuss and consider implementing subsequent features with relevant teams.

Future Testing/Evaluation

The test data and feedback in this sprint will serve as the basis for user testing in the next sprint. In order to match the tasks that the team may need to accomplish in the next sprint, we have designed the direction of testing and evaluation that may be required in the future:

• Add base image of npc in eviction card
• Add a pop-up page or window to display detailed information/clues related to each npc
• If possible, show the result of the decision after the user selects and evicts the npc

(Note: The specific functions that can be realized also need to be determined according to the task progress and cooperation of the studio.)

1. Exchange ideas with relevant Teams that may collaborate.
2. Think about the feasibility of implementation and the difficulty of integration through brainstorming and communication.
3. Test whether the integrated player eviction menu can meet the requirements of users at this stage (e.g. whether the basic image of NPC meets expectations, or whether the new pop-up page/window helps to determine who is a traitor.)
4. Discuss with relevant teams and determine if improvements are needed.
5. Invite members of the studio as well as other students (friends or potential users) to complete user testing of the design, interaction flow and functionality.