User Testing for Win lose Condition for Selecting the Traitor

Sprint3 - Team 7: User Testing for Win/lose Condition for Selecting the Traitor (After Designing)

Background

Players in Atlantis Sinks will identify the NPC that resulted in the continent's sinking. Our team focus on implementing the eviction menu, which allows players to choose an NPC they believe to be a traitor.

In the previous two sprints, our team achieved the most basic features for the player eviction menu, such as designing the entire window, displaying the NPC information on the card, and several prompt windows for different functions (confirmation box and detailed information/clues window).

In sprint 3, our team is mainly responsible for the most important parts of the win-lose logic of this game, and these parts will also rely on the eviction menu to display the current game state.For example to display the win/lose conditions on the eviction menu by popping up several different windows after player ensure their traitor selection.

This user testing will examine and evaluate the UI design and essential interaction flow of the win/lose prompt window (on the eviction menu) in order to gain feedback on aesthetic and functional outcomes in the initial design, as well as anticipated additional features for the next sprint. The suggestions and feedback collected from this user testing are valuable and favourable to our improvement of the initial design idea and the following sprint's development.

Introduction

This user test intends to determine the satisfaction and opinion of users (respondents) with the current design for the win/lose prompt window. These tests focus mostly on the presentation and functionality of the newly added features(win/lose conditions) to the eviction menu.
(Note: data from user testing may also be provided synchronously to developers for development and optimization.)

Test Plan

Users (testing participants): people from other Teams in the studio, other students with an interest in this game, and potential users.

1. Tasks:

• Enter the interface of the Eviction Menu, then arbitrarily click the SELECT button below the NPC cards to pop up the win prompt window.

(Purpose: To evaluate whether the player can understand the content of the win prompt window and what to do next step. )

• Click the SELECT button below other NPC cards to trigger the lose-prompt window.

(Purpose: To determine whether the loss window for the first incorrect selection clearly instructs the player to select traitor again.)

• Repeat the selection of other NPC cards except for the correct traitor for a total of three times.

(Purpose: To evaluate whether the player clearly understands the current state of the game and what to do next.)

• Use the prototype of the select traitor function to experience the complete flow of win-loss prompts logic.

(Purpose: To determine whether the interaction process conforms to the player's habits, and whether there is an unreasonable process.)

2. Get feedback: Responses were gathered through the questionnaire form. Besides, online interviews were conducted to complement data collection and analysis on outlier data deemed crucial or low-scoring.

Process

15 participants were chosen for the testing process, and they came from a variety of groups, including:

• People from other teams in the same studio
• Team members' classmates and acquaintances

1.Test tasks: Respondents (Users) are required to perform the four mentioned tasks outlined in the Test plan; the process and results will be documented. (10 of them were evaluated online using Zoom screen sharing. The remaining 5 users were evaluated in person.)

2.Questionnaire: All responders (users) will be asked to fill out a questionnaire regarding their experience with the mentioned tasks once they completed those tasks.

3.Online interviews: After gathering and evaluating all questionnaire forms, we supplemented our survey results with online interviews and selected three participants who are more likely to express specific suggestions or ideas.

---

Questionnaire

The distributed questionnaire to all 15 respondents that participated in the test:

The user testing questionnaire(google form)

The user testing questionnaire(Tencent questionnaire)

Online Testing and Interviews

The purpose of the interview was to elicit more concrete replies and suggestions to the particular question. And the majority of respondents will be interviewed online (via zoom) due to situational restrictions.

Important Questions

Questionnaire

Analysis and Summary

Questionnaire analysis

Q1 reflects the degree of satisfaction with the UI design of the layout and color palette.

Many people (60%) are not satisfied with the current design (including neutral). This data means that participants may expect us to make some improvements in the design, such as adding some patterns to make the design of this window more interesting. This feedback let us discover that additional factors may influence players' satisfaction with design ideas. The online interview will further discuss the interviewees' specific design methods and color matching, which will be addressed in the analysis. And then to improve the design of this win-loss window.

Q2 primarily surveys participants' satisfaction with the interaction process of the Win/Lose Prompt Window.

A large number of respondents (60%) feel that the interaction process is not so clear that players are not aware of what stage they are in the game and what to do next (including neutral). This is probably due to that:

1. The lose prompt window triggered by submitting the first time or multiple times incorrect traitor selections is the same, which might cause users confused about the game state

2. Lack of transition and connection to the next stage (e.g. task or scenario), this make the audience unclear about what to do next in order to complete the game.

These data properly indicates several possible interaction issues with this win/lose prompt window, which cannot be disregarded and should be addressed further in online interviews.

Q3 mainly investigates whether the design of the win-and-lose popover can be well combined with the storyline.

The close integration of design and story will make the player more immersed in the story of the game and lead to a better game experience because the game “Atlantis Sinks” relies heavily on the storyline.

The overwhelming majority of participants (86.7%) felt that the current design was not closely integrated with the storyline of the game. Combined with the analysis of the interaction process in Q2, those two factors resulted in bad experience and that will also lead to the current design to be unsuitable for the display of game states. Therefore, we will further determine a more specific promotion plan through interviews to make the design of the win-and-lose popover more closely combined with the plot.

Q4 is about the potential optimization methods that can be implemented in the future.

After collecting statistical data on participants' expectations for different optimization aspects, we can determine which parts may be implemented first in next sprint.

According to the data collected, adding proper animation to the win/loss popup is the most favored by the participants (46.7%), in order to make the eviction gameplay more visual, it may also be necessary to add a transition interface (animation) to show the changes of game state to the player. Sound effects are also believed to be effective in increasing player engagement. Therefore, these 3 improvements will be considered for implementation in the next sprint.

Interview analysis

In addition to the questionnaire, an interview was used to glean further specific details for the data findings.

The following feedback is about the design concept:

We found that the lose prompt window may appear multiple times, so we can change the design in the element collocation, so that players can get corresponding feedback according to the number of mistakes when choosing the wrong traitor and make it easier for them to understand the state of the game. Through the communication with the interviewees, our team found that we should unify the frame design (background), make the design style more unified and add some elements that fit the storyline. This may allow for a clearer display of the game state and the outcome of choosing the traitor, while making more use of color to express which stage the player is in.

1. Same background element: complete board (the win prompt window), slightly broken board (the first wrong selection), severely broken board (for second/multiple wrong selections).

2. Background colour: The colour of the background board is bright and brilliant yellow (the win prompt window); the colour of the background board is deepened from bright yellow to earth brown, and a small part of the colour is grey to represent damage and necrosis (the first wrong selection). The background board is complete grey and black to show the dead atmosphere (for second/multiple wrong selections).

3. Add images of different states of the figure image. For the win prompt window, a picture of a traitor being tied up will be added to show that the traitor selected by the player is correct and the traitor has been punished. In the lose prompt window for the first wrong selection, an image of a figure wearing a cloak will be used to show that the true traitor's identity remains unknown and the player's selection is wrong. For the lose prompt window for the multiple wrong selections, the figure pattern will become darker and close to hiding in the dark to imply that players will have no chance to find the correct traitor.

4. Other decorative elements: The win prompt window will add a crown to imply that the player has chosen the right traitor and will successfully pass the game; the lose prompt window for the first time will add a bone pattern to represent the damage caused by the wrong choice of the traitor; the lose prompt window for second/multiple times will use a skull to symbolise the player may lose the game.

The following feedback is about the interaction of the confirmation button:

Some of the interviewees mentioned that we should improve the confirmation button. This part needs to make it clear to users that the button can be clicked because the confirmation button is important to the interaction process and is essentially a switch that the player controls as the game goes further. So we will add some hovering effects to the confirmation button, such as deepening the button's colour and slightly reducing the size of the button while users hover around the button.

Summary

During this sprint, the bulk of functions' design and interaction methods are in anticipation. (such as the logic of win/lose conditions and the transition for the next task of saving the Atlantis continent. However, several design aspects could be improved. Eventually, the following issues above have been optimised and addressed before their delivery to developers during this sprint:

1. The designer has made some changes to the lose prompt window according to the different numbers of wrong selections to indicate to player the mistake number they have selected the wrong traitor and the current situation of the game.

2. By adding images of the figure as a design element outside the window frame, the design of the win/lose window is further integrated with the storyline so that the player can better understand the game interaction.

3. Add a hover effect to the Confirm button to improve the interaction process of the win/lose prompt window

4. By adding a brief transition interface with NPCs after selecting the traitor to implement the transition of select result and win condition.

Future Testing/Evaluation

The user testing for the upcoming sprint will be based on the knowledge obtained from the current sprint's tests and feedback. We have planned the future testing and evaluation that may be required to verify that the team's efforts are aligned with what must be done in the next sprint.

(Note: Depending on the task's progression and studio cooperation, the precise features that may be done must also be determined.)

1. Communicate with relevant Teams (like team 3 who are responsible for the game ending) that may collaborate in order to share ideas and strategies.
2. Consider the implementation's practicality and integration's difficulty via discussion and brainstorming.
3. Determine if the display of the win/lose condition meets current user expectations in next sprint (e.g., Whether the transition animation/interface of popping up win prompt window and next stage in the eviction menu can effectively help players transition to the next mission to save the continent of Atlantis; Whether it can make the eviction process more visible etc.).
4. Discuss with the appropriate teams and determine whether adjustments are necessary.
5. Invite studio members and other students (friends or potential consumers) to review the design, interaction flow, and functionality of the product.