Throughout Project 3, there were two playtesting sessions where people from all over campus could come and play the games we had made. In order to get the most information that we could from the playtester, every team had to come up with three questions to ask playtesters about their game.
The purpose of these questions is not to see whether they liked it or not or where there could be improvements, but to make sure that the game that we are creating is giving players the experience that we intend them to have.
As such, our questions need to be carefully tailored so that there aren’t subtle indicators that might influence the answers that we are given.
During our teams’ brainstorming we came up with questions such as:
- How difficult did you find balancing the head? (1-5)
- Were any parts of the game too frustrating?
- Does the game have a message?
The above three questions all contain subtle hints toward what we want the answer to be. The first two lead the playing into thinking that we want them to find the game hard and will likely skew their answers, perhaps unconsciously, towards suggesting that the game is hard. The third question leads people to believe that the games has a message just by virtue of bringing it to attention.
So we can’t use language similar to that, but we also can’t use a scaling system that you can see in the first question. The scaling system tells us very little. While it certainly tells us what they thought, it doesn’t tell us why they thought that way and gives us no indication as to whether the rating was a result of design or something else.
After culling a few more questions and having more discussion, we decided that the best way to go was to ask open-ended questions that required an answer that was at least a few words. Asking questions that can be answered with a yes or a no aren’t very helpful and questions that require a paragraph are too daunting to answer, so those types of questions aren’t viable either.
The initial three that the team chose were:
- Describe a moment when you nearly lost your balance but recovered.
- Describe a moment when you chose not to boast to someone.
- Describe a moment in the game where you failed on a ramp. How did that impact your strategy?
When asked what answers we wanted to receive from these questions. They didn’t hold up very well. The answers we gave were fairly mundane and didn’t really tell us much about the feel of the game. Our answers were:
- Second time I went up a ramp
- Before I went up a ramp
- I failed the ramp the first time I took, so I moved slower next time.
All these answers really tell us is that ramps exist and are difficult. Plus it is far too ramp-focused in a game about boasting. Also, the third question was a bit too leading, so the team reworked the questions into:
- Please describe a moment where you feel like you boasted too much.
- Please describe a moment when you nearly lost your balance but recovered.
- Please describe a moment in the game where you changed your strategy – why did you choose to change it?
These questions are far more open in terms of responses that the team believes they will receive.
When it came time to playtest, the team never changed questions between any of the playtesting sessions. The last two questions stayed because they made sure that the changes made to how the player moves and to the levels didn’t interfere with the experience that we were trying to create. However, in hindsight, the first question should have been changed because the team received everything from joke answers to confusion as to what boasting was.
The game, in the state it was in at the time, didn’t properly notify the player of who they could boast to or even the fact that they could boast at all. This means that some players missed that mechanic entirely, leaving the question obsolete.