Stolen Valour: Playtesting Report

Game Description 1

Ingoing Concerns and Questions 2

Procedure 2

Participants 3

Results 3

Analysis 3

Changes 3

Game Description

Stolen Valour is a 2D platform fighter style boss fight. The ultimate goal of the player is to defeat the boss. To achieve this, the player will have to dodge the bosses attacks, while dishing out attacks of their own. The game has a few mechanics designed to assist the player with this task.

The player has three main types of basic attack. The combo attack forces the player to stay still while attacking, but deals big damage. The dash attack allows the player to move while dealing damage, but is not as strong as the combo attack. The jump attack allows the player to attack while airborne but, like the dash attack, is relatively weak.

Defensively, the player can use parries and dodges to avoid taking damage.  A parry gives the player brief invincibility, as well as a damage boost if they are attacked during this time. A dodge gives the players invincibility while moving quickly in a direction, but is limited by a timed cooldown period.

As the player progresses, they can unlock the laser attack and the combo meter. The combo meter fills when the player deals damage to the boss, and fills faster with more consecutive attacks. The laser costs some of the combo meter to use, but is powerful and allows the player to deal damage from a long range.

Compared to our alpha build, (but before our initial playtests), we added more animations and sound effects, to both the player character and the boss. We also added a tutorial screen that explains the controls and core mechanics to the player, before they fight the boss. Other changes include minor adjustments to the damage of both the player’s attacks, and the boss’s attacks, to further refine game balance.

Ongoing Concerns and Questions

With a game like this that relies heavily on the feel of controlling the character, our biggest concern was ensuring that the character is both satisfying and fun to control. This not only includes moving, but also the feel of attacking, dodging, parry, and even getting hit. We wanted to avoid situations in which the player feels as if they have failed at doing what they wanted to because the character didn’t respond in the way they believed they should.

Another priority was the tutorial to the game. At this stage, we had just implemented a very rudimentary tutorial, so we were curious to see how effective it would be. A good tutorial would introduce the player to all the mechanics in a way that doesn’t feel confusing. Players forgetting information that was outlined in the tutorial, or skipping past it without bothering to read, would be indicators that the tutorial needs improvement.

Game balance and difficulty was also on our minds. Considering the relatively short length of the game, we wanted the game to be fairly difficult, requiring multiple attempts to beat successfully. However, we also wanted the difficulty to be fair; we wanted the player to blame themself when they take damage, instead of the game. If players complain about difficulty and want to quit while playing, that can be interpreted as a sign that the game is difficult in an unfair manner. If the player is eager to get right back into the action after dying, that’s a sign that the difficulty is fair. If the player is able to beat the boss first try with little issue, it’s a sign that the difficulty is too low.

In terms of features, the combo system is probably the most complex part of the game. We want to ensure the player isn’t confused by it, and understands how to use it to succeed. We also want the player to feel as if the combo system is important to the game, and not just an add-on they should ignore. Similarly, we want the player to be incentivized to go for parries, as opposed to just moving out of the way of attacks.

Finally, we’re looking for feedback on the aesthetic choices of the game. Does the player like our environment design? How about our character designs? How about the music? These aspects are a big part of the gameplay experience and we want them to be pleasing, as well as complement the gameplay experience.

Procedure

Each playtesting session will be roughly formatted in the same way, with the exception of the Ubisoft playtesting session as time will be limited and we want to give them freedom to give feedback as they see fit.

Before playing, each playtester will fill out a pre-survey, allowing us together some information about their demographic, skill level, and difficulty preferences. We will be able to reference this information when evaluating their later feedback, in order to better contextualize it. A player may say the game is too difficult, but if they list a preference for easy games, that feedback will mean less than if it were to come from someone who likes hard games. 

After completing the pre-survey, the player will be given some time with the game. Each player will have a minimum of 10 minutes, or a maximum of 20 minutes, with the game, although they will be able to stop when they like at any time between those periods. An observer from our team will be watching them play, taking notes and guiding them when appropriate. It’s important to note that guidance was given sparingly: we wanted to give the players the chance to figure things out organically, but if it was clear they were lost we opted to give guidance, in order to save time.

Throughout the session, the playtester will be encouraged to describe their thought processes as they experience the game, expressing the pleasures and frustrations of the game. At the end, the observer will ask them for their final thoughts of the game, as well as for any feedback they’d like to give directly. These will also be noted by the observer. An example of such note-taking can be seen here: https://docs.google.com/document/d/1Kaq2TWAzuHLgBF5FjLsp_43roZiMqA1UWzzpCKOvNiQ/edit?usp=sharing

Finally, the player will be asked to complete the post survey. The post survey consists primarily of scale rating and multiple choice questions, allowing us to more quantitatively analyze feedback. There are also some questions that provide a space for the playtester to give some final feedback that they may have not given verbally. The survey (both parts) can be found here: https://forms.gle/gHmta15W57DVXECh6

Participants

Our participants consisted of:

• Four people from the UofT Game Design Club session
• One person from Ubisoft (with three other individuals from Ubisoft giving additional input)
• Six people found by our various group members for additional testing

Each participant, except for the Ubisoft employee, took the pre-survey. Here are what the overall demographics looked like:

• Participants were generally in the 18-25 age range, with one outlier being 54.
• We had a healthy mix of casual and hardcore gamers: weeklies playtimes for participants were about evenly split between the four available choices
• The majority of our participants had at least some fighting game experience; only one of them reported none at all. However, only two of our participants reported more experience than simply having played a fighting game before. Overall, we would judge this group as relatively inexperienced with fighting games.
• On the contrary, every single one of our participants reported some experience with platformers, with about half having casual experience, and the other half being experienced or very experienced with the genre.
• The majority of our participants preferred moderately challenging to very challenging games, although there was one outlier who preferred easy games.

Results

• Our sliding scale questions allow us to get some averages on how our group of participants felt about certain mechanics and features about the game:
  • Participants were asked to rate “how easy was it to move around in-game” on a scale from 1-10. The average rating for this question was approx. 7.3.
    • Tangentially related to this question was the yes/no question “Did you enjoy controlling the character ORBITT?. 100% of our participants answered Yes to this question.
  • Participants were asked how they felt about the visual style and quality of the game, on a scale from 1-10. The average rating for this question was approx. 6.5.
  • Participants were asked to rate the boss’s difficulty on a scale from 1-5, with 5 being too hard, 1 being too easy, and 3 being the ideal amount of difficulty. The average rating for this question was approx. 3.44.
• We also had some two-answer multiple choice questions about certain aspects of the game. With these, we will examine which of the two answers received the most choices, and by how much.
  • For the question “Did the camera position/movement compliment the gameplay?”, 100% of participants answered “Yes”
  • As mentioned earlier, 100% of participants answered “Yes” to the question Did you enjoy controlling the character ORBITT?
  • For the question “Were the combo meter and laser ability useful to you while playing?”, 66.6% of participants answered “No”, while 33.3%  answered “Yes”.
  • For the question “How did you feel about the music and sounds inside the game?”, 88.9% answered “Fun, adds feedback, immersive”, while 11.1% answered “Boring, annoying, out of sync”
• We also had some questions in the post-survey which allowed the participants to give worded answers and direct feedback. Additionally, our observers took notes while the participants were playing, as well as whatever feedback they gave verbally. From these responses, we took note of some overarching trends, compliments, and complaints. Some of these include:
  • Participants did not spend much time reading the tutorial, and often were confused or had to inquire about mechanics that the tutorial was intended to teach them
  • The purpose of the combo meter was not clear to many participants
  • Players were often running directly into the boss and dying as a result
  • Most players ignored the parry mechanic entirely.
  • Players often were locked into an attack, and unable to dodge out of the way of the boss’s charging attack due to this.
  • Every player reported having fun or enjoyment in some capacity
  • Players were amused by ORBITT’s attacks and their sound effects
  • Players responded positively to the sound and music in the game, although the volume was too much for some.
  • Players often expressed frustration when running into the invisible boundaries of the battle arena
  • No player expressed frustration to the point in which they wanted to stop playing early.
  • Players would occasionally get hit in ways that did not feel fair to them. For example, they would try to jump over the boss and get hit anyways, even though visually it seemed like they shouldn’t.

Analysis

In this analysis we will mainly be focusing on the flaws that needed to be fixed. However, it is also important to understand what parts of the game are successful, in order to not make destructive changes.

• All of our players reported satisfaction with the camera; this is a good indicator that the camera won’t need any future adjustment.
• During observations, all three attacks were observed to be used by players. Furthermore, we could see the beginnings of differing playstyles between players; players would focus on using different attacks from one another, as well as focus on dodging rather than parrying, or vice versa. This is something we like to see, as diversity in gameplay is something we want to promote with our design.
• 100% of players enjoyed controlling ORBITT. While there were issues with the game-feel that we will analyze later, it’s reassuring to know that every player enjoyed controlling the main character, as tight controls are a crucial feature to these types of games.
• Players overwhelmingly enjoyed the sound and music in the game. However, in hindsight, we acknowledge that the question did not offer enough choices for participants to give a nuanced feedback on them, and perhaps some aspects could be improved. Regardless, a 89% general approval rate is a good sign.
• Players gave out graphics a 6.5 out of 10. While this is merely an ok score, we are actually fairly satisfied with it, given the time constraints. Furthermore, many participants expressed that they liked the general visual style and animation style of the game. We also received some criticism about the UI, which would also relate to this, and would thus also bring the score down. Due to this feedback, we believe that our time would better be spent addressing the other, more pressing issues with the game, rather than any sort of rehaul on the general graphic style.

The most pressing issue we noticed was the tutorial. To be blunt, it was simply ineffective at what it needed to do. Players would either skip it entirely, or read it and not retain important information, leaving them confused later. In order for players to properly enjoy the game, we need to improve the tutorial in such a way that players won’t want to skip it, and will remember the information that we want them to remember

The other recurring and pertinent issue was the combo meter. Players were routinely confused by exactly how it functioned. They were able to connect that attacking means it goes up, but past that it’s use was unintuitive. Players reported being confused by it, and often just ignored it and continued playing as if it, and the laser, did not exist. Furthermore, the survey indicated that 66% of our playerbase did not find the combo meter and laser mechanic useful, a result that'd ideally be 0%. Of course, this issue is partially related to our ineffective tutorial, but it’s also an issue relating to the UI of the combo meter; it’s confusing and intimidating to the player. In essence, the ideal solution here seems to be to simplify this mechanic a bit, so players aren’t so intimidated by it. Along with this, we can improve the UI relating to it to be more intuitive.

Our core gameplay received a respectable score of 7.3 out of 10. This means that a lot of what we are doing is good, but there are some kinks to iron out. Once such kink would be some wonky hitboxes that players encountered. Specifically, the hitbox of the boss was larger than appeared visually, and this would cause players to be hit in unfair ways. Players also did not seem to register taking damage as well as expected. Some players commented on this explicitly, while others displayed this issue purely through gameplay. These issues would be good to fix in order to improve our gameplay further.

We noticed many players didn’t seem to want to use the parry mechanic, or didn’t understand it’s purpose. We believe that part of this is tied to the aforementioned confusion regarding the combo meter, but it’s also because the benefit of parry is too weak. We believe that parry should be buffed to make it a worthwhile option for players.

We received a 3.44 for our difficulty. On this scale, we would actually prefer to see a 3, as this would indicate the difficulty is present, but fair. Furthermore, according to our pre-survey, out playerbase skewed toward preferring difficult games. If this audience is finding the game too hard, it indicates that the difficulty of the game should be reduced. We noticed two main points of difficulty which resulted in players taking damage; the boss’s charging attack, and players running into the boss while trying to attack him. With the charging attack, players often were unable to react to his windup and dodge out of the way in time while they were attacking, resulting in damage that was perceived to be unfair. This type of difficulty is not desirable and must be addressed. With the issue of players running into the boss and taking damage, it seemed as if players were being punished for the intuitive behaviour of running as the boss and attacking. This also caused frustration, and is something we need to handle. Addressing both of these should reduce the overall difficulty of the game

Changes

It was immediately apparent to us that the tutorial needed a serious overhaul. Because of this, we opted to overhaul it mid-playtesting period, in order to get feedback on whatever design we tried. We first tried splitting it up into various screens, instead of shoving all the information on a single screen. Since players are easily intimate by large blocks of text, we hoped that this would improve readability. This also allowed us to add visuals alongside our explanations.

Unfortunately, while this did improve player understanding of the game somewhat, it did not address the issue enough to our liking. Thus, we decided to do a second overhaul, incorporating gameplay into the tutorial directly. We added a brief gameplay challenge the player must complete after every tutorial section, to show that they understand the mechanic being explained. This worked a lot better; players introduced using this tutorial showed quicker understanding of the game’s mechanics than previous playtesters.

The combo system also received an overhaul. We opted to simplify it, in order to make it more understandable. The complexity of the previous combo system was simply not necessary for the game. Now, the combo meter only fills to 500, which is the cost of the laser. Previously it filled to 2000, but since we scaled back the game to not have additional specials that use it, this was unnecessary. The location on-screen was also moved near the health bar, with the “meter” being represented by the background of the player’s icon filling up. This is a much less cluttered and intimidating way to display the status of it. We also delayed the point at which the combo meter was introduced. By adding a “drop” halfway through the fight that unlocks the laser and combo meter, we prevent the player from having to learn too many mechanics too quickly. They can win the first phase using what they learned in the first tutorial, and then add the combo meter and laser onto that in the second phase. By proxy, this is also an improvement to the tutorial, as it reduces the amount of information we need to give in that initial tutorial.

We addressed some of the general gameplay and difficulty concerns with simple fixes. The boss’s hitboxes were adjusted to be slightly smaller than his model, allowing for a better player experience and less “unfair” hit detection. We also slimmed down the hitbox of the thick lasers on his new second phase attacks, since some players also complained about the hit detection on those. The big change we made was to completely drop the active hurtbox on the boss at all times; this means he is only able to do damage during his attacks. The difficulty this added to the game was not perceived as fair or fun, and this change also allows the player to do fun things, like jump on the boss's head. In order to make taking damage more apparent, we added a knockback effect to the player when they take damage. This makes it immediately apparent to the player that they have made a mistake. Players were missing the colour change that indicated damage in the original build, but there is no way they miss this knockback. Finally, we added some more windup time to the boss’s charge attack. Before, players would be attacking, then be stuck in the attacking animation before they can move out of the wayo of this attack. Now the player always has enough time to react to the windup, stop attacking, and avoid the attack. All these changes will both reduce the difficulty in specific ways that felt unfair to players, while still retaining a respectable level of difficulty for the boss.

Finally, we reworked the parry mechanic in two major ways. First, we changed the effect of the parry from “increased combo meter” to just a raw damage increase. This effect is stronger, and better balances the risk-reward of this option. Second, we doubled the length of the parry effect. Since the boss often retreats from the player after being parried, this was necessary to make the parry effect last for more than one attack. We believe that the window of time that you have to parry is well balanced; it’s not easy, but consistently doable if you have the timing down. We also received no negative feedback about this specific aspect of parry, so we decided it did not need any changes.