by Greg Reinmuth, Lead Designer on NumberShire
Let’s talk shop about motivation in educational software. Thought Cycle was founded by a group of gaming industry veterans. We partner with University of Oregon’s Center on Teaching & Learning to effectively teach core math skills to grade school students. As gaming industry vets, we are acutely aware of the need for properly motivating our players. In traditional gaming, the best method is to make something that’s really fun to play. Beyond that, there are all sorts of tricks to engage gamers. This includes things like achievements, high score boards, new levels, and unlockable outfits or gear.
When we began work on NumberShire, we were really excited about the chance to approach educational software with a vision that combined the expertise of professional teachers with what we have learned over the years as game developers. As the lead designer on the project, I want to take you on a tour of our thought process behind the player incentives that we built in to NumberShire. I’m eager to share our ideas because I feel it is important to have an ongoing discussion about how to engage kids in a way that gets them excited about learning math!
Our Students
The first thing a designer needs to do when developing a game is to determine what sort of audience the game will have. For NumberShire, we all agreed that we would have a lot of different types of students. Some would be interested in math from the onset, while others would not. Some would love being on the computer, while others would have little interest in it, or be very unfamiliar with the basics of computer use. Some would lose interest quickly without some external reward, while others would enjoy learning new math just for the sake of learning. The huge variety of students we knew we’d encounter was one of our greatest challenges.
Core Goal: Intrinsic Motivation
With this wide variety of possible students playing our game, we felt that we had to tread carefully. Our primary goal was to make the math in the game itself fun and rewarding. If a student was asked to add 4 + 4, we wanted them to feel that there was a reason for doing it successfully. Intrinsic motivation was our core goal.
To achieve this, we decided to make NumberShire feel like a real place, a town that had all sorts of math problems that needed to be solved. This seemed like a better approach than simply asking students to do isolated problems on a virtual chalkboard, which was what a lot of other educational math games were doing. By tying math activities to real events in the town, we could give students the feeling that what they were doing mattered by showing the results of their hard work. Even better, because we could demonstrate a “real world” example of a problem, students could learn WHY it was important to learn the math we were trying to teach.
During our first pass at setting this up, we debated about what to do when a student failed an activity. The curriculum experts at CTL suggested a “differentiated pathway” for students who were struggling. As the student plays, the game analyzes their performance and if they fall below a certain threshold, it triggers the differentiated pathway. Depending on the concept they are struggling with, the differentiated pathway either helps to shore up their knowledge with a simpler version of the lesson, or it revisits an earlier lesson to help rebuild their confidence. The lesson is then returned to the next day.
On the game side, we also initially tried tying success to in-game results. Successfully solving those problems improved the town, whereas failure often resulted in someone else in the town having to step in and do the work off screen. This turned out to be a mistake which we discovered during testing.
Initial Testing
We built a single day of content with these things in mind. During it, we had the student build a puppet theater for a play. Thatcher Tom, the resident builder in the town of NumberShire, needed help figuring out how many wood blocks he needed to make the theater. We built the scenario and took it to classrooms. Most kids responded positively to being asked to do the task. Overall, they were very motivated even when it got challenging. But we didn't have total engagement. Some kids were overwhelmed. Some were really sad when they failed. Some were disengaged.
Adjustments
After the initial testing, we went to the drawing board to discuss our next step. We felt that overall, our product was doing the right thing. Most kids enjoyed being asked to actively participate in the world we’d created. Our concern shifted to the challenge of bringing the outliers into the fold.
The challenged students
We felt that our attempt to treat failure negatively was a mistake. We didn't want our students to fail, but if they did, we certainly didn't want the game to rub it in their face. After many discussions about how to reframe this, we decided that we needed to adjust the tone of the game. The most important thing a student can do is to earnestly TRY to do math. If they fail but they’re trying, the game stays positive.
The disengaged students
We noticed that some students were just trying to get through each math activity to see what would happen next in the plot. They would answer as quickly as possible and were often getting incorrect answers, but they didn't care because their behavior was getting them through the parts they weren't interested in. We labelled that approach “spamming,” but initially our attempts to fix the problem by analyzing player input rates or correctness tended to also catch well-intentioned students in our net. We didn't want to punish kids who were earnestly trying. Eventually the approach that we settled on was to introduce greater levels of explanation for incorrect answers. This had the positive effect of slowing spammers down, while also increasing the instructions available for challenged students. Overall, a win on both fronts.
Moving Forward
We wanted to give an extra incentive to students in the outlier group. A large discussion arose from that. The primary concern was that if we introduced external motivating factors to bring in the outliers, we would push previously internally motivated kids to being externally motivated kids. We didn't want an entire classroom of kids who refused to work without an extrinsic reward! We decided that we would implement a rewards system and then carefully watch the results to see if the trade off was worth it. We tried to make the rewards passive so it didn't interrupt the rest of the game, but still felt personalized. We offered a choice of different pets, outfits, and art that the student could choose between at the end of each day.
We also added mini-games to the mix. Occasionally (but not every day) we have games that pop up for the students to play. They’re always placed in context with the math lesson of the day. For example, during a day when the student is taught strategies for adding 1 (“Adding one is just like saying the next number on the number line!”) we had the students collect a marble from each character. When they’re finished, they get to play a quick game of marbles. This helps add to the feeling that they've accomplishing something in the world because they get to enjoy the fruits of their labors.
In subsequent testing, we saw even fewer disengaged students, and the challenged students seemed to be getting further along without getting discouraged. Meanwhile, the overall response to the new mini-games and rewards, (especially the pets) was extremely positive. We did not see examples of students losing the will to continue because their reward expectations were not met.
Slower Play
Since we designed NumberShire so it can be used as an in-class intervention, one of the issues that we ran into was the huge disparity in student speed and the impact on class management. We talked about giving the students who had finished early something to do rather than move them to the next instructional session, and while we thought it was important, it needed to be fun but still focused. So rather than offer mini-games at the end of the lesson, the decision was made to keep the activities limited to aesthetic choices. The student could look through their closet and change their outfit, or they could pick a different pet. We also let them continue to work on math if they wanted to. This seemed to solve the problem of students finishing ahead of their classmates while keeping the whole class on intervention group at the same pace.
Success!
As a result of the decisions we made, with great feedback and guidance from teachers, we arrived at a place that seems to have maximized student engagement. We’re always discussing different ideas and refining our current methods as we push forward with kindergarten and second grade versions of NumberShire. For example, in the kindergarten version we've added knock knock jokes told by some of our characters because we noticed that kids love recounting some of the jokes we had in our first grade version. So to close out, I’ll just leave you with my favorite knock knock joke:
Knock Knock?
Who’s there?
Interrupting Cow.
Interrupting c…
MOOOOOOOOO!
No comments:
Post a Comment