Unintended Outcomes of Technology Use: Lessons from Educators from Pokémon Go

Pokemon Go! Road Rage — Image Credit: Iain
This article was originally published on Jeremy Riel’s Medium blog

What happens when classroom technology isn’t used in the way you thought it would be? Or, alternatively, what happens when students use technology exactly the way you intended, but undesirable outcomes happened? These are two questions that every educator has asked at some point as they attempted to integrate new technologies into their classrooms. Using recent examples from the Pokémon Go phenomenon, this essay tackles these two questions with an eye toward how to adapt technologies for learning when they aren’t meeting our expectations.

Since its release in July, the increasing popularity of the Pokémon Go game has provided some casual tech observers (including myself) with a good opportunity to watch the different ways in which people use apps at scale. Altogether, I’d say the players have met the designers’ expectations on how the game should be played: people walk around to catch and evolve pokémon, players work together in social groups to challenge and defend gyms, and they log in to the app regularly to keep up with their pokémon training goals. However, any use of technology can always have unintended consequences when activities interact with the other things that people do in their lives. Even if we don’t realize it, we are often doing more than one thing at a time. As we use technology to facilitate an activity, it often interacts with the other things we are doing. Thus, this interaction of activities sometimes happens in negative ways.

There are some interesting early examples about the unintended outcomes of technology use that have been emerging from Pokémon Go as people go about playing the game. These lessons can be particularly valuable for teachers and instructional designers when making decisions about technology for learning.

Pokémon Go: A tale of unanticipated effects from tech-based activity

Whenever a new technology is implemented, it’s notoriously hard to predict exactly how it will be used (or if it will be used at all!). This is because each time something is used, it’s a brand new context: every time someone starts to work on a new project, there is a unique combination of people, goals, and constraints that will influence the behavior of participants. Similarities between circumstances can help tech observers and educators predict how it might be used and what behaviors will emerge, but we can never be exactly sue. So, the best course of action is to try to anticipate some of the expected behaviors and to continually adapt the function of tech to meet these goals.

I’m sure this is what has happened at Niantic Labs, the group that makes Pokémon Go. In the initial stages of app design, it is likely that they anticipated that they wanted people to use the app to walk around, get some exercise, and capture pokémon in real-world streets. They probably also anticipated the social elements of the game, intending for people to meet up at real locations and work together (or challenge them to a duel, if on an opposing team). We can assume these are some of the desired behaviors, as they are explicitly discussed in the training materials within the app. Although people are doing what the Pokémon game makers intended, there have been some unanticipated behaviors and outcomes that have occurred. These are the result of playing the game in a public setting in which movement and navigation are required.

When we talk about the “challenges” associated with Pokémon Go, it’s actually kind of a kudos to the designers of the game. Pokémon designers made their game too well: people who play aren’t taking their eyes off their phones! As a result, players have been walking into dangerous situations without looking: blindly crossing streets, walking off of ledges like lemmings, and even navigating directly into oncoming traffic to find a pokémon. The app is succeeding at keeping people’s attention, but often to the detriment players’ other activities. It’s in this case we see the interaction effects of competing activities. You must walk to play Pokémon Go, but it cannot be separated from the separate, but parallel activity of walking in a populated area with cars, other people, and objects that you can bonk into.

However, even staying aware of one’s surroundings hasn’t been enough for some players to stay safe and avoid conflict. Pokémon participants have been finding themselves in dangerous situations at no fault of their own, as criminals have been “luring” players to public pokémon locations and then mugging them at their arrival. Many of these crimes have been reported in the few short weeks the app has been online. Challenges have also been found on car-filled streets, as players have been urged not to drive and play simultaneously. The driving issue has been a significant problem, enough so that Niantic instituted a driving warning in its latest update to the Pokémon app. Almost comically, there’s even been a report of a Pokémon Go player driving right into a police car! These unanticipated effects from regular gameplay are serious and sometimes random, but they really only made themselves apparent after people started playing the game.

Additional aspects of play have been problematic from the perspective of non-player observers. Because Pokémon is a game that is played in public, it may also interfere with other social activities and rules in which the player is not participating. This has been the case particularly with places of solemnity and reflection, such as places of worship, monuments, and protected areas. In the first week of play, Arlington National Cemetery pleaded with players to respect the cemetery and avoid catching pokémon on its grounds. That same week, pokémon players were chased out of the U.S. Holocaust Museum for irreverently catching pokémon. Just this week, lawmakers have forwarded legislation that requires “pokémon stops” to be removed at request and that some places remain untouched, such as nature preserves.

Many applications of educational technology will never have to worry about criminal activity or walking headlong into oncoming traffic. However, these problems that have revealed themselves after play are great lessons for how something innocuous like pokémon play can influence other activities. These activity interactions are what we should attend to as educators when we think about bringing technology into learning. These early Pokémon Go challenges make a pretty poignant case for fixing unintended outcomes when they occur.

iPhone Case — Image Credit Nick Della Mora

Keeping an eye on tech use

Educators can take steps to monitor tech-supported activities and adapt tech strategies when undesired outcomes are observed. To start, it’s important to make sure any tech use is principled. We should use tech for a purpose or activity and avoid letting the tech determine the activity. The following four actions can help educators maintain the goals of the activities in which their students participate and to ensure that any technologies used to facilitate these activities are resulting in the desired outcomes.

Awareness of the balance of tech and activity

A good first question to ask for a learning activity is “what is the balance of activity?” It helps to make a mental note of all of the different things students will be doing in an activity. For example, when thinking of the Pokémon Go example, participants walk, talk to other people, navigate, tap on their phones, chase pokémon, and challenge rivals to battles. Most of these activities happen simultaneously, thus creating a balance of activities. Negative outcomes can emerge from this balance.

Imagining how the balance of activity could be disrupted is another useful question. Unfortunately, this requires those of us who implement technology to sometimes think like a bad guy. In today’s world, both outsider and student trolls and bullies constantly try to disrupt activity just for the fun of it. In addition to thinking about how participants could throw a wrench in learning activities and drive students off course, it’s useful to think about safety in all things digital. Cyberbullying is a particular negative aspect of today’s digital ed tech world that teachers need to consider, and the one you’ll most likely encounter. It is easier to maintain the balance of activity if you have an idea of the activity mix going on.

Brainstorming stuff people do

If you integrate a new tech, what will students do with the tech? What kinds of activities can they do with it? How do these tech-supported activities lead to learning outcomes? All of these questions touch on a necessary point of needing to anticipate some of the common ways that students may act using tech. Some helpful questions related to finding data to help you understand what students are doing include:

  1. What kinds of work or creative products will be done by students (e.g., writing, images, drawing, videos, in-person discussions, etc.)?
  2. What kinds of analytics or participation tracking options are available in the apps I’m going to use?
  3. How will students work together, or work with people outside of the classroom (e.g., parents)?
  4. How will I know when one type of activity is different than another type? In other words, do I have a list of the types of activities that students will do?

Having a running list of what actions to look for will help you make adaptation decisions later on down the road if it isn’t going according to plan.

Secondary app interactions — what are students using?

The tech-based activities that educators integrate into class work may also be amplified or squelched by the other apps, software, and hardware that students use in their everyday lives. To monitor activities that your students are doing and keep tabs on any undesired outcomes, it’s also valuable to consider the other apps that students are using. This is especially true if students are doing any out-of-class activities, such as field trips, homework, or blended learning apps.

The types of apps that students could commonly use that could interact with class activities include chat and messaging apps (e.g., Snapchat, Kik, WhatsApp), media sharing and remixing apps (e.g., Instagram, Vine), social networks (e.g., Edmodo, Facebook, Twitter), games (including Pokémon Go!), and personal productivity apps (e.g., Google Docs/Drive, todo lists). Educators can benefit if they poll their students every so often to see what apps they use and how they use them. Try to imagine how these apps may interact with what students are expected to do for class. Sometimes the negative aspects can be immediately apparent (such as with facilitation of negative feedback or an unproductive backchannel among students), but some behaviors may not emerge until after classroom activities are underway. As such, keep an eye on how students interact with classroom tech and what apps they use to help facilitate their work.

Monitoring and adapting

Educators cannot completely anticipate the different ways that classroom technology will be used, nor the many outcomes that can happen as a result of tech-based classroom activity. As such, it is important to continue to monitor and respond to changes in how students interact. Adapting is normal and expected. When technology is used in a class for the first time, it won’t work exactly to expectation every time at launch (and it likely never will). So, we are left to the same procedures that the best app and hardware developers of our day use: you watch, use your observations to identify patterns, and you change to meet the needs of learners.

If the Pokémon story is any lesson, it shows that even the best apps have experiences with planning only going so far. Complexity in interactions requires educators and software developers alike to stop managing for one specific outcome and monitor changes as they occur. This is especially true with the uniqueness of needs, learning styles, and interests of individual classrooms.

The best strategy with new tech implementation is to continually assess whether or not the tech-based activity is aligning with the goals that were set from the outset. If you didn’t have any goals in mind, take a step back and set some to prevent the tech from “driving” you and your students. It’s also helpful to find ways to see what other activities students are doing in their everyday lives and how these activities might additionally benefit student achievement of learning outcomes. This may require us to be imaginative, though, as the benefits and technology interactions may not be immediately apparent. As such, don’t be too hasty to shut down activity that is new or you don’t quite understand. New activities and combinations of technology-based activities might lead to incredible outcomes, so keep an open mind with an eye toward goals.

What do you think?

What are your thoughts and processes for monitoring and fixing unanticipated technology problems as they arise? Are there any tips for spotting challenges and negative outcomes that you’ve used in classrooms? — tweet me at @jeremyriel and let me know what’s on your mind. I’m also always on the lookout for cool apps and resources, so if you’d like to drop me a line about what you use related to this post, let me know.