I find that I enjoy games on touch screens so much more because I find the whole idea of moving forward with W, going to the side with A and D and backwards with S just not very normal for me. . . . It’s just so nice to be able to move your finger along the screen and jump. . . . Even though the games might not be as advanced, like Minecraft doesn’t have all the elements to it, it’s just so much easier to do, and I find I enjoy games more. (Eileen from Adelaide)
As Eileen notes, being able to touch the screen directly makes gameplay “feel” more natural and enjoyable. Game designers have long understood the importance of touch and how it can be synchronized with the other senses to achieve a sense of realism. As Brendan Keogh observes, playing a digital game involves multisensorial dimensions: “Through an entanglement of eyes-at-screens, ears-at-speakers, and muscles-against-interfaces players perceive videogames as worlds consisting of objects and actors with texture, significance, and weight.” As a way to capture these sensory dimensions, we asked participants to reenact their practices, and they often intuitively mimed their practice and sensory knowing through hand gestures and body movements.
This technique, drawing on tactile digital ethnography, aimed to capture what the body remembers in and around screen practice. In the reenactment process, people reflected on their favorite games and apps as they played or used them where and how they would usually do so (e.g., in their bedroom or family room). The role of the remembering body is an important aspect of media use, especially in terms of how this memory is often tacit and sedimented through repeated action.
In this process, it became clear how the mimetic aspect of mobile games is core to the sensory pleasure experienced. An overwhelming number of games and apps available on mobile phones and tablets are mimetic to some degree; that is, they imitate or simulate real-world experience through the gesture of the touching finger (e.g., moving a block or slingshotting a bird). Ideally, mimetic games are designed so that they can be used immediately and intuitively rather than requiring instruction; they rely on experiences with which we are already familiar (pushing objects, hitting a ball, or using a slingshot). As Eileen from Adelaide observed, touch screens are intuitive and don’t require a prior understanding or knowledge of games, so players don’t have to spend time acquiring the skills to play:
We’re quite early risers, and we do things quite randomly, but we would usually get up, watch TV, and play on our iPods at the same time. . . . You can do anything, and you can still be playing on your iPod. I’m someone that likes to be doing something all the time, and I’ve noticed that I can just kind of be there flicking my finger around . . . playing a game while watching television. I find it relaxing because usually the touch screen games, they’re very basic, and you don’t have to be a genius to work them out, and I just find them flexible.
Haptic touch screens have what is called a post-WIMP interface. WIMP stands for windows, icons, menus, and pointing device, the standard tools for navigating and controlling a computer interface. Unlike WIMP-based interfaces such as the standard desktop computer, post-WIMP interfaces like touch screens invoke our bodily and visceral understanding of naive physics. For example, primary bodily sensations such as inertia and springiness can be found in many touch screen applications and games, providing the illusion that objects on the screen have mass and are affected by gravity. Naive physics can also include our body memory of hardware such as the keyboard and joystick that are simulated in many mobile games. This haptic intimacy or closeness to actual bodily experience is what makes the touch screen a device of tactile and kinesthetic familiarity.
Mimetic games on the touch screen therefore work to enfold the player in a temporary and incomplete simulation of real-world physics. The pleasure and intimacy of mimetic play were expressed by twenty-four-year-old Anna as she showed us her favorite game, Tengami, describing as she did so the satisfaction of folding and unfolding paper to reveal worlds within worlds, each with its own puzzle challenge. She said, “It’s like doing origami; I find it very relaxing and creative, but also exciting: you never know what you’ll find under the next fold.” As twenty-one-year-old Jamie from Perth played her favorite iPad game, Angry Birds 2, she described the joy of destruction and pig popping— similar to the satisfying effects of “finger-bombing” common to many mobile games—and explained how this pleasure was in part due to the way the game mimics real-time physics.
Another thirty-five-year-old stay-at-home mum, also an Angry Birds fan, reflected on her previous addiction to the Ragdoll Blaster series (physics puzzlers that require the player to fire attenuated rag dolls out of a cannon at a bull’s-eye). In particular, she enjoyed the way the rag dolls moved “like real ones,” and she laughed as she lifted and flopped her arms about in mimicry. When people talk about their mobile game play, irrespective of age and gender, they often reenact their experience as a form of body memory.
Devices that combine touch screen with accelerometer or position-recognition functionality, such as today’s smartphones and tablets, have required us to develop a new kind of “motion literacy.” This kinetic and motile learning works to overcome or adapt to the imprecise control we have over objects and actions on the screen. This is possible because of our ability to take on an “as-if” structure of embodiment—to react “as if” what we are experiencing is tangible and concrete. For example, when one plays Angry Birds, the kinetic experience of releasing an elastic band effectively becomes “condensed into the hand.” In part, this effect is achieved by kinesic natural mapping, where the body’s movement corresponds in an approximate (or as-if) way to on-screen action. The physical analogies enabled by touch screens are also accompanied by sound effects, which simulate (or stimulate) the noise of tactile feedback and increase the sense of “being-in” a discrete and tangible game world.
This microworldly experience, and the sensory pleasure and attachment it creates, was clearly described by Lucy, a forty-nine-year-old university lecturer, who told us of her ongoing obsession with Godus. Godus is an intricate world-builder that involves detailed sculpting of multiple layers of land, with each configuration unique to the player. Lucy explained how she played the game intensively for over a year, complementing her story with hand gestures that mimicked the on-screen action:
There was something about the way you could sculpt the land, and set your little workers to build or mine, that was really satisfying. . . . I would have the game open on my iPad all the time and visit my land ten times a day at least.
This bodily memory and knowledge in the hands was also called on when people did not remember the names of the mobile games they had played. Margaret, a thirty-three-year-old accountant from Brisbane, described how she played arcade-style match-three games on her phone while on public transport: “You know, those little puzzle games,” she said, raising her right hand, index finger stretched out, and waving horizontal and vertical lines in the air.
The pleasure of mimetic touch screen games is not just important at the level of sensory attachment to one’s device. The haptic and aural intimacy of mobile games also becomes part of our social and personal intimacies more generally. In reenacting her Tengami gameplay, Anna walked to her bedroom and closed the door (she lived in a shared house), explaining that she preferred to play the game alone when she needed to wind down, as she enjoyed the soundtrack, sound effects, and noncompetitive, exploratory, and self-paced nature of the game. In this way, her mobile gameplay became part of her management of alone time in the home. Jamie recalled how she would share Angry Birds gameplay with her partner in the evenings; this involved lying together on the couch or bed, passing her iPhone between them (the only rule being that the device had to be relinquished to the other if a “life” was lost). This ritual of closeness and being together was bound up in the simplicity and “swapability” of the game, and in the materiality and co-touchability of the interface.
We witnessed this kind of screen sharing often, particularly in the family use of designated phones, iPads, and tablets, and their deliberate placement in common areas of the house—practices that challenge the assumption that mobile screens are deeply private and personal. Sharing was a frequent practice particularly among siblings; while one child would play, others would watch and make suggestions, and then there would generally be tacit agreement (or sometimes more fractious negotiation) to swap roles. Charlotte, a parent from Sydney, spoke about the pleasures of observing her children play on their iPad:
It brings us together. . . . We’ve never had a negative view about things like gaming and screen time and rubbish like that. We’ve always been very positive about it. . . . I actually enjoy watching them play, even mobile games. . . . We’ll lay down in bed together, and I’ll watch them play. So it’s drawn us all closer, and it’s just another thing we can all share, like an interest in anything. And it’s not just in our home; it’s also when we’re out and about.
Charlotte noted, however, that sharing was not an option for her older teenage daughter:
I will say this, though, my seventeen-year-old, she will not allow anyone [to touch her iPad]. She loves her iPad so much. She will not allow it to leave the house. That’s her thing. So it’s at home; it’s kind of funny because it’s a mobile device, and yet she won’t take it out of the house.
These stories highlight the intimate and communal ways that haptic mobile interfaces have become an integral part of our sensory and emotional experience of domestic life, our being-with-others (or not) in the home, and the way we manage domestic space.
Copyright © 2020 by Larissa Hjorth and Ingrid Richardson. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
