ABSTRACT
The nexus of comics and games evidences an ecosystem of interplay and multilateral experimentation of porous medial boundaries. As part of an ongoing project driven by sociosemiotic multimodality and affective sciences, this chapter applies digital annotation and gameplay analysis to examine how digital games transfect aesthetics of various audiovisual narratives. I examine such illustrative examples as the second digital graphic novel of the genre-defining Japanese stealth game series Metal Gear Solid (1998–2015), the comic-game Gorogoa (2017), and the gesture-based literary art Breathing Room (2014). My focus is on analyzing how movement of in-text semiotic resources (e.g., graphics, linguistic texts, panels) and kinetics intersect imaginaries of human-media interfaces, indicating the rising influence of game design on digital interfaces and users’ temporal-spatial perceptions.
Specifically, the chapter problematizes hybridity, a common denominator of digitalized media that has arguably attracted insufficient empirical scrutiny. I contend that a multifaceted approach anchored by fine-grained multimodal analyses appropriately engages with the dynamics among intermediality/transmedial storytelling, theory/empiricism, experience/design to build a foundation for further affect-focused investigations. Combining empirical work and theories of digital media studies and multimodality, this chapter emphasizes the need for interdisciplinary research to deliberate the mutual malleability of comics and games, exemplary of evolving contemporary semiotic practices.
