Multimodalität im Schnittbereich von Medientheorie und Semiotik

Abstract

Most media theoretical approaches have paid relatively little attention to the needs and requirements of multimodality. At the same time, the most widespread theo­ries of multimodality have just as little dealt with the concept of media. As a consequence, media-theoretical positions on the phenomenon of multimodality as well as multimodal positions on the concept of media are weakened. In this article, we suggest that this situation hinders systematic empirical analysis of complex multimodal artefacts and actions and introduce a concept of media combining both multimodal-semiotic (Bateman 2013, 2016) and media or image theoretical foundations (Sachs-Hombach 2013). This foundational framework builds on our previous empirical analyses of multimodal pheno­mena in film, comics, graphic design, images and diagrams in order to articulate mutu­ally productive relationships between media, semiotic modes and genres. All three vari­ables are seen as essential for effective analysis: Media bring socio-cultural and insti­tutional aspects into play that establish „biotopes for semiosis“ (after Winkler 2008: 213); semiotic modes trace the shaping of materials so that they can carry communicative actions; while genres allow for descriptions of communicative goals ranging across media and semiotic modes (Bateman 2014a). We argue that a clear separation of the theore­tical and analytical responsibilities of media, semiotic modes and genres establishes a number of methodological principles that facilitate the concrete handling of complex mul­timodal situations as well as supporting more precise analyses of transmedial actions and artefacts (cf. Bateman et al. 2017 or Wildfeuer et al. 2020).