Highlighting Invisible Media: Television Subtitles in the Split-Attention Economy

By Hannah Marcus

A striking demand of many theoretical texts which engage with the formal aspects of media is the importance of paying attention. For Raymond Williams, for example, Marshall McLuhan’s famous formulation  – “the medium is the message” had initial appeal ‘in his apparent attention to the specificity of media: the differences in quality between speech, print, radio, television and so on’ [italics mine] (127). For theorists such as Karen Lury, this is not specific enough; she wants to highlight ‘how important it is to pay attention to the formal aspects of television, not just for the way in which they can illustrate how programmes ‘make meaning’, but how they are, in themselves, meaningful’ (5). Conversely, Kember and Zylinska tried to collapse these formal differences, asserting that ‘media cannot be conceived as anything else than hybrids’ (7). Yet whether formal differences within media are broken out, as in Lury, or collapsed down, as in Kember and Zylinska, it cannot be ignored that in critical culture, in consumer culture, and in producer culture, some media are considered more worthy of attention than others.

This article explores what happens, when we pay attention to a medium which has often gone unnoticed, and indeed whose functionality is bound up in the question of going unnoticed: the television subtitle. In Reading Sound, Sean Zdenek states: ‘no one has really treated captioning as a significant variable in multi-modal analysis, on par with image, sound and, and video. No one has considered the possibility that captions might be as potent and meaningful as other kinds of text we study in the humanities’ (xiii). In this paper, I argue that subtitles as a medium are a perfect space to explore media centricity. In light of McLuhan’s assertion that ‘the “content” of a medium is always another medium’ (8), subtitles are not so much hybrid as ‘contingent’ – they deserve individual attention, and yet cannot be separated from their parent medium, whose content becomes their message. Furthermore, the recent changes in the use of subtitles, and the improvements to their formal qualities, serve to illustrate why the question of which media we pay attention to, and in what way, is still deeply relevant.

There is a growing body of qualitative evidence that suggests there is a change in the way Western, English-speaking audiences are interacting with subtitles. As Jason Kehe wrote in Wired: ‘Essential for deaf people and English language learners, and scientifically shown to promote reading comprehension and retention, subtitles have only recently become essential for many TV watchers.’ [1] One only has to turn to Reddit [2] to find reams of anecdotes which suggest that for a growing portion of people, watching television with subtitles is becoming a norm. However, the fact that more people are using subtitles does not necessarily mean they are paying more attention to them; in fact, it could be the exact opposite.

Building on McLuhan’s formulation of how to analyse media forms, the work of Jay Bolter and Richard Grusin (2000) and Neil Postman (2006) proves particularly helpful in unpacking  how and why this shift in attention is happening, and what it might mean.  Postman, a self-professed disciple of McLuhan, sits firmly in the formalist camp, announcing: ‘the decline of the Age of Typography and the ascendency of the Age of Television’ (8). Postman claims that ‘this change-over has dramatically and irreversibly shifted the content and meaning of public discourse, since two media so vastly different cannot accommodate the same ideas’ (8). Postman’s comment does not consider the subtitle in this argument, as  the form of a subtitle belies this, combining typography and television in service of the same message. In a different way, Lury also distinguishes a line between text and visual culture, arguing that this combination could indeed struggle to make such an accommodation. Speaking of forms of presentation which make use of text and image in conjunction, (specifically, here, news channels), she says: ‘if the situation is disorienting, it is because, as viewers, we are constantly being asked to fluctuate between these two perspectives or ways of interpreting information on-screen’ (165). However, in the twenty-first century, this fluctuation could now be said to be a consistent part of how content is consumed in a hyper-mediated world, which Bolter and Grusin defined as ‘our culture’s contradictory imperatives for immediacy and hypermediacy’ (5), for both total immersion and awareness of that immersion. Bolter and Grusin’s ‘multiplicity of windows’ (33) now extend beyond individual screens, and a person could theoretically be switching between three different types of content on laptop, television and phone simultaneously. In this context, the disorientation of moving between text and image within a single window on a single stream feels marginal; subtitles are remediated by the broader context of multiple competing attentions. Which medium are we truly paying attention to, at any given moment?

One could frame this split attention as a reification of Postman’s “Now…this” theory: ‘a means of acknowledging the fact that the world as mapped by sped-up electronic media has no order or meaning’ (99). In this sped-up world, arguably all media become as worthy of attention, and therefore as unworthy, as one another. Yet the split-attention environment suggests a new hierarchy by creating a need for subtitles above and beyond those who are not Deaf or hard-of-hearing.

When “changes in one’s abilities based on environment, device, or other temporary conditions” create situational disabilities for able-based people, closed captioning can step in to provide access for a wide range of viewers, regardless of hearing ability (Zdenek xii).

For when switching between different media, the temporal shifts of text on screen become not just familiar, but an advantage. The viewer can anticipate dialogue and then look away, or catch dialogue they might have just missed. If, as John Fiske argues, popular culture has become a ‘producerly text’, a text whose ‘writerly reading is not necessarily difficult, that does not challenge the reader to make sense out of it’, but which still contains the ‘openness’ of a writerly text’, then subtitles are the medium which facilitates this producerly potential (103-104).

Yet as tempting as it is to pin the increased attention of subtitles on this split-attention environment, we also should not ignore the implications of the formal changes to the medium. Because the truth is, up until recently, televisual ‘subtitles [were] almost always badly designed. Illegible typefaces drift on- and off-screen at the wrong moments, lurking so low that the bottoms of the letters are chopped off, and obstructing the audience’s view of gripping twists in the plot, or especially beautiful scenes.[3]’ An example, although less extreme, of the improvement in subtitle form can be seen in comparing the difference between seasons One and Two of Netlfix’s House of Cards (Willimon, 2013). In the first season, bold capitals splash across the bottom of the screen, regardless of what they’re obscuring.By the second, the subtitles are in small-caps and the font is sans-serif; clean and innocuous. Not only that, the subtitles would also occasionally move about the screen, so as not to obscure crucial action, or to align with those that were speaking, using a format known as ‘pop-on style’ (Zdenek 65). In other words, they have got better. This change signals a dramatic evolution in the medium, but it is difficult to determine whether this is a cause or an effect of the increased usage. One argument might be that now that subtitles are improving, people are starting to use them more widely, and in doing so are finding a solution to the problem of split attention. As Zdenek states: ‘when captioning supports universal design principles, it reminds us that “all technologies [are] assistive”’ (295). This view is one that Williams would define as ‘symptomatic technology’, where changes in a ‘complex of technologies’ function as symptom of change, neither pure cause or pure effect (13). In this case, paying attention to the medium and working to improve its functionality becomes a universal good, benefiting multiple user groups. In this development, we are reminded that the main goal of mediation is to communicate a message, and the improvement of subtitles is paradigmatic of this fact.

However, when considering that the controversy of all-caps in style guides was enough of a problem that Zdenek had almost despaired of a solution, (‘in the absence of any sustained opposition […] such practices will continue and may even be increasing today’ (25)), this symptomatic explanation feels unsatisfactory. If we turn to Williams, the reason that ‘despite the legal advances, new digital technologies, and growing awareness of disability issues, closed-caption TV hasn’t changed much since its debut in 1980’ becomes more definitively bound up in where the attention of television producers lies (Zdenek 24-25). Williams observes that:

A need which corresponds with the priorities of the real decision-making groups will, obviously, more quickly attract the investment of resources and the official permission, approval or encouragement on which a working technology […] depends (Williams 19).

In other words, when it is not a priority for the producers of television content to produce better television subtitles, the medium will get no attention and therefore will not develop. As Udo and Fels suggest, ‘the primary user is actually much more covert: the broadcaster and media producer, who use these services as a means of placating government requirements’ (22). After all, when surveyed by Zdenek, ‘given the alternative of not having any captions, participants were generally satisfied with the quality of closed captioning’ (70). The formal inadequacies of subtitles are by this logic a symptom of the lack of attention that is paid to the needs of the Deaf and hard-of-hearing by the producers of consumer culture. The fact that we are seeing an improvement in subtitles as a form might then be an example of what Jay Dolmage describes as ‘interest convergence’. Drawn from critical race theory, this suggests that ‘conditions change for minorities only when the changes can be seen (and promoted) as positive for the majority group as well’ (2005). Paying attention to a usually ignored medium can reveal a sub-message about whose needs are valued by our producerly culture.

Yet this concept is complicated by the idea of invisibility of subtitling forms. How can subtitles as a medium claim to be important when their most effective functioning is to be as invisible as possible whilst still being present – secondary to the message they are attempting to communicate? Max Deryagin, a designer of subtitles, rates subtitle quality based on how unnoticeable they are (2019). This echoes Bolter and Grusin: ‘the logic of immediacy dictates that the medium itself should disappear and leave us in the presence of the thing represented’ (5-6), whilst ‘designers of hypermediated forms ask us to take pleasure in the act of mediation’ (14). Subtitles complicate this, because when they as a medium ‘disappear’, they do so in order to leave us in the presence of another medium, the audio-visual medium of the television show that we are watching. For the pragmatic, interpretative function of subtitles, they can aid either immediacy or hypermediacy, because without them the Deaf or hard-of-hearing would not be able to immerse as effectively into what they are watching. ‘Immediacy depends on hypermediacy’ (Bolter and Grusin 6). This creates an interesting paradox: the more attention we pay to subtitles as a form in their own right, the less we should be paying attention to them when they perform a perfect function. When it comes to questions of accessibility, the medium has to be considered first, but only in service of reaching a space where it does not need to be considered at all.

Subtitles, then, can help illustrate what message is sent when we pay more attention to some media than others. At some point in the last four years, someone in media production started paying attention to subtitles as a form, as the move from all-caps to small-caps, so wished-for by Zdenek, signifies. However, the exact effects of this change are still being realised, and understanding of the implications of improvements in subtitle forms are complicated further by Williams’ discussion of the development of technology:

Original intention corresponds with the known or desired practices of a particular social group, and the pace and scale of development will be radically affected by that group’s specific intentions and its relative strength. Yet at many subsequent stages other social groups, sometimes with other intentions or at least with different scales of priority, will adopt and develop the technology, often with different purposes and effects (125).

We might wonder: are Netflix subtitles getting better because their algorithm is telling them that more people are watching with subtitles, or because of the settlement reached in a lawsuit filed by the National Association of the Deaf, where ‘Netflix agreed to caption 100% of its library by 2014’ (Zdenek 22)? By the logic of interest convergence, the slow development of subtitles as a form could be directly attributed to the relative lack of strength of Captioning Advocates speaking for the Deaf and hard-of-hearing – and it is only now that the mainstream consumer, due to the needs of a split attention economy, is ‘paying attention’ to subtitles that they are improving. Yet Williams’ discussion of the different priorities of an alternative group challenges Zdenek’s assumption around the positivity of universal design. Are the needs of those who use subtitles as a way of navigating multiple-media landscapes the same as the needs of those who struggle to hear? As subtitles continue to develop as a medium in their own right, it remains to be seen whether they will do so in a way which benefits those who need them the most, or begin to cater to a group with ‘different scales of priority.’ When ‘captioning is the difference between understanding and misunderstanding, inclusion and being left out’ (Zdenek 70) then worrying about the content of the message becomes secondary to considering who has access to it.

It is easy for disciples of McLuhan to deny the importance of the content of a television show when they have the choice of accessing that content with ease. Subtitles remind us that whilst all experience may be mediated, Williams’ challenge of ‘universal accessibility’ (151) being at stake is still relevant, because not all groups of people have the same access to the same messages. Williams, when predicting where television innovation would progress, states:

it is from this generation, raised on television, that we are continually getting examples and proposals of electronic creation and communication which are so different from orthodox televisions as to seem quite a new technology and cultural form (133)

If that was Postman’s ‘Age of the Television’, (whether positively or negatively framed), then I believe that we now exist in not the age of the computer, social media, or ‘the digital’ but ‘The Age of the Caption’, a space where image and text are becoming more and more intertwined by the many different digital forms. Whether the improvements to subtitles as a form are a cause or a symptom of this shift remains to be seen, as does how the medium will continue to develop. Will it be a net good, as Zdenek believes, with the principles of universal design meaning that mainstream consumers being more interested in watching with subtitles will benefit the groups for whom the need is greatest? Will it have the opposite effect, meaning the form of captioning develops more to benefit those with split attention, at the expense of the needs of the Deaf or hard-of-hearing? What might the effect be on globalised media distribution for translated subtitles? The answer lies in the next generation of producers, and who they think should be hearing their messages. Subtitles remind us that ‘all technologies [are] assistive’ and for many, still, the medium is purely what allows people to access the message. Until everyone has equal access to all media, and all messages, we should be paying attention to both.

Notes

[1] https://www.wired.com/story/closed-captions-everywhere

[2] https://www.reddit.com/r/movies/comments/84h87j/not_deaf_but_finding_it_hard_to_listen_to/

[3] https://www.nytimes.com/2007/05/25/style/25iht-design28.1.5866427.html?

Cite this article

Hannah Marcus. “Highlighting Invisible Media: Television Subtitles in the Split-Attention Economy” . Alluvium, 8.1 (2020): n. pag. Web. 5 May. https://doi.org/10.7766/alluvium.v8.1.02

About the author

Works cited

Bolter, Jay David, and Richard Grusin. Remediation: Understanding New Media. MIT Press, 2000. (London: MIT Press)

Deryagin, Max. “Subtitle Appearance Analysis. Part 1: The Font.” Max Deryagin’s Subtitling Studio, 2019, https://www.md-subs.com/saa-subtitle-font.

Dolmage, Jay. “Disability Studies Pedagogy, Usability and Universal Design.” Disability Studies Quarterly, vol. 25, no 4, 2005, http://dx.doi.org/10.18061/dsq.v25i4.627.

Fiske, John.  Understanding Popular Culture. Routledge, 1989.

Kehe, Jason. “The Real Reason You Use Closed Captions for Everything Now.” Wired, 26 Jun. 2018, https://www.wired.com/story/closed-captions-everywhere/.

Kember, S., and Zylinska J. Life After New Media: Mediation as a Vital Process. MIT Press, 2012.

Lury, Karen. Interpreting Television. Hodder Arnold, 2005.

McLuhan, Marshall. Understanding Media: The Extensions of Man. Routledge, 1964.

Postman, Neil.  Amusing Ourselves to Death: Public Discourse in the Age of Show Business (20^th Anniversary Edition). Penguin, 2006.

Rawsthorn, Alice. 2007. “The director Timur Bekmambetov turns film subtitling into an art.”  The New York Times Online, 27 May, 2007, https://www.nytimes.com/2007/05/25/style/25iht-design28.1.5866427.html?.

Udo, J.P., and D. I. Fels. 2010. “The Rogue Poster-Children of Universal Design:

Closed Captioning and Audio Description.” Journal of Engineering Design, vol 21, no

2–3, pp. 207–221.

Williams, Raymond. 1990. Television: Technology and Cultural Form. Routledge, 1990.

Willimon, Beau, creator. House of Cards. Netflix, 2013-2019.

Zdenek, Sean. Reading Sounds: Closed-captioned media and popular culture. University of Chicago Press, 2015.