Stephanie and Tyne recently spoke on a panel in a new series by the Centre for Artificial Intelligence and Digital Ethics (CAIDE) at the University of Melbourne entitled ‘Technology and the world around us.’ Alongside Dr Vanessa Bartlett and Dr Marc Cheong, we explored the theme of Art, Literature & Technology: Understanding Human Experience, Expression & Emotion in the Datafied Age. 

Here you can read Stephanie’s thoughts on the crucial role of Literature in helping us navigate pressing contemporary concerns around privacy, affect-recognition technology and human emotion.  

The face has long been considered a privileged site for conceptualising the human. Our faces help us articulate relationships with each other, with different forms of cultural, racial and ethnic alterity, with other species, and also, according to many philosophers, with the divine.

We observe faces all the time, assessing them according to changing cultural standards of beauty, normality or disfigurement, and through a series of complex cognitive, perceptual and increasingly technological modalities. The most obvious way of analysing facial emotion and expression is through visual images. Facial emotion in art, sculpture, cinema and other visual media is dramatic and immediate. Equally, in more self-conscious and analytic studies, in the long history of physiognomy, for example, images of human faces are consistently used to illustrate the differences between emotions. This history stretches from medieval and early modern manuals, to the drawings of Charles le Brun in the seventeenth century, and Johann Lavater in the eighteenth, to Charles Darwin’s treatise on emotions in the nineteenth century, and the influential work of psychologist Paul Ekman in the late twentieth century. 

In contrast, literary texts offer far more layered and indirect representations of the face, facial gesture and emotion. These representations are less immediate, but have the advantage of slowing us down to consider some of the cognitive, affective and cultural pathways by which we understand each other’s faces. Fiction and film are sometimes referred to as “empathy machines.” I think “empathy” is only one of many things these media offer, but the idea of literature as a kind of “machine”, with its own systems and patterns for showing and teaching us about emotions is an intriguing one. 

I’m going to share just one example that shows how literature both appeals to, and also teaches us about these patterns and processes of emotional understanding. It is a very famous example, from Jane Austen’s Persuasion, written in 1818. 

Several years before the main action of this novel begins, the heroine, Anne Elliot, was encouraged by her friends to reject the marriage proposal of a man who was not of the right wealth or class. Anne’s health and appearance suffer, but she eventually recovers her sense of self. In the novel, the former lovers are thrown together again by circumstance, and their relationship is strained. But one day they are out walking in a group and encounter a stranger, who is obviously struck by Anne’s appearance: 

It was evident that the gentleman, (completely a gentleman in manner) admired her exceedingly. Captain Wentworth looked round at her instantly in a way which shewed his noticing of it. He gave her a momentary glance, — a glance of brightness, which seemed to say, “That man is struck with you, — and even I, at this moment, see something like Anne Elliot again.” 

Austen does not describe faces here; Wentworth’s head moves, but Austen does not describe any specific movement of eyes or mouth. There is no obvious named emotion here, either. But it is a very powerful moment of recognition and acknowledgement through the medium of two glances. Wentworth “sees” Anne anew through the eyes of another man – and it is important that this man’s gaze is not lubricious or improper. The stranger’s class and bearing are completely gentlemanly, so his admiring look is validated by the narrator as worthy of our attention, and consonant with the cultural and class-based ethics of proper behaviour in the novel. 

Wentworth’s own quick glance has three components. First, it is a gift to Anne; something he gives her. As such it recalls their previous intimacy as well as opening up the possibility of renewing such intimacy. Second, it is a glance of brightness that we may imagine as lighting up his own face, as he also looks with renewed interest on her own. Third, and most importantly, Austen then deploys what I call the ‘speaking face’ trope. 

In this trope, a face is not only expressive; it seems to speak its own silent discourse. The face is given its own rhetoric and syntax. Wentworth’s face here speaks in a complex sentence that moves from what Wentworth seems to see in the other man’s expression, to what he sees as he looks anew at Anne, and the comparison with what seems to be his memory of “Anne Elliot”, the woman he used to love.

Three times Austen stresses how quickly this happens: it is instantaneous, it is “momentary,” and it is transient, “in this moment”. Yet narrative time slows down here, as this glance of brightness is analysed in detail. This slowness gives the readers time to work their way into the understanding of this expression and its significance. At the level of dramatic action, it is just a quick glance as the stranger passes; at a narrative level, action slows down to emphasise and reveal the workings of the gaze and the way a glance between two people can carry a tremendous weight of mutual recognition, understanding, and feeling. 

This trope is first used in English by Geoffrey Chaucer in the fourteenth century and has a long, though unwritten history, in English literature.

From what we have found so far, when the face speaks in this “as if”, or “seeming to say”, the face always speaks in the first person, and in the present tense. And it is always truthful. Faces can express all kinds of emotion in fiction; they can tell stories that are true or false. Faces may express or mask true feelings, but the rhetorical conventions of the speaking face suggest that facial discourse does not lie. This is not to say that all characters can read faces accurately: but readers of fiction can. Literature creates an artificial world, if you like, in which readers are interpellated as perfectly reliable face-readers. 

The rhetorical trope of the speaking face is also naturalised in everyday speech, in political commentary, the way we put captions on photographs of our pets and babies, and in popular song. But it is nevertheless a complex one that has been developed in tandem with the conventions of the novel, and one that also performs a pedagogical function. It is not a translation of a visual image; rather it seems to produce a visual image through its appeal to a common understanding of how a face might seem to speak, and more precisely, what a face that could be said to say “That man is struck with you… etc.” might look like. Equally, this trope also encourages the reader to feel how their own face might move if it expressed such a feeling, in what Guillemette Bolens calls kinesic knowledge. And in this romantic context, this trope encourages us to think what it might feel like to receive such a glance: in this case, as a gift from an old lover. 

This is a very different world from the simple equivalence of a static image of a face paired with a single emotion word: happiness, anger, etc. Austen’s is a particularly complex example of the speaking face, but it demonstrates how literary expression and the textures of language can produce layers of mediated intelligence that combine emotion, expression and the rhetoric of the face. Through that very simple tag — “seemed to say” — the reader’s emotions and embodied cognition are thus brought into play as full participants in the production of meaning. 

To read more about our panel or listen to the recording, head to the CAIDE event page here.