Social Interaction. Video-Based Studies of Human Sociality.

2023 Vol. 6, Issue 2

ISBN: 2446-3620

DOI: 10.7146/si.v6i2.134841

Social Interaction

Video-Based Studies of Human Sociality


Transformations of Disgust in Interaction:
The Intertwinement of Face, Sound and the Body


Sally Wiggins & Leelo Keevallik

Linköping University

Abstract

Expressions of disgust have typically been studied as isolated faces or voices but rarely as embodied practices in everyday interaction. Building on multimodal interactional research on emotions and sensoriality, this paper addresses disgust as a unique topic at the intersection between psychological theory and interactional facts. A case of an adult enacting post-consumption disgust is analysed, detailing the transformation of the facial, auditory, and embodied expressions across interactional sequences and in collaboration with others. The paper showcases the variability of disgust expressions and their involvement in social actions such as displaying stamina or stoicism toward challenges.

Keywords: disgust, embodiment, facial, multimodal, vocal

1. Introduction

Despite its long history, most research into disgust expressions in humans has been curiously limited to a focus on individual facial expressions removed from the surrounding social context (Ekman & Friesen, 1978). This has led to several theoretical and methodological restrictions: the heavy reliance on faces as being the primary locus of emotional expression, the separation of facial and vocal components, and the almost universal use of individual participants in experimental designs. That disgust might be dynamic – performed and understood through movement and change – rather than expressed purely in static faces has only recently been highlighted as a methodological concern (Goodwin et al., 2012). Moreover, a predominant amount of research has focused on the face to the neglect of other constitutive and contextual features (Abramson et al., 2021; Farley, 2021; Israelashvili et al., 2019; Lecker et al., 2017; Reschke et al., 2018), such as the fact that disgust is also enacted for others. In short, too much attention has been placed on disgust as an individual emotion assumedly displayed in a static facial expression. When one considers the breadth of theoretical work on disgust and the implications it has for cultural, health, moral, and social practices (Curtis, 2011; Knapp, 2003; McNally, 2002; Miller, 1997), work on disgust expressions has a long way to go to capture some of these neglected aspects.

This paper draws together psychological and interactional research to showcase the potential of video analyses of social interaction for scholarship on disgust. Building on recent developments in multimodal interaction research, we offer an analysis of a series of nonlexical vocalisations of disgust that demonstrates how they are produced in relation to the ongoing social interaction (see also Goodwin et al., 2012). We argue that more attention needs to be focused on how disgust is enacted through a combination of faces, sounds, and bodily gestures, and that it needs to be understood in terms of its local interactional relevance and consequences. While often treated in both psychological and interactional research as one of a range of emotional expressions, it is important to dissect the specific features of disgust, which offer a unique opportunity to target the interface of the sensorial capacities of the human body and its interactional affordances, alongside the materiality of ingestible substances. Before detailing how disgust can be approached from an interactional perspective, we first discuss some notable findings on facial expressions and sounds of disgust to sketch out the methodological challenges that arise from this research field.

2. Facial Expressions of Disgust

While Darwin's writings on disgust (Darwin, 1872 [1965]) included a range of bodily movements, the face has been almost exclusively the focal point for research on disgust expressions over the past fifty years (Farley, 2021). Since the development of Ekman and Friesen's Facial Action Coding System (FACS) (Ekman & Friesen, 1978), research has typically examined the facial production of disgust or the recognition of disgust in the faces of other people (e.g., Rozin et al., 1994). This research area usually asks people to examine photographs of faces representing disgust, and thus a static version of facial gesture has been favoured, exhibiting certain facial gestures that have become recognisable as the 'disgust face'. This includes the core features of a nose-wrinkle (AU09, 'Action Unit 09' on the FACS) and lip-curl or upper-lip raiser (AU10), and optional mouth gape with or without a tongue extension (Cohn et al., 2007). While the FACS has provided a standardised scheme through which facial gestures can be coded, it has been criticised for oversimplifying the varieties of disgust and their elicitors (Fridlund, 2017; Stevenson et al., 2019).

The disgust face is also problematic in that it takes no account of movement. There is a concern that the use of static faces without additional bodily or situational context leads not only to limited ecological validity but also a constraint on empirical findings (Trautmann et al., 2009). Instead, 'dynamic' faces–video-recordings of moving faces as people express disgust–have been proposed as a more effective way to elicit recognition (Van Der Schalk et al., 2011). For instance, a review by Torro-Alves (2013) of 20 studies that compared static versus dynamic faces of emotions concluded that moving faces are processed differently to static images and enable a better recognition of emotions in others. Addressing this concern, collections of naturalistic video recordings of disgust and other emotions are beginning to be assembled for shared use by researchers (Valstar & Pantic, 2010; Wang et al., 2021).

A concern with the noted trends in facial expression literature is that, despite the assumed visceral nature of disgust, most studies focus on individual faces in isolation from the rest of the body. The potential for other bodily movements and reactions to disgust has not been considered, and it is only in recent years that aspects such as static body posture have been addressed (Aviezer et al., 2012; Lecker et al., 2017; Lopez et al., 2017; Reschke et al., 2018). The trade-off in seeking details of muscle features at a minuscule level has meant that the face is not seen in relation to the physical or social context in which it is placed. Not only does this obscure the body of the person producing the expression, but it also isolates this person as if in a social vacuum (for similar critique, see Goodwin et al., 2012). The interactional dynamics of disgust are impossible to investigate in such settings; we return to this later when we discuss interactional research on emotions and sensory practices.

3. Sounds of Disgust

While facial features of disgust have been extensively examined, the phonetic details of disgust displays have only recently been investigated. This may be due in part to the attention focused on the lexicon in linguistics, although evidence is increasingly building to demonstrate the importance of what has been termed non-lexical vocalisations (Keevallik & Ogden, 2020), sound objects (Reber, 2012), affect vocalisations (Scherer, 1994), non-linguistic affect vocalisations (Hawk et al., 2009), liminal signs (Dingemanse, 2020), and nonverbal vocalisations (Lima et al., 2013). It is indeed disgust sounds rather than lexical items (e.g. the word 'disgusting') or sounds that induce disgust in others (Cox, 2008; Natalini et al., 2020) that are the focus here, because of their intimate connection to the rest of the body. Compared with lexical speech, non-lexical vocalisations have been shown to enable listeners to very quickly recognise emotions such as disgust in a speaker's voice (Castiajo & Pinheiro, 2019; Lima et al., 2013) and this is maintained across the lifespan (Amorim et al., 2021; Morningstar et al., 2018). The limited work on disgust sounds in interaction has found that these are often isolated sounds with scant surrounding lexical content (Wiggins, 2013) and that they may be more likely to be used by children over five years old and adults (Wiggins, 2014), which suggests that they could be to some extent socialised rather than purely visceral. The full interactional dynamics concerning disgust sounds are yet to be explored.

Work that has been conducted on vocal disgust expressions – what Goffman briefly named revulsion sounds, such as 'ew', 'ugh' or 'yuck' (Goffman, 1978, p. 803) – has focused primarily on disgust as an individual emotional state and on the acoustic or prosodic qualities of emotion sounds when performed by actors or other research subjects (Anikin, 2020a, pp. 29-30), while Youtube-based collections of disgust sounds in real-life contexts are starting to emerge (Anikin & Persson, 2017). The aim of this research is still to perceptually classify decontextualised exemplars. Some studies have even argued that different basic emotions can be predicted from specific combinations of acoustic features, such as amplitude, pitch, and spectral profile (Sauter et al., 2010). Among other things, the pitch and amplitude of these sounds can mean that they are salient and thus difficult to ignore (Amorim et al., 2021; Anikin, 2020b). It has also been claimed that the recognisability of disgust sounds is much lower than for other emotional expressions (Banse & Scherer, 1996), suggesting they are not easily interpretable out of context. The acoustic similarity between the 'grunts of disgust and moans of pleasure' (Anikin, 2020b, p. 1256) provides further evidence that their distinctiveness is likely to be contextually-bound. Similar to what has been done in facial expression literature, other embodied or contextual aspects are not taken into account in studies on sounds.

The way in which disgust sounds are produced might also have functional qualities. Darwin (1872 [1965]) argued that human reactions to repel a contaminating object, such as spitting out food, clearing the throat, or blowing away offensive odours, would result in both the facial and vocal elements of disgust. The conventionalisation of these expressions might then have developed over time (Goffman, 1978; Majid, 2012). Darwin also made an association between disgust and sounds produced toward the back of the mouth, motivated by clearing one's throat. Furthermore, velar/uvular fricatives and glottal stops have been tentatively linked with unpleasant odours (Speed et al., 2021). To reject food from the mouth also involves the lips, and Wierzbicka's research on disgust interjections produced in languages such as Polish (fu/tfu) and German (pfui) provides evidence for the involvement of lips in the labiodental fricative (f) for disgust sounds (Wierzbicka, 1992). The connection between facial movements (e.g. retraction of the lips) during disgust performances and subsequently produced sounds has been investigated by Chong et al. (2016, 2018). Their work illustrates how facial movements when expressing disgust can affect vowel quality (in particular, the frequencies of formants 1 and 2), suggesting that future work could reasonably consider both facial and sound elements simultaneously. A notable omission here is the lack of attention to phonetic variation of both lexical and nonlexical expressions and the functional importance of details such as articulatory transformations, sound extensions, and repetitions. For example, Goodwin et al. (2012) transcribe the sounds produced when reacting to another's behaviour as the conventionalised, albeit extended, English ew and produced once by each speaker (see also Wiggins, 2013). In our case, the displays are repeated across the sequence with notable variations and produced by the speaker whose ingestion is at the centre of the disgust sequence. We therefore aim to explore the articulatory transformations of the sounds and ask what different social actions are accomplished through them, as opposed to semantically similar lexical items.

4. An Interactional Perspective on Disgust

While the previously discussed work on facial gestures and sounds of disgust have, separately, made some progress, they leave out a consideration of the complementary features of these modalities. In everyday settings, while it is possible to express disgust purely using the face with no vocal element, disgust sounds are difficult to perform without any accompanying facial gestures. Work is thus needed that examines these modalities of disgust expression as they are spontaneously produced in combination with one another. Furthermore, while some of the above experimental studies consider the temporal evolvement of a single-modality display, none of them considers their multimodal transformation across an interactional event.

Meanwhile, research within interactional fields has demonstrated the multimodal organisation of emotion expressions (e.g., Peräkylä & Sorjonen, 2012; Ruusuvuori & Peräkylä, 2009) and, more recently, that of sensoriality including taste (e.g., Goico et al., 2021; Mondada, 2018, 2020b). This emerging body of work has contributed to our understanding of how emotion expressions are not only coordinated through various vocal, facial, and embodied gestures but also organised within interactional frameworks and ongoing actions. Disgust has briefly been examined within this field as an example of how emotional stance is organised across the unfolding interaction and involves several bodies (Goodwin et al., 2012). The current paper builds on this earlier work by targeting disgust upon ingestion across a longer sequence, and specifically elevates the focus on the production and placement of non-lexical vocalisations of disgust as a unique phenomenon that can be studied in its own right and in relation to lexical formulations.

Considering calls for greater use of dynamic faces in research and of the noted importance of a multimodal examination of disgust, we argue that it is important to consider dynamic bodies within social interaction. Recent work has established that tasting, both at official tasting sessions and in everyday shopping, is a multimodal and socially coordinated accomplishment (Mondada, 2018, 2020b). Similar to Mondada's concept of multimodal gestalts (Mondada, 2016), our aim here is to consider how embodied gestures of disgust can be understood as the coordination of various modalities by one person and in relation to actions by different participants in social interaction. We thereby contribute to more diverse methods of analysing disgust – such as when it occurs spontaneously in naturalistic settings – that more fully capture the richness of the embodied behaviours and social practices involved, including articulatory features and their occasioned evolvement over time. The specific research question used to guide this study was: how do multimodal expressions associated with ingestion-based disgust unfold interactionally within an everyday context?

5. Methods

The data extract for this paper is taken from a large corpus of video- and audio-recorded family mealtime interaction, recorded in England and Scotland between 2000 and 2014. Full written consent was obtained from the parents in each family and the project has undergone ethical scrutiny from the Psychology Department Ethics board at the University of Strathclyde. Consent was also given to use anonymised images or video clips from the recordings in research publications. Families were provided with two small video cameras to self-record their mealtimes as often as possible over a period of two to four weeks. The positioning of the cameras, timing of recordings, and returning of data to the researchers was therefore entirely under the control of the families themselves. An interest in disgust expressions was developed following ongoing analyses of the data, and due to the unpredictable nature and relatively infrequent occurrence of such expressions, their presence in the corpus is serendipitous. Indeed, it is the apparent elusive quality of these expressions in natural settings that has meant that many researchers make use of experimental settings to induce disgust in participants, with ensuing methodological problems of ecological validity.

The data extract used in this paper was identified during a manual search of the data corpus for instances of disgust expressions. It is taken from a mealtime recorded in Scotland in which the adults have finished eating while their infant is still chewing on small pieces of fruit. For simplicity, we refer to the participants as Mum, Dad, and Infant. A limited amount of demographic information is known about the participants: they were all living in the west of Scotland at the time of the recording, were white, and had English as their first language. The infant was aged 8 months, Mum was in her early 30s and Dad in his early 40s; there were no other children in the family at that time. The family video-recorded 14 meals with their infant in total and this extract was taken from the tenth recording; by this time the family would have therefore acclimatised to the presence of the video cameras. The extract was chosen as it was a particularly clear example that featured not only verbal references to a food being 'disgusting' but was also accompanied by non-lexical vocalisations and facial movements suggestive of disgust. The actual consumption takes place out of view of the cameras (which makes it fundamentally different from experiments where people are observed in immediacy of the putative experience) but was forewarned earlier in the meal by Mum herself. About 10 minutes prior to the data extract, Mum told Dad that she was going to consume a teaspoonful of coconut oil, twice a day, following a friend's suggestion about its health benefits. The extract spans 45 seconds in total, from the brief pause before the first mention of disgust to the point at which the conversation moves onto other topics.

The analysis details the unfolding multimodal interaction during which Mum spontaneously produces disgust expressions following the consumption of the coconut oil. Multimodal interaction analysis (Goodwin, 2000; Mondada, 2007) was used to analyse the data, paying particular attention to the sequential positioning and coordination of non-lexical sounds, facial expressions, gaze, gestures, and the interaction between different participants in the mealtime. The analysis also features a focus on the pitch, place of articulation, and duration of the vocalisations alongside their facial movements to document the inherently varied nature of these sounds. It details the multimodal and interactional production of disgust and demonstrates how it is oriented to not only as an individual experience but also as a reportable event that is furthermore targeted repeatedly in relation to ongoing talk.

6. Analysis

Our focus episode begins with a verbal announcement of disgust, which is subsequently evidenced through various instances of non-lexical vocalisations, facial movements, and embodied gestures. The key analytical points to note are how disgust is not only dynamically expressed through a combination of modalities but also treated as an interactionally topical matter. The full episode has been divided into five sequential extracts to allow for a detailed analysis of each section. Extract 1 opens with a short pause before Mum's announcement of something disgusting from the kitchen. It is at this point that the interactional relevance of disgust is first initiated.

Extract 1. (International Phonetic Alphabet transcriptions between //)

  Open in a separate window

The extract begins with a verbal announcement of disgust (line 2) while Mum is out of sight of the others. The announcement consists of an assessment in the form of a copula clause in past tense, treating the event as having already occurred, with the adjective disgusting in topical focus. The statement is furthermore intensified by the adverb absolutely and through emphasised prosody on the descriptive terms. The precise lexical choices formulate a taste event for the benefit of other participants, even though the consumption takes place elsewhere. The use of a teaspoon places it more firmly in the category of taking medicine, rather than as a food to be consumed; its spatial location also situates it as a personal rather than social event. That the spoon can be heard dropping soon after (line 4) points to the target of the assessment as having been something that can or should be handled with a spoon. Despite the consumption of the oil in this manner, the reportability of the disgusting taste then becomes an interactionally relevant matter and the focus of the unfolding sequence.

The source of the disgust is not, however, immediately apparent. The oblique reference to 'that' (line 2) is queried by Dad (line 4) and first responded to with a non-lexical sound by Mum (line 7). The sound is a uvular-velar fricative, far down in the throat, thus hearably emanating from the position where edible substances are swallowed (Speed et al., 2021). The fricative quality can also emerge when people clear their throats to loosen an offensive substance.1 All of this is suggestive of some problem in Mum's throat, which is being made available to others through the sound in the vocal tract. Indeed, the sound draws Dad's and then Infant's gaze to the kitchen (line 9), possibly orienting to visible signs of trouble or simply mobilising a response (Stivers & Rossano, 2010) to Dad's question. The timing of the non-lexical sound suggests a display of being overcome by disgust (see Goffman, 1978, 'flooding', 'natural overflowing'), as the verbal answer to Dad's question is considerably delayed, although the verbal announcement of disgust before the sound echoes a pattern seen in other expressions of disgust during family mealtimes (Wiggins, 2013). The lexical item thus seems important for conveying the particular conventionalised stance, with the subsequent bodily grounded sound inheriting part of its meaning from that semantic field. Had the sound been purely based on visceral sensations, we would not expect to see such integration with the ongoing interaction, using it as a response to a question by another interactant rather than in the immediate aftermath of the spoonful entering Mum's mouth. In being flexibly usable for interactional purposes, lexical and non-lexical items are functionally similar.

In the opening sequence of this extract, not only is disgust verbally introduced in a way that opens an interactional space, it is also enacted through embodied movements and sounds before the problem source is specified: the lexically intensified verbal assessment performed with emphasis, the dropping of the spoon (line 4), the non-lexical vocalisation (line 7), and the pause (line 9), all occur before the mention of the brief response 'that coconut' (line 10) while walking back to the table. Verbally answering the question displays Mum as sufficiently in control to do so, not flushed out by concerns with the failing body (Hofstetter et al., 2021), thus balancing the display of disgust with interactional obligations. It is only when entering the dining area that Mum then provides a verbal response (line 10) to Dad's question, in the form of an anaphoric reference ('that'), referring to earlier talk and thus treating the other interactants as knowledgeable about her consuming the coconut oil. While the video footage is not sharply focused at this point, there is a slightly down-turned mouth but no observable nose-wrinkle or tongue protrusion. Overall, within just a few seconds of interaction, there is evidence of verbal, facial, vocal, and embodied orientations to something that is verbally classified as disgust. If we were to rely on much of the literature on disgust that typically targets it as a momentary event involving distinct facial or vocal features, we could assume that the episode has come to an end here. We can instead observe an engagement over time that goes well beyond what has been established in earlier studies (however, see Goodwin et al., 2012). Extract 2 continues directly after Extract 1, when Dad then receives the answer and formulates his understanding of the event, as Mum walks back into the room.

Extract 2.

  Open in a separate window

As Mum returns to the dining room table, her face is briefly not seen in the video but her bodily movements remain at a slow walk. Dad's gaze is still on Mum as he smiles and produces a slightly teasing 'oh you've had a lovely-' (line 11) and gustatory 'mmm' (Wiggins, 2002) in response to the clarification about the coconut oil. That humour and disgust are associated has long been acknowledged, although to date there are limited empirical examples of laughter and disgust appearing together in interactional research (e.g. Bateman, 2020; Wiggins, 2013). In this example, the humour is aimed at the person who is apparently experiencing the sensation. Dad gazes at Mum and addresses the turn to her but engages in the episode as a non-suffering observer. Mum does not respond explicitly to this, and while she may not see the smile with her gaze directed downwards, there is no doubt as to the positively-valenced although possibly ironic 'lovely' and 'mmm' that are uttered in a recognisable up-down contour. Indeed, Mum develops the 'that was disgusting' turn (lines 12–13) in overlap with Dad. This disaligning second assessment works to re-address the experience as, in the first place, concerning her body rather than being expressible by another person.

What appears to be happening, therefore, is that both embodied and vocalised behaviours work together to produce the disgust as simultaneously private but also a publicly available event that can be discussed within certain limits: Dad can be involved as an observer but not as a co-experiencer. The repetition of the disgusting claim and embodied display, despite also being a disaligning response in an assessment sequence launched by Dad, underlines the extension of the suffering as still ongoing rather than a momentary past event. The beginning of Mum's turn in line 13 with the exclamatory 'oh my god' suggests a renewed attention to and the ongoing relevance of the taste experience leading up to the repeated formulation of the taste as disgusting. Dad's gaze movement to the infant at the end of the turn, and the infant's own gaze down to their food treats the current assessment sequence as finished, although as seen in Extract 3 below, the episode is not yet over and a further two non-lexical sounds are produced to express the disgust alongside hand gestures and continued facial expressions suggesting repulsion. In contrast to earlier studies (Goodwin et al., 2012; Wiggins, 2013, 2014), the disgust displays are here not juxtaposed with a single momentary event but repeated across considerable timeframes and for varying interactional reasons.

Extract 3.

  Open in a separate window

At this point in the sequence, Mum returns to her seat. The first non-lexical vocalisation (line 16) is a voiceless uvular plosive followed by a velar fricative. Again, the sound is suggestive of something problematic occurring in the throat and the plosive beginning underlines the 'coughing out' character of the vocalisation, thus displaying even more urgent bodily concerns.The sound reflexively specifies the lexical concept of disgust as in this case concerning something that is best expelled from the throat. Mum's facial display becomes more expressive with a gaping mouth and brows furrowed further. There is an absence of a lip-curl or nose-wrinkle despite disgust being otherwise clearly marked through sound, gesture, and facial features that are more closely associated with distaste2 than disgust in theoretical studies that try to distinguish between the two (Weiland et al., 2010). Leaning back, Mum then utters a vocal sound (line 5), a mid-open back vowel that transforms into an open one and ends in a velar fricative, rendering it more as a moan (Hofstetter, 2020), thus evading clearcut classification as a revulsion sound or a moan of suffering. Mum maintains her facial display including the furrowed brows, so the display emerges as a re-actualisation of her putative earlier problem, achieving an extended rather than instantaneous concern of having swallowed something unpleasant. Dad, however, continues his ironic line of positiveness about taste with a triple 'yum, yum, yum', a formulation that regularly precedes or follows gustatory mmms and works to further enhance the semantics of the non-lexical sound (Wiggins & Keevallik, 2021). The couple thus playfully disaffiliates with each other's stances in this assessment sequence regarding Mum's taste experience.

During the movement to sit down, Mum picks up a piece of plum from her plate and, after her vocal and verbal turn, puts it into her mouth. She is also holding up her right hand pushing it away from her body as if holding off something revolting. Although there is no mutual eye gaze between Mum and the others during this period, her verbal output – 'it tastes like hospitals' (line 19) – suggests that she is still orienting to the interactional context and the presence of the others as recipients. This turn is produced not in overlap with Dad but rather in a conversational slot that also synchronises with her sitting down. The movements highlight the bodily bases of her verbal claims. This all amounts to continuation of the embodied vocal display of something unpleasant in her mouth, marked through repeated and phonetically varied disgust sounds ranging from throat expulsion to moans, interlacing them with verbal formulations about taste, accounting for her stance as opposed to that of Dad's.

Like many sensory practices, nuances of disgust are notably hard to articulate verbally, and sensory descriptions often come in the form of tastes like or smells like (Majid, 2021; Mondada, 2020a). This is what Mum utters in line 19, following three productions of non-lexical disgust items and two references to it being 'disgusting'. The semantics of the word 'hospitals' is potentially ambiguous regarding positive or negative valence, although the facial gesture at this moment suggests an unpleasant association, and the turn itself attracts the gaze of Dad. Putting a piece of plum into her mouth in full view of Dad could be a way of closing the topic or of dealing with the unpleasant taste in a publicly accessible manner. In this segment we can thus see how the situationally emergent concern is negotiated between the parties: Mum continues the display of being overwhelmed with the bodily experience and renews it through vocalisations, formulations, and other bodily resources, while the semantics of Dad's verbal turn and his smile suggest a humorous receipt of these displays. Their oppositional stances are thus maintained throughout the sequence, with Mum's assessments re-asserting her primary access to the taste and its experience. Importantly, the facial displays and sounds are repeated in coordination with the evolving interaction, rather than being irregular, instantaneous, and purely individual behaviours. The interactional relevance of the disgust is then further developed by Dad in Extract 4, in which he pursues the topic with a question about Mum's willingness to follow further advice from the person who suggested the coconut oil.

Extract 4.

  Open in a separate window

It is during this section of the interaction that Dad responds verbally to the situation with a suggestion about a hypothetical and perhaps extreme scenario in which Mum may be given the advice of eating lard. Here, there are further embodied movements that might accompany disgust: it is exactly after Dad says the word lard that Mum moves her hand to cover her mouth and closes her eyes, in a gesture that could suggest avoiding further contamination from unpleasant substances or the prevention of subsequent vomiting. As a vivid movement, this attracts a brief gaze by Infant, and Mum's whole forehead becomes furrowed when she repeats yet again 'that's disgustin' (line 24), now in the present tense, which might suggest that it is rather about the current question posed to her. The embodied display as well as the verbal turn remain somewhat ambiguous as to whether they represent her body or the stance taken to ongoing talk, thus balancing between the demands of the body and the requirements of interaction. While she turns her gaze to Infant, beginning to shift away from the individual embodied focus, Dad continues his pursuit of her previously stated commitment to the health project and consume the coconut oil twice a day.

While Mum's embodied movements and verbal expressions continue the multimodal unfolding of the disgust expression and her performance of a solitary engagement with her own body, there are also shifts to indicate a transition out of the emotional stance with Mum's eye gaze moving to Infant (line 25) and visible chewing and swallowing of the plum (lines 25 and 27). Her swallow while her eyes are closed accounts for the lack of her verbal response and suggests resilience or perhaps stoicism. In the final section of the interaction, in Extract 5, we then see how the sequence is fully closed with further orientation to the challenge Mum has set herself and a final non-lexical and embodied display.

Extract 5.

  Open in a separate window

When Mum opens her eyes immediately prior to Extract 5, her face becomes more relaxed, and Dad asks for a final confirmation of her commitment to the oil regime, to which she immediately responds positively. The only remaining facial gesture is the slightly furrowed brow. The conversation proceeds in a rather matter-of-fact manner before the very last disgust sound in this episode, in line 43. As already pointed out, neither the face nor the sound show merely prototypical, unified, and conventional features established through close descriptions of facial muscle work or the orthographic inventions to render the sound in English as an ew or ugh. It lies in the nature of those displays that they do not use the phonemic inventory of regular language and are instead performative of the human body here and now. The sound in the last extract begins in a mid-central (schwa) vowel and transforms into an open back one before being released into an outbreath (velar fricative), thus being partially similar to the earlier disgust sounds in this episode. They feature vowels that are open and further back in the oral cavity, also deployed in moans and pain cries, as well as various fricatives formed in the throat area. The vowel is diphthongised and the outbreath bears similarities to a sigh, thus in effect constituting a more complex meaning than merely 'disgust', 'sadness', or 'disappointment' (or any other simplifying label for emotion displays). The sound is accompanied by a distinct lowering of the corners of the mouth that visibly engages her throat muscles all the way down to her chest (suggestive of AU21, neck tightener, according to FACS). The facial expression is closely matched to the vocal sound although the face is not displaying what are treated as typical disgust features. We are thus highlighting the variability of display features, contesting any uniform classification of either facial displays or sounds as representing a specific emotion.

The final embodied display could be treated as further evidence of the challenge to regularly consume coconut oil that Mum has set herself while simultaneously enacting body trouble. Positioned around 40 seconds after the initial comment 'that was absolutely disgusting' (line 2, Extract 1), it is analytically interesting in how it appears to conclude the discussion of the incident: it emerges sequentially as being the last reified stance on behalf of Mum, which establishes her as having the last word (or sound) on the matter. At the moment of this final display (line 43), Mum's body is facing the Infant to whom she has recently spoken and leaned forward to arrange their food. The Infant themself has their gaze down toward their food and is absorbed in picking up pieces to eat, while Dad is mostly gazing at the Infant with brief glances to Mum. The disgust sound enacted here is therefore in an interactional framework where the Infant's eating practices, rather than Mum's consumption of the coconut oil, is in focus. Only a second later, Dad utters 'that good' to the Infant, in quite the opposite assessment of taste to disgust, and the focus for the family members moves on to other matters.

7. Discussion

The multimodal analysis presented here provides a detailed account of disgust expressions produced in a natural context that transform over time through facial gestures, non-lexical sounds, and embodied movements while simultaneously being embedded within the ongoing interaction with other family members. By scrutinising an everyday episode, we were able to document a variety of disgust sounds, facial expressions, and gestures not typically seen in experimental studies nor analysed to this extent in interactional research on disgust. The sounds seen in the current episode are also produced alongside various facial movements with variable temporal coordination, providing further evidence that it is beneficial to examine faces and sounds in combination, rather than as distinct and separable entities. These sounds and facial gestures transform over the course of the sequence, prolonged by an orientation to the ongoing interaction. Furthermore, they are combined with various other embodied movements, such as head tilted downwards with gaze down or eyes closed (Extracts 1, 2, 4, 5), hand pushing away (Extract 3), and hand covering mouth (Extract 4). These movements can emerge before or alongside the vocal expression that indexes bodily trouble: the mouth can be covered first, the eyes closed as a further performance of self-absorption in the taste experience, with the verbal turn emerging last (Extract 4). Finally, the gestures and sounds are produced in an interactional context in which even a preverbal infant is treated as an active participant and part of the discussion about the challenges of consuming the coconut oil (Extract 5).

The episode demonstrates that very few of the features of the complex display were what has been established as 'standard' in studies of emotions that target facial muscles, orthographically rendered forms of response cries, or experimentally discovered acoustic features. What was particularly notable in the data presented was that the classic 'disgust face' with lip-curl and nose-wrinkle was not seen throughout the 45 second episode we analysed, despite the speaker making several explicit claims to the substance as 'disgusting' and five instances of a disgust sound. What was seen through the facial expressions was presumably something more akin to distaste as a related category or sub-grouping of disgust (Chapman & Anderson, 2012; Chapman et al., 2017; Weiland et al., 2010), particularly since they were produced after substance intake rather than being visually induced. We could see our instances as representing a sensation of consuming something distasteful,3 thus explicitly targeting the body when it comes to contact with materiality.

We also identified sounds that have not previously been included in experimental and YouTube-based studies on disgust in response to visual stimuli (Anikin & Persson, 2017), such as fully voiceless sound combinations. Examining the full complexity of this episode, however, we cannot see any empirical proof for those sounds being functionally different from the voiced ones: they seem to represent a similar (ongoing) experience. We would rather suggest that the range of vocal tract sounds included under the umbrella term 'disgust' be broadened, taking into account the local specificity of each and every production, and acknowledging that there may be ambiguity with sounds performing other emotions or sensations. Otherwise, the term risks being artificially narrowed down to what experiments suggest as prototypical displays, elicited as performances without any real-life consequences. Likewise, conventional orthographies do not do justice to the multitude of articulatory features involved. Despite its possible phylogenetic origin in distaste, disgust could easily remain an analytic umbrella term for reactions on a variety of matters, either presented visually, through touch, smell, or, for that matter, taste. In short, the embodiment of disgust is far richer than what has been established in earlier studies, with the moment-by-moment transformations as potentially meaningful for the evolving interaction, while the repetitions of ostensibly 'immediate' bodily reactions provide evidence of the use of the displays as contributions to unfolding social actions. In this example, they could be described as performing stamina in the face of a difficult challenge, and conveying a stoic stance through disaligning assessments. Lexical formulations alongside non-lexical vocalisations mutually delimit and specify the concrete social meanings conveyed.

The data reported here is clearly far removed from the standardised experimental settings of most research on disgust that specifically target moments of ingestion; direct comparisons between these paradigms of research are thus difficult to make. Our aim here, however, is to offer an example of spontaneous expressions of disgust in a series and, through their analysis, further evidence the importance of the embodied and interactional context for research on disgust in particular. Expressions produced in controlled environments typically involve individuals–who are either producing or perceiving disgust expressions–and thus can tell us very little about how disgust becomes integrated into social practices. By artificially delimiting the eligible sounds, experimenters may miss a huge portion of the actual sounding practices. What also makes this data clip unique in comparison with previous literature is that it occurs immediately following the ingestion, rather than the avoidance of, a disgust-relevant object. The episode analysed here provides an illustration of what can happen in a longer time frame after the actual ingestion of a substance and how a sensation can be brought into interactional relevance across sequences of actions. Importantly, the substance is not something that is being eaten together with others (such as could happen if a child expresses disgust at food at the family meal, for instance) and thus the experience is solitary despite being expressed interactionally.

In conclusion, the analysis offers two specific contributions to research on disgust. The first is in highlighting the importance of movement and temporality, in that gestures have a sequential trajectory as much as they do a fixed static position, as do the sounds that may feature considerable quality changes across time. More research attention should be placed on dynamic faces and sounds in social interaction, and particularly taken from naturalistic settings. Among other things, it may be the case that the language-specific conventionalised sounds such as the English ew are uttered upon visually occasioned displays (such as described in Goodwin et al., 2012) while the pharyngeal sounds are more useful in displaying distaste upon ingestion, building on the sensation of taste. Interestingly, this would provid a socially organised and variably vocalised distinction between the hitherto merely theorised concepts of disgust and distaste. The second main contribution of our work here is to provide support for recent developments (e.g., Farley, 2021, and others) of the need to move beyond the face to consider the body, face, and voice combined in the production of emotional expressions. We would go one step further to suggest that the interactional context is also important: it is not only the sounding and expressing body but the body in collaboration with other bodies that is relevant here (see also Goodwin et al., 2012). Another participant can provide a related, if contrastive, formulation of bodily experience, and thereby launch a negotiation of one participant's body trouble. Through this kind of multimodal and seqeuntial exploration, the social and moral implications of disgust (Curtis, 2011; Miller, 1997) and related emotions can be more fully explored.

Acknowledgements

The authors would like to thank the journal editor and reviewers for their careful attention to detail and constructive feedback on an earlier version of the manuscript. The research was supported by funding from the Swedish Research Council grant VR 2016-00827 'Vocal practices for coordinating human action' and we are very grateful for comments on the earlier version of the paper by Emily Hofstetter, Adrian Kerrison, Agnes Löfgren, and Hannah Pelikan.

References

Abramson, L., Petranker, R., Marom, I., & Aviezer, H. (2021). Social interaction context shapes emotion recognition through body language, not facial expressions. Emotion, 21(3), 557-568.

Amorim, M., Anikin, A., Mendes, A. J., Lima, C. F., Kotz, S. A., & Pinheiro, A. P. (2021). Changes in vocal emotion recognition across the life span. Emotion, 21(2), 315-325.

Anikin, A. (2020a). Human nonverbal vocalizations. Lund University.

Anikin, A. (2020b). The link between auditory salience and emotion intensity. Cognition and Emotion, 34(6), 1246-1259.

Anikin, A., & Persson, T. (2017). Nonlinguistic vocalizations from online amateur videos for emotion research: A validated corpus. Behavior research methods, 49(2), 758-771.

Aviezer, H., Trope, Y., & Todorov, A. (2012). Body Cues, Not Facial Expressions, Discriminate Between Intense Positive and Negative Emotions. Science, 338(6111), 1225-1229.

Banse, R., & Scherer, K. R. (1996). Acoustic profiles in vocal emotion expression. Journal of personality and social psychology, 70(3), 614.

Bateman, A. (2020). Young children's affective stance through embodied displays of emotion during tellings. Text & Talk, 40(5), 643-668.

Castiajo, P., & Pinheiro, A. P. (2019). Decoding emotions from nonverbal vocalizations: How much voice signal is enough?. Motivation and Emotion, 43(5), 803-813.

Chapman, H. A., & Anderson, A. K. (2012). Understanding disgust. Annals of the New York Academy of Sciences, 1251(1), 62-76.

Chapman, H. A., Lee, D. H., Susskind, J. M., Bartlett, M. S., & Anderson, A. K. (2017). The face of distaste: A preliminary study. Chemical Senses, 42(6), 457-463.

Chong, C. S., Kim, J., & Davis, C. (2016). The sound of disgust: How facial expression may influence speech production. Interspeech conference, San Franscisco, USA.

Chong, C. S., Kim, J., & Davis, C. (2018). Disgust expressive speech: The acoustic consequences of the facial expression of emotion. Speech Communication, 98, 68-72.

Cohn, J. F., Ambadar, Z., & Ekman, P. (2007). Observer-based measurement of facial expression with the Facial Action Coding System. The handbook of emotion elicitation and assessment, 1(3), 203-221.

Cox, T. J. (2008). The effect of visual stimuli on the horribleness of awful sounds. Applied Acoustics, 69(8), 691-703.

Curtis, V. (2011). Why disgust matters. Philosophical Transactions of the Royal Society B: Biological Sciences, 366(1583), 3478-3490.

Darwin, C. (1872 (1965)). The expression of the emotions in man and animals. The University of Chicago Press.

Dingemanse, M. (2020). Between Sound and Speech: Liminal Signs in Interaction. Research on Language and Social Interaction, 53, 188-196.

Ekman, P., & Friesen, W. V. (1978). Manual for the Facial Action Coding System. Palo Alto, CA: Consulting Psychologist Press.

Farley, S. D. (2021). Introduction to the Special Issue on Emotional Expression Beyond the Face: On the importance of multiple channels of communication and context. Journal of Nonverbal Behavior, 45(4), 413-417.

Fridlund, A. J. (2017). The behavioral ecology view of facial displays, 25 years later. The science of facial expression, 77-92.

Goffman, E. (1978). Response Cries. Language, 54, 787.

Goico, S., Gan, Y., Katila, J., & Harness Goodwin, M. (2021). Capturing multisensoriality: Introduction to a special issue on sensoriality in video-based fieldwork. Social Interaction. Video-Based Studies of Human Sociality, 4(3).

Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32, 1489-1522.

Goodwin, M.H., Cekaite, A., & Goodwin, C. (2012). Emotion as stance. In A.Peräkylä & M-L. Sorjonen (Eds.), Emotion in interaction, (pp. 16-41). Oxford: Oxford University Press.

Hawk, S. T., Van Kleef, G. A., Fischer, A. H., & Van Der Schalk, J. (2009). "Worth a thousand words": absolute and relative decoding of nonlinguistic affect vocalizations. Emotion, 9(3), 293.

Hofstetter, E. (2020). Nonlexical “Moans”: Response cries in board game interactions. Research on Language and Social Interaction, 53, 42-65.

Hofstetter, E., Keevallik, L., & Löfgren, A. (2021). Suspending syntax: Bodily strain and progressivity in talk. Frontiers in Communication, 6, Article 663307.

Israelashvili, J., Hassin, R. R., & Aviezer, H. (2019). When emotions run high: A critical role for context in the unfolding of dynamic, real-life facial affect. Emotion, 19(3), 558-562.

Keevallik, L., & Ogden, R. (2020). Sounds on the Margins of Language at the Heart of Interaction. Research on Language and Social Interaction, 53, 1-18.

Knapp, C. (2003). De-moralizing disgustingness. Philosophy and Phenomenological Research, 66, 253-278.

Lecker, M., Shoval, R., Aviezer, H., & Eitam, B. (2017). Temporal integration of bodies and faces: united we stand, divided we fall? Visual Cognition, 25(4-6), 477-491.

Lima, C. F., Castro, S. L., & Scott, S. K. (2013). When voices get emotional: A corpus of nonverbal vocalizations for research on emotion processing. Behavior research methods, 45(4), 1234-1245.

Lopez, L. D., Reschke, P. J., Knothe, J. M., & Walle, E. A. (2017). Postural communication of emotion: Perception of distinct poses of five discrete emotions. Frontiers in Psychology, 8.

Majid, A. (2012). Current emotion research in the language sciences. Emotion Review, 4(4), 432-443.

Majid, A. (2021). Human olfaction at the intersection of language, culture, and biology. Trends in Cognitive Sciences, 25(2), 111-123.

McNally, R. J. (2002). Disgust has arrived. Journal of Anxiety Disorders, 16, 561-566.

Miller, W. I. (1997). The anatomy of disgust. Harvard University Press.

Mondada, L. (2007). Commentary: transcript variations and the indexicality of transcribing practices. Discourse Studies, 9, 809-821.

Mondada, L. (2016). Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20, 336-366.

Mondada, L. (2018). The multimodal interactional organization of tasting : Practices of tasting cheese in gourmet shops. Discourse Studies, 20, 743-769.

Mondada, L. (2020a). Audible Sniffs: Smelling-in-Interaction. Research on Language and Social Interaction, 53, 140-163.

Mondada, L. (2020b). Orchestrating Multi-sensoriality in Tasting Sessions: Sensing Bodies , Normativity, and Language. Symbolic Interaction, 4(1): 63-84.

Morningstar, M., Ly, V. Y., Feldman, L., & Dirks, M. A. (2018). Mid-adolescents' and adults' recognition of vocal cues of emotion and social intent: Differences by expression and speaker age. Journal of Nonverbal Behavior, 42(2), 237-251.

Natalini, E., Dimaggio, G., Varakliotis, T., Fioretti, A., & Eibenstein, A. (2020). Misophonia, maladaptive schemas and personality disorders: a report of three cases. Journal of Contemporary Psychotherapy, 50(1), 29-35.

Peräkylä, A. & Sorjonen, M-L. (Eds.), Emotion in interaction. Oxford: Oxford University Press.

Reber, E. (2012). Affectivity in interaction: Sound objects in English. J. Benjamins. Amsterdam.

Reschke, P. J., Knothe, J. M., Lopez, L. D., & Walle, E. A. (2018). Putting “context” in context: The effects of body posture and emotion scene on adult categorizations of disgust facial expressions. Emotion, 18(1), 153-158.

Rozin, P., Lowery, L., & Ebert, R. (1994). Varieties of disgust faces and the structure of disgust. Journal of personality and social psychology, 66, 870-881.

Ruusuvuori, J., & Peräkylä, A. (2009). Facial and verbal expressions in assessing stories and topics. Research on Language and Social Interaction, 42(4), 377-394

Sauter, D. A., Eisner, F., Ekman, P., & Scott, S. K. (2010). Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. Proceedings of the National Academy of Sciences, 107(6), 2408-2412.

Scherer, K. R. (1994). Affect bursts. Emotions: Essays on emotion theory, 161, 196.

Speed, L. J., Atkinson, H., Wnuk, E., & Majid, A. (2021). The Sound of smell: Associating odor valence with disgust sounds. Cognitive Science, 45(5), e12980.

Stevenson, R. J., Case, T. I., Oaten, M. J., Stafford, L., & Saluja, S. (2019). A Proximal Perspective on Disgust. Emotion Review, 11(3), 209-225.

Stivers, T., & Rossano, F. (2010). Mobilizing Response. Research on Language & Social Interaction, 43(1), 3-31.

Torro-Alves, N. (2013). Recognition of static and dynamic facial expressions: a study review. Estudos de Psicologia (Natal) 18, 125-130.

Trautmann, S. A., Fehr, T., & Herrmann, M. (2009). Emotions in motion: Dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations. Brain Research, 1284, 100-115.

Valstar, M. F., & Pantic, M. (2010). Induced disgust, happiness and surprise: An addition to the MMI Facial Expression Database. In Proc. 3rd Intern. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect (p. 65)

Van Der Schalk, J., Hawk, S. T., Fischer, A. H., & Doosje, B. (2011). Moving faces, looking places: validation of the Amsterdam Dynamic Facial Expression Set (ADFES). Emotion, 11(4), 907.

Wang, J., Poskanzer, C., & Anzellotti, S. (2021). Extracting dynamic facial expressions from naturalistic videos.

Weiland, R., Ellgring, H., & Macht, M. L. (2010). Gustofacial and olfactofacial responses in human adults. Chemical Senses, 35(9), 841-853.

Wierzbicka, A. (1992). The semantics of interjection. Journal of Pragmatics, 18(2-3), 159-192.

Wiggins, S. (2002). Talking with your mouth full: gustatory mmms and the embodiment of pleasure. Research on Language & Social Interaction, 35, 311-336.

Wiggins, S. (2013). The social life of 'eugh': Disgust as assessment in family mealtimes. British Journal of Social Psychology, 52, 489-509.

Wiggins, S. (2014). Family mealtimes, yuckiness, and the socialisation of disgust responses by preschool children. In P. Szatrowski (Ed.), Language and food: Verbal and nonverbal experiences (pp. 211-232). John Benjamins Publishing Company.

Wiggins, S., & Keevallik, L. (2021). Enacting gustatory pleasure on behalf of another: The multimodal coordination of infant tasting practices. Symbolic Interaction, 44(1), 87-111.


1 Note that these kinds of voiceless sounds have been regularly omitted from sound collections, such as in Anikin (2020, p. 13, 31).

2 In psychological literature, distaste is often theoretically distinguished from disgust as being the rejection of foods based on sensory rather than moral grounds, although this distinction is yet to be evidenced in empirical research.

3 It is interesting to note that the word 'distasteful' in English has moral connotations, although 'distaste' is usually treated theoretically as a purely sensory matter.