Social Interaction
Video-Based Studies of Human Sociality
Surprise About News or Just Receiving Information?
Moving and holding Both Eyebrows in Co-Present Interaction
Carolin Dix1 Alexandra Groß2
1University of Innsbruck 2University of Bayreuth
Abstract
We focus on the conversational use of raising both eyebrows in response to a new occasion or information. Two fundamental patterns were found to come into play as frequent visual practices of change-of-state marking: the continuous moving of the eyebrows up and down and the holding of both eyebrows raised. The eyebrow move marks the receipt and unproblematic understanding of news, either as part of responses or as the recipient's activity during turn production. In contrast, the eyebrow hold appears as an essential part of a salient visual news mark practice displaying surprise or astonishment. While the move is embedded in minimal and unobtrusive change-of-state practices allowing the interlocutors to move on, the hold treats the information received as worthy of further elaboration. However, verbal and embodied practices of news receipting and news marking may diverge in such a way that contradicting conversational demands are contextualised.
Keywords: raising both eyebrows, facial gestures, change-of-state marker, news receipting, news marking
1. Introduction
Participating in social interaction means orienting towards and constantly balancing asymmetries of knowledge in such a way that a mutual understanding of new information is accounted for on the moment-to-moment basis of sequential interactional conduct. Establishing local intersubjectivity in co-present interaction naturally involves using the face as an embodied conversational resource while delivering, receiving and positioning oneself towards pieces of information that have surfaced during talk.
To date, several studies in the tradition of Conversation Analysis (CA) and Interactional Linguistics (IL) have investigated how recipients verbally respond to the delivery of new information by using minimal tokens like oh, ah, echt ('really') and achso ('I see') (e.g. Heritage, 1984; Imo, 2009; Golato & Betz, 2008; Golato, 2012; Barth-Weingarten et al., 2020), which Heritage (1984) first called change-of-state (hereafter: CoS) tokens . Besides making the results of cognitive processes intersubjectively available, CoS tokens may treat the preceding talk as remarkable (Marmorstein & Sczcepek Reed, 2023), thereby operating as news marks (Heritage, 1984). They further display emotions like surprise or astonishment (Imo, 2009: 64), and are frequently followed by explicit assessments (Heritage, 1984, 1998). In this context, the raising of both eyebrows (hereafter: RBE) has already been described as part of CoS-moments in conversation (e.g. Gudmundsen & Svennevig, 2020), following observations from emotion psychology that RBE as an essential part of a prototypical surprise expression constitutes 'an exclamation over something amazing, incredible etc.' (Ekman, 1979: 184). However, as this contribution will demonstrate, the facial movement of RBE not only contextualises the inner state of surprise but is part of multimodal practices to indicate different facets of a changed state of mind1 while complying with local conversational demands and establishing new relevancies for the further progress of talk.
Therefore, the goal of our paper is twofold: while building our analysis on a collection of different CoS-moments, that is, responses to elicited as well as unelicited informings and noticings of conversational actions or extra-conversational events, we, first, aim to demonstrate that eyebrow raises are frequently involved in these conversational contexts where a changed state of mind is displayed. Second, we will show that two patterns of RBE serve as contextualisation cues with regard to the quality of the producer's change of state display. In particular, we distinguish two forms of RBE: (1) moving the eyebrows up and down without holding a peak position and (2) holding the eyebrows raised at the highest point for a brief moment in time before quickly lowering them to the neutral position. As will be shown later, the move typically marks (unproblematic) news receipt (see Section 5.1), while the hold appears within salient practices of treating the information not only as new but moreover as surprising or astonishing (see Section 5.2). They both may comply with already-established conversational demands in the context of news delivery, but it is the hold in particular that actively works towards the shaping of the upcoming conversational path.
The following video extract provides an introductory example of our research object, in this case a salient eyebrow hold for displaying surprise in the face of a new and potentially delicate piece of information.
Figure 1. Car ride.
In what follows, this paper starts with a literature review outlining the previous research on eyebrow raises (in interaction) (Section 2) and on verbal and multimodal practices of how interlocutors deal with newness and unexpectedness in conversation (Section 3). We then introduce our corpus, describe the methodological approach of the analysis (Section 4) and continue with the presentation of the results (Section 5). Our findings are summed up and discussed in Section 6.
2. Raising Both Eyebrows
The human face is a dense cluster of several resources (e.g. forehead, eyebrows, eyelids), which is characterised by an omnipresent perceptibility: in contrast to other parts of the human body (such as arms and legs) the face can be 'silenced' only to a limited extent. Even when wearing a face mask, resources such as the eyes, eyebrows and forehead are still visible to other people. This high degree of perceptibility has facilitated previous research on how to 'read' faces – primarily with respect to the inner emotional states of the producers. Darwin points to the inextricable link between the presentation of cognitive and emotive states like surprise and RBE: 'Attention is shown by the eyebrows being raised; and as this state increases into surprise, they are raised to a much greater extent with the eyes and the mouth widely open' (Darwin, 1872/2007: 281). From the perspective of emotion psychology, Ekman and colleagues (Ekman & Friesen, 1969: 80; Ekman & Oster, 1979: 531) identify RBE as an essential part of a prototypical surprise expression. However, Ekman (1979: 183) also mentions RBE as one of the 'most frequent facial actions employed as conversational signals' (Ekman, 1979: 173). Even though Ekman and colleagues describe details of the spatial physiology of RBE in terms of Action Units 1 and 2 (Ekman, 1979: 173), they only mention one temporal unfolding: the eyebrow flash – a rapid upward-downward eyebrow movement, which was found to be involved in greetings (Ekman, 1979: 187) – while neglecting other patterns like the eyebrow hold. Although not putting his focus on the multimodal conduct of social interaction, Ekman observes that RBE often appears together with verbal tokens such as oh or wow and additional facial movements such as raised upper eyelids (Ekman, 1979: 178 and 181).
From the perspective of communication theory, Bavelas and Chovil (e.g. 2018) strengthen the observation that RBE is used as a conversational facial gesture (Bavelas et al., 2014a: 16). According to their investigations, RBE has multiple pragmatic functions, one being the marking of prominence in syllables, words and phrases (Bavelas et al., 2014a: 18; Bavelas et al., 2014b: 121), which expands on a finding by Ekman (Ekman, 1979: 183). Based on half-experimental and experimental methods (e.g. perception tests), quantitative approaches in linguistics underpin these observations showing that RBE is significantly aligned with pitch accents during speech (Swerts & Krahmer, 2008, 2010; Granström & House, 2005; Ambrazaitis & House, 2017). Therefore, RBE is referred to as part of a visual prosody (Beskow et al., 2009: 10) contributing to the production and perception of prosodic prominence (Ambrazaitis & House, 2017: 100f). Interestingly, the authors observe that linguistic tokens in news readings (in several cases the lexeme new) are frequently accompanied by RBE.
Moreover, the interactional functions of RBE have been described as centring around the organisation of talk in general, that is, as an embodied discourse marker when using it for structuring turns (Ekman & Friesen, 1969; Chovil, 1991/1992; Flecha-Garcia, 2010; Swerts & Krahmer, 2010; Ambrazaitis & House, 2017). RBE can also be used by interlocutors as an attention-getting device to (re)establish focused interaction (Bublitz & Kühn, 1981; Guaitella et al., 2009: 220). Further, a current speaker might signal that he or she intends to hold or – vice versa – yield the turn (Ekman, 1979: 185). Among recipients, raised eyebrows can be used as a listener's response and backchannelling device or as a device for requesting (more) information (Ekman, 1979: 186). Bavelas et al. (2014a) describe it as a meta-communicative device commenting on the topic of talk, which can – from a CA perspective – be related to the concept of multimodal stance-taking. A systematic review of how RBE is used as an embodied stance-taking device, such as for indexing obviousness, is provided by Andries et al. (2023).
Another group of studies investigates the co-occurrence of RBE with interrogative syntactic structure (e.g. Chovil, 1991/1992; Wierzbicka, 2000; Crespo Sendra et al., 2013; Bavelas et al., 2014a: 20, Bavelas et al., 2014b: 124; Cruz et al., 2017). These investigations follow the assumption that RBE can indicate questions and therefore occurs as a visual question mark (Ekman, 1979: 185). However, it seems that RBE is a rather unspecific cue for interrogativity, since participants 'rely more on intonation than on facial gestures' (Cruz et al., 2017: 11). Nevertheless, Nota et al. (2021) find in a large-scale quantitative study investigating facial signals in questions and responses that RBE signals the intention to pose a question.
Several authors have investigated the role of RBE for constructing conversational actions, using methods from multimodal CA. Aiming at differentiating responses versus questions by looking at action-specific configurations of the face, Nota et al. (2021) show that the combination of gaze shifts with eyebrow raises frequently mark responsive actions. Further, RBE has been mentioned as part of offering practices (Kärkkäinen & Keisanen, 2012) and within the context of repair initiation (e.g. Oloff, 2018; Stolle & Pfeiffer, 2023/this volume; Li & Wang, 2023/this volume). Clift and Rossi (2023/this volume) describe how RBE in the transition space between turns can either press an addressee to respond to disaffiliative moves like challenges or invite a response to an affiliative action such as a joke. Used by recipients, RBE can furthermore operate as a visual device displaying active recipiency, understanding and empathy in social interaction (Kupetz, 2015: 10f).
The present contribution takes up the results of three further studies in the tradition of multimodal CA that identify the RBE as part of CoS-practices: Mondada (2011), Heath et al. (2012) and Gudmundsen and Svennevig (2020). Their results will be described in the next section.
3. Responding to New Information in Conversation
In knowledge-related conversational actions like informings and responses to requests for information, a certain degree of prominence is assigned to the piece of information within an evolving stretch of talk (Sorjonen, 2001: 282). Respondents, on their part, mark the delivery of the news and signal that they have undergone a change in their 'orientation or awareness' (Heritage, 1984: 299). As recurrent conversational solutions for this kind of conversational task, verbal elements in different languages have been identified and described as change-of-state tokens (first: Heritage, 1984) or news particles (Koivisto, 2016). Without prespecifying their form, Golato (2012) uses the term change-of-state markers, which we adapt in this study, since it seems to fit best for referring to embodied practices. Regarding respondents' verbal practices indexing that an announcement is news-for-them (Maynard, 1997: 104), Thompson et al. (2015) describe interactional objects of varying sizes: In particle rich languages like German or English, verbal CoS-markers can take various forms ranging from single particles and combinations (ah, ach, ach so, oh, ja, aha, etc.), adverbs (echt? ehrlich? 'really?') to phrases (Ich verstehe 'I see') or complex metacommunicative statements (Imo, 2009: 62–63). To date, verbal CoS-tokens have been the subject of various studies for American and British English, Finnish, Danish and German, for example. The most prominent element that has been investigated in several studies (e.g. Heritage, 1984, 1998; Golato, 2012; Barth-Weingarten et al., 2020) is the interjection oh. In English, it might display a changed cognitive state or independent epistemic access, while in German it can be used to display disappointment or affiliation facing negative information (Golato, 2012: 249).
Regarding the functional potential of response particles in general, Sorjonen (2001) points out that they can accomplish different jobs in addition to displaying a changed state of mind (see also Golato, 2012). Variations between similar forms of particles are grounded in a fine-grained division of labour, so that 'within each class, each member is a lexicalization of certain types of interactional meanings' (Sorjonen, 2001: 284f). Sorjonen classifies them into (1) the producer's epistemic assumptions, (2) the affective assumptions and (3) the assumptions of the progression of the interaction. Since these different functional areas are important for our own analysis of RBE as CoS marker, we shall describe them briefly in the following.
Firstly, the producer can position him- or herself on an epistemic gradient between less knowing and more knowing (Thompson et al., 2015). The scalarity of newness or, further, the extent to which a respondent's awareness has changed points to the fact that the division between CoS-markers and acknowledgement tokens like okay (Helmer et al., 2021) seems to be gradual. For CoS's endpoints, Maynard (1997: 107–108) coined the terms news marks and news receipts. Some of them indicate the kind of cognitive change the producer has gone through: for example, when choosing the German particle combination ach so 'I see' a respondent signals that the new information stands in contrast to former assumptions and that interlocutors now share the same epistemic level (Golato & Betz, 2008). By using ach JA 'oh yeah' with focus accent on the second part, respondents claim 'that their coparticipants' utterance is not new to them but instead constitutes just-now-remembered information' (Betz & Golato, 2008: 61), whereas ^ACHja (with a pitch peak on the first part) is described as a withholding practice in order to signal lacking access to the relevant information for the next turn. In contrast, by using a single ach 'oh', speakers index the receipt of new informational content but, at the same time, display that a further negotiation is necessary in order to check whether they have fully understood (Golato & Betz, 2008; Golato, 2010).
Secondly, in addition to signalling that a piece of information is new and relevant (Imo, 2009: 58), respondents may also accomplish some kind of assessment (Heritage, 1984). They may further display (dis)affiliation with the interlocutor's affective stance, which contrasts with merely neutral registering of information (e.g. Sorjonen, 2001: 280). The display of emotional involvement, such as surprise, appears within a conversational ecology, which may be shown by interlocutors preparing the surprise reaction several turns in advance (Wilkinson & Kitzinger, 2006), that is, 'to anticipate the unexpected' (Heath et al., 2012: 216). Displays of surprise or astonishment have been related to different minimal units, called response cries (Goffman, 1981), reaction tokens (Wilkinson & Kitzinger, 2006) as well as news marks viewed as verbal expressions of 'ritualized disbelief' (Heritage 1984: 339). An emotive quality, however, is not limited to the use of minimal lexical elements such as really: Rossi (2020) describes the (partial) repetition of the lexical material of the informing as another verbal practice to convey the speaker's astonishment.
Thirdly, the choice of specific particles, the combination of particles or phrasal expressions in response to (question-elicited) informings 'affects the way the conversation unfolds' (Hilmisdóttir, 2016: 134). Heritage (1984) describes for really versus oh really that the conversation might entail different paths when providing the recipient with different information about one's own stance while choosing one or the other CoS-practice. In this respect, closing relevance is assumed to be highly intertwined with the type of interactional stance respondents take. When respondents formally acknowledge the action of informing with CoS-tokens like the German ah or okay, they treat the topic as closing-relevant (e.g. Helmer et al., 2021). In contrast, news marks like echt 'really' or ach 'oh' seem to demand sequence expansion while signalling incredulity, astonishment or surprise. However, Gubina and Betz (2021) assert that the German echt 'really' might be used to continue or curtail the topic. In a similar vein, Weber and König (2023) describe the difference between news marks and CoS-markers as a continuum, and regarding the Finnish aha versus aijaa Koivisto (2016) shows that the larger action context has to be taken into account in order to capture the specific functions CoS tokens can fulfill.
So far, the practices used to signal the overcoming of an epistemic asymmetry in social interaction when displaying various facets of a new interactional stance have primarily been described with a focus on verbal linguistic resources. However, Thompson et al. (2015: 69) point to the fact that intonation rather than lexical choice might be the decisive resource for displaying different interactional stances and launching sequence closure versus prompting further talk. Moreover, bodily CoS-resources can be found to take a significant role within CoS-moments, particularly RBE: there are a few studies in the tradition of multimodal CA that have observed how interlocutors display a changed state of mind using RBE as a facial gesture. Mondada (2011: 548) provides a case of displaying a new and deeper understanding in instructional settings, which involves maximally raised eyebrows. Using such a salient gesture, understanding is constructed as interpersonal achievement, since it is publicly displayed and potentially consequential for the ongoing conversation. We will show that this particularly holds true for salient eyebrow movements such as extended holds, whereas RBE can also be part of rather minimal responsive practices of marking news receipt.
Focusing on the resolution phase of vocabulary-oriented sequences in second language interaction, Gudmundsen and Svennevig (2020) identify the change-of-state face as a recurrent facial configuration 'which consists in raised eyebrows and widened eyes and is produced while the head and/or the torso is raised upwards' (Gudmundsen & Svennevig, 2020: 1). A similar facial configuration with open eyes and open mouth is described by Heath et al. (2012), who focus on the emerging production of surprise when confronted with unexpected installations in museums. The authors find displays of surprise being inextricably linked to an occasion (an object or event) while serving to render actions and events intelligible. Here, 'the eyes and eyebrows provide a resource to enable others to scrutinize the immediate environment and detect the “surprisable”' (Heath et al., 2012: 220).
The following analysis takes up these previous results. It aims to expand on them by giving new insights into manifestations of RBE and their interrelation with specific conversational functions within CoS-moments.
4. Data and Methods
This study builds on nearly five hours of video-recordings from three different naturally occurring German face-to-face interactions, one of them dyadic and two in a multiparty constellation, recorded with several cameras (e.g. 360°camera, GoPro): a games evening (4 participants), a dinner-cooking event (7 participants) and a car journey (2 participants).
Using methods of multimodal CA (e.g. Mondada, 2019), we first conducted a qualitative analysis of instances of RBE within the data. Based on this, we found that RBE frequently occurred as an embodied CoS-marker (see Section 5 for further details). Thus, we built a functional sub-collection of all CoS-moments. The collection comprises a total of 301 CoS-moments. We coded the sequential context and examined for all instances whether or not an RBE was produced. RBE occurred in the following sequential contexts:
- in response to initial, unelicited informings (CoS-marking as second turn),
- in third turns after question-elicited informings (including other-initiated repair) (CoS-marking as third turn),
- in noticings (CoS-practice as first turn) and lastly
- as recipient activity during the interlocutor's production of informings (CoS-marking during turn production).
We treated these sequential contexts as conversational moments of a changed state of mind involving 'the transmission of information from an informed to an uninformed party' (Heritage, 1984: 304). Thus, we adopted a broad concept of change of state marking, not only including the conversational marking of revised assumptions (such as in German ach so 'I see'), but also including the marking of an increase of knowledge and of having understood the information delivered.
Detailed case analysis then revealed two fundamental patterns of RBE during CoS-moments: the move, which is constituted by a continuous upward and downward movement of the brows, and the hold, which exhibits an un-dynamic phase of 'frozen' eyebrows held up in such a way that the forehead forms horizontal wrinkles.
As will be described in Section 5, the two patterns were identified as exhibiting different functional potentials. Consequently, all examples were coded according to how the upward-downward movement of the eyebrows unfolds. We did not further differentiate between finer patterns of muscular activity of RBE represented in the Ekmanian Action Units 1 and 2 of the Facial Action Coding System (FACS, Ekman et al., 2002). Instead, we coded which verbal CoS-token(s) were involved in turn production as well as how RBE was orchestrated with other visual resources.
For the transcription, we combined the conventions of the Gesprächsanalytisches Transkriptionssystem (GAT2, Selting et al., 20092) for the verbal and vocal resources with the symbols of the International SignWriting Alphabet (ISWA, Sutton, 2010; Parkhurst & Parkhurst, 2008) for the bodily resources. The multimodal transcripts involve still images for a holistic impression of the most salient moment as well as separate lines for a differentiated view of the single visual resources (Dix, 2022; Dix, 2023/this volume).3
5. Results
We found that in 126 instances out of all 301 CoS-moments the respondents raised their eyebrows, which is 41.9% of all cases. Of the 126 RBE instances, 66 were realised as a hold and 60 as a move. 86 (28.6%) instances of CoS-marking in the collection were realised solely by the use of embodied resources without any accompanying verbal CoS-tokens. Among these purely embodied CoS-marking practices, RBE was involved in 59 instances. The verbal part of the multimodal change-of-state practices included particle-shaped CoS-markers like echt 'really', was 'what', ach so 'I see' and ah ja 'oh yes' produced as a stand-alone element or turn-initially.4
Our major result concerns the functional difference between the two manifestations of RBE: while the temporal unfolding appears to follow a continuum with the short move (flash) and the long hold as the endpoints of this scale, we found the structural patterns of continuously moving versus visibly holding both eyebrows raised to be involved in different multimodal practices accomplishing categorically distinctive conversational tasks while treating a new occasion or information.
In the following, we focus on two prominent functions5 of multimodal CoS-marking involving RBE, which exhibited a clear picture regarding the distribution of eyebrow moves versus holds: signalling news receipt and thereby displaying a cognitive CoS (57/126) rather than news marking and displaying surprise or astonishment facing new information (40/126). When signalling news receipt by the use of RBE, it was predominantly the eyebrow move which was observable (44/57). As the case analyses in Section 5.1 show, the CoS is produced shortly before or synchronised with minimal news receipt tokens (ah [ja] 'oh [yes]'; ach so 'I see'), while the whole multimodal gestalt appears to be rather small. In contrast, for displaying surprise or astonishment the eyebrow hold is the pattern of choice (34/40) and is typically embedded in a salient multimodal practice involving prosodically marked (Selting, 1995) CoS-tokens as well as a widening of the eyes and the mouth (see Section 5.2).
Based on extracts from the data collection, the interactional uses of moving and holding both eyebrows as part of practices signalling news receipt and displaying surprise or astonishment will be illustrated in the following sections.
5.1 The eyebrow move as a visual news receipt marker
Using a (rapid) eyebrow move, respondents frequently indicate an instance of visual news receipt. Thus, they treat (elicited or unelicited) information as newsworthy, while – as (part of) responses – they do not actively work towards topic expansion. Produced during turn-production, the move is used by recipients as a continuer, that is, for signalling ongoing recipiency and marking of the successful receiving of a piece of information. The move may further be used in the noticing of occasions, such as moves of fellow players during the activity of a board game. As an unobtrusive and minimal CoS-practice, we found the eyebrow move in all sequential contexts to exhibit a mainly retrospective focus complying with the already-set conversational demands, while not actively working towards more topic talk about the piece of information received or the occasion noticed.
Excerpt 1 illustrates how the eyebrow flash is involved in a multimodal practice signalling a changed state of mind. It occurs during an insertion sequence within the superordinate activity of telling. The example is taken from a conversation between two young women during a car journey which we have already seen in the introductory example (see Figure 1). Jenny, the driver, is telling Kate about how she and some friends gathered at a fast-food restaurant in the middle of the night. The transcript contains the representation of head movement (line H), brow movement (line B) and movement of the eyelids (line E).
Extract 1. Zufällig / Accidentally'
Open in a separate window
In partial overlap with Jenny's TCU (l. 1–2), Kate initiates an insertion sequence and queries whether they had arranged the nightly meeting in advance or met there coincidentally (l. 3, 5). This is done in the form of an alternative question. The second candidate answer (or did you organize anything), however, is uttered in overlap with Jenny's response (l. 4) in which Jenny first claims lacking memory of how the meeting came about and then utters the strong assumption that all groups just met there without knowing that all the other people would be there as well (l. 6–10). Jenny's turn is produced with disfluencies and several instances of self-initiated repair (l. 9–12), which makes the possible completion point of the response turn difficult to project (e.g. Auer, 2005). At the point where Kate's response sets in, Jenny has already uttered the finite copola sind ('are', l. 8) (which is used as an auxiliary to the perfect tense with verbs of movement) and a possible subject (alle GRUppen 'all groups'), which she then replaces with fAst alle GRUppen ('almost all groups', l. 12). So, while the TCU is grammatically still incomplete at this point the relevant information has already been delivered (which is that the groups met coincidentally). In overlap with Jenny's repair, Kate first utters the acknowledgment token hm and then the CoS-token ach SO ('I see', l. 13) with falling intonation and focus accent on the second syllable (Golato & Betz, 2008). Simultaneously with the latter, she produces an eyebrow flash ( l. 13/B) together with a short head lift (, l. 13/H). While this multimodal CoS-practice makes transparent Kate's knowledge process of having excluded one of the two contradicting assumptions regarding the occasion of the nocturnal meeting (thereby adopting a potentially revised state of mind), it particularly follows the local conversational demands of treating the informing as successful and sufficient, enabling Jenny to finish the interactional work on the response.
In Excerpt 1, Kate's ach SO ('I see', l. 13), together with the eyebrow flash and the head lift, can be seen as a rather unobtrusive practice in terms of supporting the re-entry into the main strand of talk after having clarified the occasion of the nocturnal meeting at the fast-food restaurant. It is here produced in third position after an elicited informing, which is a typical sequential environment for moves deployed as visual resources within minimal news receipt-marking practices. Generally, we see in our data that the eyebrow move is also used as part of recipients' CoS-practice during multi-unit turns such as tellings or explanations. Here, it may encourage more talk, as it displays that one is cognitively following the information which is currently being delivered.
In all sequential positions, the whole multimodal gestalt appears to stay minimal, thereby enabling interlocutors to either follow up on or return to the main strand of talking. Therefore, we see the eyebrow move as part of minimal news receipt practices exhibiting a mainly retrospective orientation while being in the service of the progressivity of talking. As part of minimal responsive or recipient practice, eyebrow moves may give interlocutors an insight into the producer's processing of new information, but they do not actively intervene in the further sequential progress. Moreover, Excerpt 1 shows that RBE is deployed in a spatial constellation in which facial resources cannot be perceived by the interlocutor: during the whole sequence, there is no mutual gaze between Jenny and Kate. This observation is in line with findings of Schoonjans (2018), who describes systematic co-occurrences of modal particles and manual gestures as multimodal constructions.
5.2 The eyebrow hold for displaying surprise or astonishment
Instead of being embedded in a subtle practice of news receipt (as seen in Excerpt 1), the eyebrow hold goes beyond signalling unproblematic understanding and the receipt of news. Rather, it operates as an essential part in displaying an affective change of state. Following Heritage's (1984) view on CoS-markers, we see eyebrow holds as markers of ritualized surprise (or astonishment), and consequently as bound to the conversational demand of treating the information received as remarkable (Marmorstein & Szczepek Reed, 2023). This interactional relevance can have been set up in the previous turn(s) or in the larger activity context, but it can also be reflexively established through the CoS-practice itself: that is, treating something as being astonishing/surprising means turning a piece of information into astonishing/surprising news.
The following example presents a case in which a response to question-elicited information manifests in such a way that the less knowing person displays surprise when being provided with the required information. Monica (Mnc), Julia (Jul), Peter (Ptr) and Bernd are having a games evening together. Besides the activity of playing a game called TAC, the interlocutors update each other about their kids and their daily lives. At the beginning of the excerpt, a new topic, 'holiday destination', is initiated by Monica, who asks an information question: wohin fahrt ihr jetzt im SOMmer ('where will you go in summer', l. 1) directed to Peter and Julia while she is having a bite of a salty stick.
Extract 2. An den Atlantik / To the Atlantic'
Peter and Julia both start to answer Monica's question in overlap (l. 3 and l. 4). When Peter has finished his answer – specifying the date rather than the destination of the holidays (wir fahren im sepTEMber, 'we're going in September' l. 3) – Julia continues informing Monica that they are planning to stay on the Atlantic coast for four weeks (l. 5). In overlap with the information about the destination, Monica utters woHIN nach ('where', l. 6), thereby contextualising that Peter and Julia have indicated only the duration and time of a trip so far and holding them accountable for not answering the original question. Immediately afterwards, Monica produces a lengthened, high and level pitched !E:CHT! ('really', l. 7) as a response to Julia's informing. Aligned with this newsmark, Monica raises her eyebrows (, l. 7/B) and holds them raised (, l. 7/B), simultaneously widening her eyelids (, l. 7/E). Monica's multimodally produced 'assertion of ritualized disbelief' (Heritage, 1984: 339) ratifies the elicited information about the holiday destination as fitting her displayed lack of knowledge but moreover treats it as unexpected and surprising. Therefore, she keeps track of this topical agenda by posing a follow-up question when she queries the exact location of the destination. Thereby, she not only invites elaboration as described for German echt (Gubina & Betz, 2021), but actively takes the lead in the following line of sequences and evokes on-topic talk (Schegloff, 2007: 155) by posing a follow-up question (wieder nach 'again to', l. 9).
In sum, Excerpt 2 shows that lexical choice (news mark echt 'really'), marked prosody (Selting, 1995) with high level pitch and greater volume, and the facial resources of holding raised eyebrows and widened eyes intertwine while displaying emotional involvement (surprise) as a consequence of sudden awareness. The holding of RBE plays an essential part in this multimodal CoS-practice; its precise temporal synchronisation with the other verbal, vocal and bodily resources make the surprise display accountable to the recipients. In contrast to the eyebrow move as part of unobtrusive CoS-practices complying with conversational demands of marking the receipt of information and acknowledging their newsworthiness (while enabling either sequence closure and/or the continuation of a larger turn), the salient CoS-practice including the eyebrow hold establishes a conversational focus to elaborate on. This can also apply to informings, such as Julia's in line 5, which are not at all designed as transmitting something surprising or astonishing in any manner.
In contrast, the next two excerpts, 3.1 and 3.2, illustrate how an overarching conversational demand of appreciating information as being astonishing is complied with by using the eyebrow hold. Excerpts 3.1 and 3.2 follow each other. They are taken from a cooking event in which an expert (the Consultant, Thm) explains to the participants how to use a special cooking machine. Part of the event is that all attendees eat the dinner they have cooked beforehand while the expert tells them how and what to cook using the machine (see Figure 2). The transcript contains the representation of head movement (line H), brow movement (line B) and movement of the lips (line L).
Figure 2. Cooking event
Extract 3.1 Seit wann Thermomix / Since when thermomix' (Part 1)
During the parallel activities of having dinner and receiving explanations about the cooking machine, Anna (Ann) follows on from the expert's explanations with the resumption marker und 'and'. She produces a request for information about the year of invention directed to the expert, Thm (l. 1–2), and provides a candidate answer (von diesem jahr 'from this year', l. 3). In her response, the expert rejects the candidate answer (l. 5) and informs Anna that the machine had already been invented in 1984 (l. 6). While continuously gazing at the expert, Anna reacts to the informing with a hold of both eyebrows (, l. 7/B) and a slight raising of her head (, l. 7/H) while opening her mouth in the form of a silent oh (, l. 7/L) as a visual stand-alone practice. By raising her eyebrows and holding them at the peak of the movement, Anna displays astonishment when confronted with the long period of time the machine has already existed, which clearly stands in contrast to her prior expectations. Her astonishment about the year of invention is consistent with the larger activity context: the whole event is framed by testing and watching the extraordinary functions of the cooking machine, which are highlighted and appreciated again and again while cooking and having dinner.
Extract 3.2 Seit wann Thermomix / Since when thermomix' (Part 2)
Open in a separate window When the expert takes up Anna's response by giving an additional piece of information (das ist die vierte generation 'this is the fourth generation', l. 8), Anna raises her eyebrows again from a half-raised position in the form of an eyebrow flash (, l. 9/B). This short move is accompanied by the verbal astonishment-marker wow which is produced in a whispering voice (l. 9). This second response to the elaboration of the expert's response provides an interesting case, since Anna verbally treats the informing as astonishing and – at the same time – subsequently withdraws this display while treating the additional piece of information. The eyebrow flash as a visual marker of news receipt and a temporal gaze aversion combined with the whispered particle wow thus follow the contradicting requirements of displaying continuing astonishment and having received sufficient information. However, it takes another TCU by the expert (l. 10/11), which is responded to by a head nod only (, l. 12/H), before Anna is able to withdraw from being the prior addressee. Directly afterwards, Anna turns her head (, l. 13/H), bringing it back to neutral position (, l. 13/H) towards her plate and continues eating. What is apparent in this instance of RBE within Excerpt 3.2 is that verbal and embodied resources may diverge with regard to their CoS-quality and the conversational work they are doing in favour of either closure or expansion. While verbally following the conversational demands of being interested and astonished, Anna subsequently reduces her visual CoS marking during the elaborate response for displaying that her enquiry has been clarified sufficiently.
6. Summary and Conclusion
By applying the methodological repertoire of Multimodal Conversation Analysis to mundane conversations, we have outlined that RBE constitutes an example of how facial movements serve as facial gesture accomplishing specific interactional functions in conversational moments of receiving new information. Specifically, RBE operates as a frequent visual change-of-state marker: by using it as stand-alone gesture, embedded in a purely embodied gestalt (e.g. including a head lift) or aligned with verbal CoS-tokens, interlocutors treat the foregoing talk as new, informative, surprising or astonishing.
We have shown how different qualities of an interlocutor's sudden changed state of mind in the face of a new occasion or information are constructed and contextualised by the use of two distinct patterns of the raising of both eyebrows: the move and the hold.
The eyebrow move is used to mark news receipt, operating mainly retrospectively and thereby enabling sequence closure and/or the continuation of a main strand of talk. As recipient activity, the move is deployed for displaying ongoing attention and the unproblematic receiving of some piece of information. It is generally oriented towards the prior action of a news delivery as being completed successfully and gives priority to the speaker's agenda.
In contrast, the eyebrow hold (frequently accompanied with bodily resources like open mouth, widened eyes and diverse verbal news marks and prosodic emphasis) appears as an essential part of a typical facial surprise display (Heath et al., 2012), contextualising that the information received is remarkable. Thus, it is not RBE in general that displays surprise as Ekman suggests, but only the hold as one specific manifestation involving a (more or less extended) brief moment in time. Moreover, displaying surprise is not independent of conversational demands. Our investigation is in line with previous studies which have already shown that the emotive potential of a specific facial configuration does not emerge in a social vacuum; rather, it is produced in the service of local conversational requirements, such as affiliating to some piece of information being presented as newsworthy and surprising. We have shown that a hold of the eyebrows constitutes an affiliative move in the sense that being astonished is established as expectation within the larger activity context. Furthermore, the multimodal surprise display involving RBE is adapted to the question of whether to close down or continue with an ongoing topic. Regarding this, the hold may be used for introducing a new affective stance while turning the mentionable into a surpriseable while the producer might actively take the lead on the ensuing conversational path. The hold then iconically holds the local conversational activity for 'zooming in' to the surprising piece of information and for (re)directing the conversational path.
Together with other resources, the hold forms the multimodal gestalt of an affective CoS-marker (Gudmundson & Svennevig, 2020), which itself is maximally salient, involving the hold of maximally raised eyebrows, widened eyes, head lift and sometimes open mouth or horseshoe mouth. We suspect that the salience of the hold (compared to a move as a comparably minimal news receipt practice) facilitates the recognition of the whole practice as such, that is, its accountability, which parallels the newsworthiness of the information. It supports the activity of jointly producing and reacting to surpriseables (see also Heath, 2012: 216).6 This can also be deduced by the presentation of the hold to the interlocutor: in the examples given above, the facial configuration of the surprise display including the hold is produced during mutual gaze, and it is actively rendered visible through body orientation towards the recipient.
However, in other cases of our collection not only the eyebrow move but also the hold – together with other elements of the CoS-face – are produced in the absence of mutual gaze or clear visibility for others, particularly in the side-by-side constellation of the car journey interaction. Hence, while we can see in the examples above that the surprise display involving the eyebrow hold – on the one hand – clearly serves as being publicly available in order to visibly comply with conversational demands and engage in more talk about the news, it appears – on the other hand – as a solidified facial configuration in the sense of a multimodal construction (e.g. Schoonjans, 2018). In order to corroborate our assumption, future studies could investigate, on the basis of larger collections, whether systematic co-occurrences of specific CoS-tokens and manifestations of RBE are identifiable and if their use is related to visual availability for the interlocutor.
Our results further suggest that the division between news marks and mere news receipts/change-of-state markers not only seems to be a fluid one (Weber & König, 2023), but also has to be reconsidered multimodally, as verbal and embodied practices of information receiving and news marking may intertwine: specifically, elements of verbal news marking (e.g. wow, echt 'really') which have been related to sequence expansion (however, see Gubina & Betz, 2021) may be accompanied by embodied practices which are instead related to sequence (and topic) closure (such as the eyebrow move or gaze aversion). Verbal and visual resources thus might implement contradicting conversational functions of appreciating information as being astonishing/surprising versus – at the same time – displaying that the received information is sufficient.
Acknowledgements
We are grateful to the participants for their permission to let us record the video material and analyse and publish selected parts. We would also like to thank Karin Birkner and the two anonymous reviewers for extensive and helpful comments to a previous version of this paper. All remaining errors are our own.
References
Ambrazaitis, G. & House, D. (2017). Multimodal prominence: Exploring the pattern and usage of focal pitch accents, head beats and eyebrow beats in Swedish television. Speech Communication, 95, 100-113.
Andries, F.; Meissl, K.; de Vries, C.; Feyaerts, K.; Oben, B.; Sambre, P.; Vermeerbergen, M. & Brône, G. (2023). Multimodal stance-taking in interaction – A systematic literature review. Frontiers in Communication, 8, 1187977.
Auer, P. (2005). Projection in Interaction and Projection in Grammar. Text, 25(1), 7-36.
Barth-Weingarten, D.; Couper-Kuhlen, E. & Deppermann, A. (2020). Konstruktionsgrammatik und Prosodie. OH in englischer Alltagsinteraktion. In Imo, W. & Lanwer, J. P. (Eds.), Prosodie und Konstruktionsgrammatik. deGruyter, 35-73.
Bavelas, J. & Chovil, N. (2018). Some pragmatic functions of conversational facial gestures. Gesture, 17(1), 98–127.
Bavelas, J.; Gerwing, J. & Healing, S. (2014a). Including facial gestures in gesture-speech ensembles. In Seyfeddinipur, M. & Gullberg, M. (Eds.), From Gesture in Conversation to Visible Action as Utterance: Essays in honor of Adam Kendon. John Benjamins, 15–34.
Bavelas, J.; Gerwing, J. & Healing, S. (2014b). Hand and Facial Gestures in Conversational Interaction. In Holtgraves, T. M. (Eds.), The Oxford Handbook of Language and Social Psychology. Oxford University Press, 111–130.
Beskow, J., Granström, B. & House, D. (2009). Focal Accent and facial movements in expressive speech. Working Papers Lund University, 52, 9–12.
Betz, E. & Golato, A. (2008). Remembering relevant information and withholding relevant next actions: The German token achja. Research on Language and Social Interaction, 41(1), 58-98.
Bublitz, W. & Kühn, P. (1981). Aufmerksamkeitssteuerung. Zur Verstehenssicherung des Gemeinten und des Mitgemeinten. Zeitschrift für Germanistische Linguistik, 9(1), 55–76.
Chovil, N. (1991/1992). Discourse-Oriented Facial Displays in Conversation. Research on Language and Social Interaction, 25, 163–194.
Couper-Kuhlen, E. & Barth-Weingarten, D. (2011). A system for transcribing talk-in-interaction: GAT2. Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion, 12, 1–51.
Crespo Sendra et al. (2013). Perceiving incredulity. The role of intonation and facial gestures. Journal of Pragmatics, 47, 1–13.
Cruz, M.; Swerts, M. & Frota, S. (2017). The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties. Laboratory Phonology: Journal of the Association for Laboratory Phonology, 8(1), 1–24.
Darwin, C. (2007). The expression of emotion in man and animals. Filiquarain Publishing. (Original work published 1872).
Dix, C. (2022). GAT 2 trifft das International SignWriting Alphabet (ISWA): Ein neues System für die Transkription von Multimodalität. In: Schwarze, C. & Grawunder, S. (Eds.), Transkription und Annotation gesprochener Sprache und multimodaler Interaktion: Konzepte, Probleme, Lösungen. Narr, 103–131.
Ekman, P. (1979). About brows. Emotional and conversational signals. In Cranach, M. von et al. (Eds.), Human ethology. Claims and limits of a new discipline. Cambridge University Press, 169–202.
Ekman, P. & Friesen, W. V. (1969). The Repertoire of Nonverbal Behavior. Categories, Origins, Usage, and Coding. Semiotica, 1(1), 49–98.
Ekman, P.; Friesen, W. V. & Hager, J. C. (2002). Facial Action Coding System: an ebook for pdf readers. a Human Face.
Ekman, P. & Oster, H. (1979). Facial expressions of emotion. Annual Review of Psychology, 30, 527–554.
Flecha-Garcia, M. (2010). Eyebrow raises in dialogue and their relation to discourse structure, utterance function and pitch accents in English. Speech Communication, 52, 542–554.
Goffman, E. (1981). Forms of talk. University of Pennsylvania Press and Blackwell.
Golato, A. (2010). Marking understanding versus receipting information in talk: Achso and ach in German interaction. Discourse Studies, 12(2), 147–176.
Golato, A. (2012). German oh: Marking an Emotional Change of State. Research on Language and Social Interaction, 45(3), 245–268.
Golato, A. & Betz, E. (2008). German ach and achso in repair uptake: Resources to sustain or remove epistemic asymmetry. Zeitschrift für Sprachwissenschaft, 27, 7–37.
Granström, B. & House, D. (2005). Audiovisual representation of prosody in expressive speech communication. Speech Communication, 46, 473–484.
Guaïtella, I.; Santi, S.; Lagrue, B. & Cavé, C. (2009). Are eyebrow movements linked to voice variations and turn-taking? An experimental investigation. Language and Speech, 52(2/3), 207–222.
Gubina, A. & Betz, E. (2021). What do News mark-Type responses invite? The Response Space After German echt. Research on Language and Social Interaction, 54(4), 374–396.
Gudmundsen, J. & Svennevig, J. (2020). Multimodal displays of understanding in vocabulary-oriented sequences. Social Interaction – Video-based studies of Human Sociality, 3(2).
Heath, C.; vom Lehn, D.; Cleverly, J. & Luff, P. (2012). Revealing surprise. The local ecology and the transposition of action. In Peräkylä, A. & Sorjonen, M. L. (Eds.), Emotion in Interaction. Oxford University Press.
Helmer, H.; Betz, E. & Deppermann, A. (2021). Coordination of OKAY, nods, and gaze in claiming understanding and closing topics. In: Betz, E. et al. (Eds.), OKAY across Languages: Toward a comparative approach to its use in talk-in-interaction. John Benjamins: 364–393 (=Studies in Language and Social Interaction 34).
Heritage, J. (1998). Oh-prefaced Responses to Inquiry. Language in Society, 27, 291–334.
Heritage, J. (1984). A change-of-state token and aspects of its sequential placement. In: Atkinson, M. & Heritage, J. (Eds.): Structures of Social Action. Studies in Conversation Analysis. Cambridge University Press, 299–345.
Hilmisdóttir, H. (2016). Responding to informings in Icelandic talk-in-interaction: A comparison of nú and er pað. Journal of Pragmatics, 104, 133–147.
Imo, W. (2009). Konstruktion oder Funktion? Erkenntnisprozessmarker („change-of-state tokens“) im Deutschen. In Günthner, S. & Bücker, J. (Eds.), Grammatik im Gespräch. Konstruktionen der Selbst- und Fremdpositionierung. deGruyter, 57–86.
Kärkkäinen, E. & Keisanen, T. (2012). Linguistic and embodied formats of making (concrete) offers. Discourse Studies, 14(5), 587–611.
Kaukomaa, T.; Peräkylä, A., & Ruusuvuori, J. (2014). Foreshadowing a problem: Turn-opening frowns in conversation. Journal of Pragmatics, 71, 132–147.
Koivisto, A. (2016). Receipting information as newsworthy vs. responding to redirection: Finnish news particles aijaa and aha(a). Journal of Pragmatics, 104, 163–179.
Kupetz, M. (2015). Empathie im Gespräch. Eine interaktionslinguistische Perspektive. Stauffenburg.
Marmorstein, M. & Szczepek Reed, B. (2023). News marks as an interactional resource for indexing remarkability: a qualitative analysis of Arabic wallāhi and English really. Contrastive Pragmatics, 1–37.
Maynard, D. (1997). The news delivery sequence: Bad news and good news in conversational interaction. Research on Language & Social Interaction, 30(2), 93–130.
Mondada, L. (2011). Understanding as an embodied, situated and sequential achievement in interaction. Journal of Pragmatics, 43, 542–552.
Mondada, L. (2019). Contemporary issues in Conversation Analysis. Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics, 145, 47–62.
Nota, N.; Trujillo, J. P. & Holler, J. (2021). Facial Signals and Social Actions in Multimodal Face-to-Face Interaction. Brain Sciences, 11, 1017.
Oloff, F. (2018). ”Sorry?!/”Como?”/”Was?” – Open Class and Embodied Repair Initiators in International Workplace Interactions. Journal of Pragmatics, 126, 29–51.
Parkhurst, S. & Parkhurst, D. (2008). A cross-linguistic guide to SignWriting. A phonetic approach. available online: http://www.signwriting.org/archive/docs7/sw0617_Cross_Linguistic_Guide_SignWriting_Parkhurst.pdf (retrieved 26th Jul 2023).
Rossi, G. (2020). Other-repetition in conversation across languages: Bringing prosody into pragmatic typology. Language in Society, 49(4), 495–520.
Schegloff, E. (2007). Sequence Organization in Interaction. A Primer in Conversation Analysis I. Cambridge: Cambridge University Press.
Schoonjans, S. (2018). Modalpartikeln als multimodale Konstruktionen. Eine korpusbasierte Kookkurenzanalyse von Modalpartikeln und Gestik im Deutschen. De Gruyter.
Selting, M. (1995). Prosodie im Gespräch. Aspekte einer interaktionalen Phonologie der Konversation. Niemeyer.
Selting, M. et al. (2009). Gesprächsanalytisches Transkriptionssystem 2 (GAT2). Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion, 10, 353–390. Retrievable online: http://www.gespraechsforschung-online.de/heft2009/heft2009.html (retrieved 26th Jul. 2023).
Sorjonen, M. L. (2001). Responding in Conversation. A study of response particles in Finnish. John Benjamins.
Sutton, V. (2010). The SignWriting Alphabet. Read and Write any Sign Language in the World. ISWA Manual 2010. The SignWriting Press. Retrievable Online: http://www.movementwriting.org/symbolbank/ (retrieved 26th Jul 2023).
Swerts, M. & Krahmer, E. (2008). Facial expression and prosodic prominence: Effects of modality and facial area. Journal of Phonetics, 36, 219–238.
Swerts, M. & Krahmer, E. (2010): Visual prosody of newsreaders: Effects of information structure, emotional content and intended audience on facial expressions. Journal of Phonetics, 38(2), 197-206.
Thompson, S. A.; Fox, B. A. & Couper-Kuhlen, E. (2015). Grammar in everyday talk: Responsive actions in conversation. Cambridge University Press.
Wierzbicka, A. (2000). The Semantics of Human Facial Expressions. Pragmatics & Cognition 8(1), 147–183.
Weber, K. & König, K. (2023). Responding to Informings in German and Low German. Presentation at IPrA conference. Brussels, 9th¬–14th July 2023.
Wilkinson, S. & Kitzinger, C. (2006). Surprise as an Interactional Achievement: Reaction Tokens in Conversation. Social Psychology Quarterly 69(2), 150–182.
Appendix. Transcription symbols
Symbols of the Gesprächsanalytisches Transkriptionssystem (GAT2; Selting et al. 2009) for the representation of verbal conduct:
WOhin? | main accent |
OH:::: | stretching of sounds |
? | raising intonation |
, | half raising intonation |
- | level intonation |
; | half falling intonation |
. | falling intonation |
[ ] | overlap / simultaneous speech |
( ) | incomprehensible utterance |
(.) | micropause |
(-) | short pause up to 0.5 sec |
(--) | short pause up to 0,8 sec |
(1.0) | longer pause with specific duration |
<<p>wow> | piano (low voice) |
<<p>wow> | pianissimo (very low voice) |
brows in raised position | |
eyelids widened | |
eyes open | |
eyes closed | |
mouth rounded, open | |
upward movement of both brows | |
downward movement of both brows | |
head in straight position | |
movement to the right |
B | Brows |
E | Eyes/Eyelids |
H | Head |
L | Lips |
1 Unlike other studies which contrast news marking and change-of-state marking, we use the term change of state in a broader sense, which also comprises noticeable changes in affective stance. Therefore, by using the term change-of-state marker, we may also refer to news marks.↩
2 Translated and adapted for English by Couper-Kuhlen & Barth-Weingarten (2011).↩
3 See Section 8 for an overview of the conventions of transcription used in this article.↩
4 It must be noted that we did not encounter cases of the raising of just one eyebrow (the left or the right). In contrast, we observed that frowns (the lowering of both brows) also occurred in the sequential environments of CoS-moments, which we excluded from analysis assuming that frowns do other things than RBE (see e.g. Kaukomaa et al., 2014).↩
5 In this contribution, further functions of CoS-marking involving RBE which we encountered during data analysis will not be at the centre of our analysis, which is concerned with assessing the new information, displaying a deeper and/or revised understanding or the remembrance of lost memories, for instance, in response to explanations. While we found both eyebrow holds and moves to be involved here, the head lift seems to be even more essential for performing these functions, as Mondada (2011) and Gudmundsen and Svennevig (2020) have already observed in instructional activities. RBE instances in Excerpts 1 and 3.1, however, also involve displaying the revision of former claims as functional shades.↩
6 This also applies to instances in our collection in which the multimodal surprise display is not only used as a response but also as a part of multimodal noticing practices while the producer gazes at the noticeable. Due to space limitations, we could not provide an example in which the hold is used as practice of noticing extra-conversational events (see e.g. Heath et al., 2012). ↩