Multimodal Prosodic Phrasing in Infant-Directed Speech: Testing the Cumulative-Cue Hypothesis with Gesture Restriction

Abstract

Human communication is inherently multimodal; the relation between speech and co-speech gestures is well established and facilitates face-to-face interaction. However, the underlying cognitive mechanisms remain unclear. Important insights emerge from research on modality disruption and the Cumulative-Cue Hypothesis proposed for prosodic prominence. Based on the Cumulative-Cue Hypothesis and modality-disruption studies, speakers are expected to redistribute communicative effort within the visual modality and only enhance acoustic cues in cases of additional effort. By investigating the effects of hand gesture restriction on acoustic and visual prosodic phrasing cues in infant-directed speech, the Hypothesis can be extended, offering a broader framework for understanding human communication. Given its engaging and effortful nature, infant-directed speech may be particularly susceptible to modality disruptions, providing a suitable test context. Interactions between three German tutors and 12 Dutch infants (aged 4–5 months and 8–9 months) were recorded in two conditions: Hands Free and Hands Restricted. A total of 732 segmented intonational phrases and 61,151 corresponding video frames were analyzed to compare the tutors’ acoustic cues (pitch maximum, pitch minimum, final syllable duration, pause duration) and visual cues (eyebrow movement frequency and intensity) at final intonational phrase boundaries between conditions. Under hand restriction, eyebrows were raised more frequently, but only during interactions with younger infants. Acoustic cues remained unaffected by hand restriction for both age groups. The finding for the younger group supports the Cumulative-Cue Hypothesis and suggests it extends to prosodic phrasing. The age-related differences point to changes in communicative intent and speakers’ adaptability to infants’ developmental needs, yet they complicate the generalization of the findings and warrant further research.