HAL Id: hal-02273492
https://hal.archives-ouvertes.fr/hal-02273492
Submitted on 29 Aug 2019
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Orofacial Somatosensory Effects for the Word Segmentation Judgement
Rintaro Ogane, Jean-Luc Schwartz, Takayuki Ito
To cite this version:
Rintaro Ogane, Jean-Luc Schwartz, Takayuki Ito. Orofacial Somatosensory Effects for the Word
Segmentation Judgement. ICPhS 2019 - 19th International Congress of Phonetic Sciences (ICPhS
2019), Aug 2019, Melbourne, Australia. 2019. �hal-02273492�
Orofacial Somatosensory Effects
for the Word Segmentation Judgement
Rintaro Ogane
1, Jean-Luc Schwartz
1, Takayuki Ito
1,21
Univ. Grenoble Alpes, CNRS, Grenoble INP*, GIPSA-lab, Grenoble, France,
2Haskins Laboratories, New Haven, USA
* Institute of Engineering Univ. Grenoble Alpes
Introduction
Results
Discussion
This research was supported by the European Research Council under the European Community’s Seventh Framework Programme (FP7/2007-2013 Grant Agreement no. 339152, “Speech Unit(e)s”).
We also thank Nathan Mary and Dorian Deliquet for data collection and Silvain Gerber for statistical analysis.
Acknowledgments
Somatosensory inputsassociated with facial skin deformationsystematically biased the perceptual judgementsdepending on the stimulation onset timing, but not depending on the amplitude differences.
Orofacial somatosensory systemcould be involved in the process of lexical perception concerning word segmentation.
Summary
Methods
auditory processing system auditory-visual interaction
somatosensory information
visual information
phoneme syllable lexicon sentence
auditory-somatosensory interaction American English vowels
(Ito et al., 2009)
CV sequences (Gick & Derrick, 2009)
How does somatosensory inputaffect the processing of speech ? Do somatosensory inputsassociated
with facial skin deformationmodify the lexical perception?
Speech perceptionis an interactive processwith multiple modalities and some perceptuo(multisensory)-motor connections (Schwartz et al., 2012).
leads the first vowel corresponds to the second vowel
Significant effect of stimulation timing (χ² (8) = 39.04, p< 0.01).
−Pt3 << Pt2, Pt5, Pt6, Pt7 and Pt8 (p< 0.03).
−Pt0 >> Pt3 and Pt4 (p< 0.04).
Experimental setup (Ito et al., 2009)
Extend the validity of somatosensory effectsin speech processing.
Corpus : 17 French phrases lead elision.
−Phrase examples.
“l’apesanteur” - “la pesanteur” (/lapœzɑ̃tœʀ/).
“l’attache” - “la tache” (/lataʃ/).
“l’affiche” - “la fiche” (/lafiʃ/).
−3 speaking accent styles.
Sa0 : neutral & natural speech.
Sa1 : accented on the 1st vowel, /lataʃ/.
Sa2 : accented on the 2nd vowel, /lataʃ/.
A directional relationship between facial stimulation& articulatory gesture.
−The phrases were divided into 4 groups based on the 2nd vowel in the phrase.
Participants : 31 native French speakers.
−20 for Exp. 1 and 11 for Exp. 2.
Do amplitude differences of somatosensory inputs provide different effects in lexical perception ?
Identification test.
−Task : to identify which word you heard ?
−The judgement probability for “la _____”
(e.g. “la tache”) responses.
−Stats : Linear Mixed-Effects Model analysis.
Exp. 1
Exp. 2
“l’attache” vs. “la tache” ?
Pronunciation : /lataʃ/
Meaning : “the string” vs. “the stain”
Experiment 1 Experiment 2
Sa0 Sa0, Sa1, Sa2
Pt1-Pt8 & no stimulation condition (Pt0) Pa0, Pa1, Pa2
Experiment conditions for each experiment.
Somatosensory stimulation.
−Upward direction.
−A half-wave 6 Hz sinusoidal pattern.
speech production speech perception
phoneme (Ito et al., 2009)
syllable (Gick & Derrick, 2009)
sentence motor
somatosensory
lexicon
Somatosensory effectsin lexical perception seem compatible with vertical articulatory gesture(Vatikiotis-Bateson et al., 1999). somatosensory inputs
Exp. 1
No effect of stimulation amplitude, alone (χ² (2) = 0.06, p> 0.97) or in interaction with accent (χ² (4) = 0.39, p> 0.98).
Significant effect of speaking accent (χ² (2) = 179.97, p< 0.01).
−Sa1 << Sa0, Sa1 << Sa2, Sa0 << Sa2 (p< 0.01).
lexical perception
articulatory gesture in speech sounds Amplitude differences
Exp. 2
facial stimulation vertical
spreading (e.g. /lafiʃ/) rounding
(e.g. /lapœzɑ̃tœʀ/) vertical (e.g. /lataʃ/)
(front) (back)
Stimulation onset timing
verticalN = 5 spreadingN = 5
rounding N = 7 horizontal N = 12
horizontal (rounding + spreading) Contact : [email protected]
McGurk effects (McGurk & MacDonald, 1976).
Word segmentation (Sell & Kaschak, 2009).
Lexical processing in French (Strauß et al., 2015).
Speech detection in noise (Sumby & Pollack, 1954; Erber, 1969; Grant & Seitz, 2000; Bernstein et al., 2004; Kim & Davis, 2004).
verticalN = 5 spreadingN = 5
roundingN = 7 horizontal N = 12
Temporal relationships.
