Stronger neural response to canonical finger‐number configurations in deaf compared to hearing adults revealed by FPVS‐EEG

Authors: Margot Buyle, Aliette Lochy, Valentina Vencato, Virginie Crollen

Abstract

The linguistic counting system of deaf signers consists of a manual counting format that uses specific structures for number words. Interestingly, the number signs from 1 to 4 in the Belgian sign languages correspond to the finger‐montring habits of hearing individuals. These hand configurations could therefore be considered as signs (i.e., part of a language system) for deaf, while they would simply be number gestures (not linguistic) for hearing controls. A Fast Periodic Visual Stimulation design was used with electroencephalography recordings to examine whether these finger‐number configurations are differently processed by the brain when they are signs (in deaf signers) as compared to when they are gestures (in hearing controls). Results showed that deaf signers show stronger discrimination responses to canonical finger‐montring configurations compared to hearing controls. A second control experiment furthermore demonstrated that this finding was not merely due to the experience deaf signers have with the processing of hand configurations, as brain responses did not differ between groups for finger‐counting configurations. Number configurations are therefore processed differently by deaf signers, but only when these configurations are part of their language system.

INTRODUCTION

Humans possess a grasp of quantitative concepts that has presumably developed independently of, and before, language (Butterworth, 1999; Carey, 1998; Dehaene, 1997; Dehaene, 2001; Wynn, 1998). This ability is considered as the root of the approximate representation of non‐symbolic quantity and has been attributed to the so‐called approximate number system (ANS) (Barth et al., 2008; Barth et al., 2006; Barth et al., 2005; Dehaene, 2011; Feigenson et al., 2004). Besides this non‐symbolic representation of numbers, a variety of specific numerical codes such as number words (one, two, three, etc.) and Arabic numerals (1, 2, 3, etc.) has also to be mastered. This abstract learning is strongly anchored in our language system (Miller et al., 1995), and requires children to progressively move from concrete mathematical skills based on physical objects towards a more symbolic mathematical ability focused on numerals (Kolkman et al., 2013).

Over the last decades, it has been assumed that canonical finger configurations could facilitate this process by providing concrete referents to abstract number symbols (Butterworth, 1999; Gunderson et al., 2015; Soylu et al., 2018; Van den Berg et al., 2022). These canonical finger configurations include finger‐counting and finger‐montring gestures, which are typically used in a given culture to respectively count and show numerosities with the fingers. Finger‐counting and finger‐montring therefore differ in terms of purposes (i.e., counting for finger‐counting versus communicating a quantity for finger‐montring), but also differ in terms of handshape specificities. In many European countries, the thumb and the index fingers are, for example, successively raised to count to two (finger‐counting), while the index and middle fingers are simultaneously raised to communicate about the same quantity (finger‐montring). Despite these specificities, finger‐counting and finger‐montring nevertheless both allow a direct recognition of the magnitude they represent (Di Luca & Pesenti, 2008). Support for this latter assumption can be found (1) behavioural studies showing that canonical finger configurations are more efficiently enumerated and processed than other (non‐canonical) finger configurations (Di Luca et al., 2006; Di Luca & Pesenti, 2008; Lafay et al., 2013; Marlair et al., 2021; Noël, 2005; Soylu et al., 2019; van den Berg et al., 2021; Van den Berg et al., 2022), and (2) brain research demonstrating that canonical finger configurations induce higher activation of occipital and parietal brain regions (i.e., the identified brain substrate of number‐selective (S. Dehaene & Cohen, 2007; S. Dehaene et al., 2003; Nieder & Dehaene, 2009)) than non‐canonical representations (Marlair et al., 2021; Proverbio & Carminati, 2019; Soylu et al., 2019; van den Berg et al., 2021, Van den Berg et al., 2022).

Due to their frequent use in early number learning and daily communication, culture‐specific number gestures therefore become semantically associated to other (non‐symbolic and abstract) number representations (van den Berg et al., 2021). These associations are weaker with non‐canonical number gestures (due to their non‐familiarity), but may in contrast increase when finger configurations are embedded in a language system such as sign language. In Belgium, the number signs from 1 to 5 interestingly correspond to the finger‐montring habits of hearing individuals (see Figure 1). As research has shown that deaf signers develop specific cognitive processes for perceiving fingers and hand configurations (Baker et al., 2005; Emmorey et al., 2003; Muir & Richardson, 2005), we could therefore hypothesise that deaf signers will process these configurations differently than hearing controls.

To investigate this question, brain responses to number gestures were compared in deaf signers and hearing controls with a highly sensitive and implicit method combining frequency‐tagging and electroencephalography (EEG) (i.e., Fast Periodic Visual Stimulation, FPVS) (Retter & Rossion, 2016; Rossion, 2014). FPVS consists in presenting a category of stimuli at a frequency X, which triggers a general visual response in the brain, measurable at exactly that frequency X. Within this periodic presentation, another category of stimuli (deviant) is inserted at a frequency Y. If the deviants are discriminated, then a measurable neural response occurs at the frequency of stimulation Y for this category of stimuli. This method provides an objective marker of category discrimination and has already been successfully applied to the discrimination of visual quantities (Guillaume et al., 2018; Marlair et al., 2021), digits grouped on the basis of parity (odd/even) or magnitude (small/big) (Guillaume et al., 2020; Marinova et al., 2021), or digits among letters (Lochy & Schiltz, 2019). It has also been successfully used to investigate visual processing and brain reorganisation in deaf individuals (Benetti et al., 2017; Bottari et al., 2020; Gwinn & Jiang, 2020; Retter et al., 2019; Stroh et al., 2022) and is therefore particularly adapted to unravel changes associated with number categorizations in deaf signers.

The present study included two experiments, both of them (1) non‐canonical finger‐number configurations at 6 Hz as base stimuli, and (2) canonical finger‐number configurations as deviant stimuli at 1.2 Hz (see Figure 2). Sign language configurations (i.e., signs for deaf signers but finger‐montring gestures for hearing controls) were the deviant stimuli of Experiment 1, while finger‐counting configurations (i.e., finger‐counting gestures for both deaf and hearing participants) were the deviant stimuli of Experiment 2. If the linguistic aspect (i.e., signs) of canonical hand configurations elicits a more salient processing, then deaf signers should show stronger discrimination responses or different response topographies than the hearing controls in Experiment 1. No group differences should in contrast emerge in Experiment 2 as the finger configurations used as deviant stimuli correspond to finger‐counting gestures in hearing controls as well as in deaf signers.

EXPERIMENT 1

Methods

Participants

Two groups of Dutch and French‐speaking adults were a group of 21 congenitally deaf signer adults (9 males, 10 French, Mage = 39.1 years ± 2.92), and a control group of 21 hearing adults who did not know sign language (9 males, 10 French, Mage = 38.8 years ± 3.15) (see Table 1 for a detailed description of the participants). All participants were recruited in Belgium and had normal or corrected‐to‐normal vision and no neurological problems. Hearing controls were matched to deaf participants for age (F(1, 40.0) = .006; p = .94, n ^2^ = .000); gender (X ^2^ (1, 42.0) = .000, p = 1.00); handedness (X ^2^ (1, 42.0) = 1.11, p = .29); educational level (F(1, 39.0) = .58; p = .45, n ^2^ = .015); and mother tongue (French vs. Dutch) (X ^2^ (1, 42.0) = .000, p = 1.00). All deaf participants indicated to be fluent in sign language (M ~~age of acquisition~~ ± SE = 6.48 years ± 2.68) and reported it to be their preferred way of communication (for more details, see Supplementary Table 1). Both oral and written instructions in Dutch and in French were given. Additionally, instruction videos in sign language were presented to deaf participants. Participants provided their written informed consent and the procedures were in line with the Declaration of Helsinki. The study was approved by the “Comité d'Ethique hospitalo‐facultaire Saint‐Luc‐UCLouvain” (2019/19AOU/357).

Stimuli and procedures

Stimuli

Non‐canonical hand configurations (i.e., atypical number gestures) representing numbers 1 to 4 were used as the base category (see second line Figure 1), while canonical hand configurations of numbers from 1 to 4 (i.e., signs for deaf signers; typical finger‐montring gestures for hearing controls) were used as the deviant stimuli (see first line Figure 1). These two sets of stimuli consisted of 48 drawings of finger‐number configurations (including mirror images). Namely, four numbers (1 to 4) designed in three different drawings in original and in mirror orientation were used for the two types of stimuli (base vs. deviant stimuli) (see Figure 1).

FIGURE 1: Stimuli that were used in this experiment (finger‐montring 1 to 4). First line represents the canonical configurations. Second line represents the non‐canonical ones. Three different drawing designs (and their mirror images) were used for each numerosity.

Procedure

Each sequence consisted in 60 s of stimulation with an additional 2 s of gradual fade in at the beginning and 2 s of gradual fade out at the end of the sequence. Within each sequence, stimuli were presented at a constant frequency rate of 6 Hz (i.e., six images per second, 167 ms per image) by means of sinusoidal modulation of contrast from 0 to 100% (see Figure 2a). Non‐canonical number configurations were used as the base category presented at 6 Hz (see Figure 2b), and every fifth item (6/5 so at a 1.2‐Hz frequency) was a canonical number configuration (i.e., the deviant category). Different drawings of hand configurations were used to ensure that any resulting finding reflects generalization beyond specific visual features. The stimuli in each base/deviant category were randomly presented at the centre of the screen with no immediate repetition of the same stimulus.

Each sequence was repeated three times. A fixation cross appeared at the centre of the screen 2–5 s before the sequence started and stayed at the same position during the entire sequence. To maintain a constant level of attention throughout the stimulation, participants were instructed to focus on a fixation point and to detect brief task‐irrelevant colour‐changes by pressing the space bar (from blue to red, for 200 ms, 10 random colour changes per sequence).

The experiment was created and executed with a software running on JavaScript (Java SE Version 8, Oracle Corporation, USA). The experiment took place in a quiet, low‐lit room. During testing, the participant was seated in front of a table, on which a monitor with an 800 × 600 pixel resolution was placed at a distance of 1 m to display the task.

EEG data acquisition and analysis

EEG was recorded at 2048 Hz using the BioSemi Active II system (BioSemi, The Netherlands). Sixty‐four channels were positioned at the standard 10–20 system locations together with four additional posterior electrodes (PO9, I1, I2, PO10). The Common Mode Sense active electrode and the Driven Right Leg passive electrode were used as reference and ground electrodes, respectively. The magnitude of the offset of all electrodes was held below 50 mV.

EEG analyses were carried out using Letswave 5 (https://www.letswave.org/), which is an open‐source toolbox running on MATLAB (The MathWorks, USA). Data files were first resampled to 512 Hz to reduce analysis time, to then pre‐process the files using a Fast Fourier Transform (FFT) band‐pass filter with cut‐off values of .10–100 Hz, and an FFT multinotch filter to attenuate electrical noise at three harmonics of 50 Hz. The fade‐in and fade‐out periods were excluded from the analysis, and channels that showed a high level of artefacts or noise were interpolated (maximum 10% of the channels). Each sequence was then segmented again from stimulation onset until 60 s, in order to contain the largest amount of integer presentation cycles (50 cycles of 1.2 s at 1.2 Hz). Channels were re‐referenced to the common average of all electrodes and the signal from all repetitions of each condition per participant was averaged.

An FFT was computed to obtain the normalized amplitude spectrum for each channel during frequency domain analysis. Frequency resolution of the resulting spectra was .017 Hz (1/60 s), which allows unambiguous identification of the response expected at the frequencies of interest (i.e., 6 Hz for the base stimulation and 1.2 Hz and harmonics for the deviant category detection). Individual FFT data were averaged across participants to allow group analysis. To determine the significant harmonics for both the base (i.e., elicited by the non‐canonical stimulation at 6 Hz) and the deviant (i.e., elicited by the canonical presentation at 1.2 Hz) responses, Z‐scores were computed at every channel. For each discrete frequency bin (x), Z‐scores were calculated as Zx=x−meannoisestandard deviationnoise, for which the noise was defined as the 20 surrounding bins of each target bin, excluding the immediately adjacent bins and the extreme (min and max) bins (Lochy & Schiltz, 2019). The number of significant harmonics was determined as the largest chain of consecutive harmonics showing a Z‐score larger than 1.64 (p < .050, one‐tailed, testing signal level > noise level).

The signal‐to‐noise ratio (SNR) for the EEG spectrum was computed by dividing the amplitude at each frequency by the average amplitude of 20 surrounding bins (10 on each side) (Liu‐Shuang et al., 2014). Finally, sums of baseline‐corrected amplitudes at the deviant category frequency (1.2 Hz) and significant harmonics (excluding the base stimulation frequency) were obtained to quantify and visualize general topographies of the target response for each participant and per group.

Statistical analyses

Statistical analyses were carried out using IBM SPSS statistics 26 software for Mac OS Monterey 12.6 (Armonk, NY). Statistical significance was set at p < .050 for all computations. Data were checked for normality of distribution and presented as mean ± standard error (M ± SE). Amplitude values were transformed using the square root (sqrt) transformation. Results were analysed using a linear mixed model (LMM) with Amplitude as dependent variable, Group (deaf, hearing controls) and Hemisphere (left, right) as fixed factors, as well as considering the Group × Hemisphere interaction. Subject was indicated as random effect, to control for any variability within the groups. First, base rate amplitudes (in μV) were analysed. These involve the visual responses synchronized with the general stimulation frequency (i.e., 6 Hz), that could vary with attention or certain morphological factors. Second, amplitude values (in μV) of the deviant rate were analysed by taking into account the amplitudes of the base rate as covariate. This approach was opted as we compared groups, and differences in brain responses, not specific to our manipulation, can therefore not be excluded. Many studies have indeed already indicated that deaf individuals develop functional and structural brain reorganisation, especially following cross‐modal plasticity (e.g., Alencar et al., 2019; Bottari et al., 2011; Cardin et al., 2020; Dell Ducas et al., 2021; Scott et al., 2014; Vachon et al., 2013). Given that such differences can be due to physiological (e.g., skull thickness, gyri folding, etc.) or even attentional factors, we wanted to control for their potential involvement in the neural responses to the deviant stimuli. Bonferroni post hoc analysis was applied when appropriate to control for family‐wise error rates, while Benjamini–Hochberg corrections were applied to control for false discovery rates (for the latter, see Supplementary Material).

RESULTS

Base rate responses

Responses synchronized with the base frequency (6 Hz) were significant up to eight harmonics (from 6 to 48 Hz). The baseline‐corrected amplitudes were summed and then the responses of the 68 electrodes were ranked. The electrodes showing the highest responses were all located in the parieto‐occipital region (see Figure 3), as observed in our previous FPVS study using similar non‐canonical finger‐number configurations (Marlair et al., 2021). The amplitudes of the channels PO8, O2 and contralateral channels PO7, O1 were averaged into two regions of interest (ROIs). The LMM showed a significant effect of Hemisphere (F(1, 40.0) = 8.91, p = .005) with the right hemisphere showing stronger responses (M ± SE = 1.13 ± .049 μV) than the left hemisphere (M ± SE = 1.00 ± .049 μV). No Group effect (F(1, 40.0) = .87, p = .36) was found, but a significant interaction between Group and Hemisphere (F(1, 40.0) = 4.95, p = .032) was observed. Responses in the left hemisphere were not significantly different among groups (M ± SE = 1.01 ± .069 μV for deaf signers, M ± SE = .99 ± .069 μV for hearing controls; p = .90), while responses for the right hemisphere were marginally significantly higher for the hearing control group (M ± SE = 1.04 ± .069 μV for deaf signers, M ± SE = 1.22 ± .069 μV for hearing controls; p = .074). The Subject intercept was significant (p = .001). Nevertheless, the inclusion of Subject as random effect in the model already controlled for its potential influence on the fixed effects.

Categorical discrimination responses

Categorical responses elicited by the presentation of canonical configurations at a 1.2 Hz rate were significant from 1.2 to 9.6 Hz (7 harmonics excluding the base stimulation frequency) across the two conditions. Scalp topographies of the sum of significant harmonics suggested posterior bilateral responses (see Figure 4). As shown in several EEG studies examining canonical finger‐number configurations (Soylu et al., 2019; van den Berg et al., 2021, Van den Berg et al., 2022), the channels showing the greatest amplitude of response were located in the parieto‐occipital region. Baseline‐corrected amplitudes were summed and ranked for the 68 channels to identify the channels with the highest responses. Channels O2, PO4 and contralateral channels O1, PO3 were averaged to create right and left ROIs. Statistical analysis indicated no significant Hemisphere difference (F(1, 43.8) = .13, p = .72), nor a significant Group × Hemisphere interaction (F(1, 42.0) = .079, p = .78). There was no significant contribution of the base rate amplitudes (F(1, 75.4) = 3.53, p = .064), but a significant effect of Group (F(1, 39.9) = 3.22, p = .010). The final model lead to the same conclusion after removing the non‐significant interaction from the Response amplitudes of deaf participants (M ± SE = .43 ± .044 μV) were significantly higher from those of hearing controls (M ± SE = .26 ± .045 μV, p = .010). Note that amplitudes are displayed for a base rate reference at 1.00 μV, and that the Subject intercept was significant (p = .007).

Interim discussion

Canonical number configurations from 1 to 4, corresponding to signs in deaf individuals and finger‐montring gestures in hearing individuals, were periodically inserted in a stream of non‐canonical hand configurations. In this way, the brain responses of deaf signers and hearing controls could be compared. Our results showed a statistically significant effect of Group, with deaf participants showing significantly higher responses than hearing controls. This result supports the hypothesis that finger‐montring configurations are processed differently in deaf signers.

EXPERIMENT 2

Introduction

Results of Experiment 1 could be due to two factors. First, the stronger response amplitudes in deaf individuals could reflect that for them only, the used canonical configurations belong to their linguistic system and have become signs. The results could alternatively be merely due to the greater experience deaf signers have with processing hand configurations.

To test these alternative hypotheses, we developed a second EEG experiment using canonical finger‐counting configurations as deviant stimuli. These configurations are canonical, they represent numerosities, but they do not belong to any specific language system. They can thus be considered as counting gestures for both groups. If deaf individuals have simply more experience with processing hand configurations, then they should also show stronger responses than hearing individuals using finger‐counting gestures. If results of Experiment 1 were due, however, to the fact that finger‐montring configurations were processed as signs of a linguistic system in deaf individuals only, then, no difference should emerge between groups in Experiment 2.