Authors: G. Meinhardt, B. Meinhardt-Injac, M. Persike
Categories: Psychology and Cognitive Neuroscience, face inversion effect, face-specific processing, individual differences, selective attention, Research Article, 1001, 205, 42
Source: Royal Society Open Science
Doi: 10.1098/rsos.181350
Authors: G. Meinhardt, B. Meinhardt-Injac, M. Persike
Numerous studies have reported impairments in perception and recognition, and, particularly, in part-integration of faces following picture-plane inversion. Whether these findings support the notion that inversion changes face processing qualitatively remains a topic of debate. To examine whether associations and dissociations of the human face processing ability depend on stimulus orientation, we measured face recognition with the Cambridge Face Memory Test (CFMT), along with experimental tests of face perception and selective attention to faces and non-face objects in a sample of 314 participants. Results showed strong inversion effects for all face-related tasks, and modest ones for non-face objects. Individual differences analysis revealed that the CFMT shared common variance with face perception and face-selective attention, however, independent of orientation. Regardless of whether predictor and criterion had same or different orientation, face recognition was best predicted by the same test battery. Principal component decomposition revealed a common factor for face recognition and face perception, a second common factor for face recognition and face-selective attention, and two unique factors. The patterns of factor loadings were nearly identical for upright and inverted presentation. These results indicate orientation-invariance of common variance in three domains of face processing. Since inversion impaired performance, but did not affect domain-related associations and dissociations, the findings suggest process-specific but orientation-general mechanisms. Specific limitations by constraints of individual differences analysis and test selection are discussed.
In the last two decades, the face inversion effect has received much attention across all disciplines of face perception research. However, opposing views persist about its implications for underlying face processing mechanisms as well as its potential neural underpinnings [1–4]. In a widely accepted view, the strong impairment in face recognition and identification tasks with inverted stimuli reflects qualitative changes in facial information processing. Several lines of evidence support a qualitative account of the face inversion effect. First, and phenomenologically striking, the impression of grotesqueness in face pictures with rotated eyes and mouth (‘Thatcher illusion’; [5,6]) is greatly attenuated in upside-down faces. This led to the conjecture that spatial-configural information processing is intact in the usual upright orientation, but disrupted by inversion [7–9]. Moreover, different measures of holistic face processing, all indicating this mode in the upright orientation, suggest that the degree of part integration is notably attenuated when faces are inverted [10–12]. Results from neuroimaging studies suggest that inverted faces engage face-tuned areas in the ventral stream, but also areas that preferentially respond to non-face objects, which might indicate a change of the processing route [4,13]. This was corroborated by the observation that patients with object agnosia could handle upright faces well, but had problems with inverted faces, suggesting that their efficient processing requires more than intact face processing mechanisms [14]. Further, the N170, an event-related potential from occipito-temporal electrodes, shows a significant delay for inverted compared to upright faces [15,16]. These results might indicate involvement of different mechanisms for upright and inverted faces.
On the other hand, evidence accumulates for the claim that upright and inverted faces rely on the same, or tightly overlapping sets of visual cues, and are processed by the same mechanisms, but with reduced efficiency for upside-down presentation. In line with this view, it was shown that there is no stringent distinction of ‘featural’ and ‘configural’ processing in terms of the inversion effect [3,17,18]. Measuring activation in face-selective brain areas showed strong object category selectivity rather than orientation selectivity [18,19]. Response classification methods revealed usage of the same cues in face discrimination tasks, but with less efficiency for inverted faces [20]. Facial cue summation experiments showed that inverted features were integrated in a less efficient manner than upright features, but according to the same rules [21]. Readdressing the spatial frequency tuning of upright and inverted face identification, a recent study found no evidence for orientation-related changes in spatial frequency tuning [22]. Critical evaluation of the results obtained in studies measuring whether face parts can be attended independently (see above) showed that the influence of non-attended face parts was diminished, but not abolished when faces were inverted (e.g. [11, fig. 4], [12, fig. 5]), indicating that results are not at odds with a quantitative account of the face inversion effect (see also [23] for further evidence). Moreover, neuroimaging studies do not provide unique evidence in favour of qualitative changes. While it is corroborated that face inversion engages additional processing regimes in face and non-face object sensitive areas, such as the occipital face-selective region (OFA), the superior temporal sulcus (fSTS) or the lateral occipital complex (LOC), the functional relevance of face-related activity in these areas remains unclear. Activity which varied in close correlation to the behavioural inversion effect was first found only in the fusiform face area (FFA) [13], while a later study also reported involvement of the OFA [24], suggesting the functional relevance of the same face-tuned core-units for processing both upright and inverted stimuli.
In another recent psychophysical study, sensitivity to part-based and relational manipulations of faces and houses was measured in a sample of 77 observers [25]. A correlation analysis was performed to reveal potential common category-specific or process-specific (spacing-tuned or feature-tuned) processing. The authors replicated the finding of similar inversion effects for both types of image manipulations, and found evidence for two domain-general, but process-specific factors (see [25], table 3), each explaining about 30% of variance. Relational, or ‘spacing’, conditions had high loadings only on the first factor and part-based conditions only on the second. Moreover, loadings were equally high for upright and inverted presentation. These findings suggest shared processing routes for upright and inverted stimuli, with no changes by inversion. Quite consistently, common variance was found to be process-specific but unspecific for orientation.^1^
A recent study on the developmental trajectory of part-based and configural object recognition across adolescence also found evidence for distinct processes with specific developmental trajectories [26]. Part-based processing was found to saturate early in adolescence, reaching a plateau after 11–12 years of age, while configural processing steadily improved well into young adulthood (see [26], figure 2). Inversion effects were found for both kinds of processing, while the distinct, process-specific trajectories had exactly the same temporal course in either orientation. In line with the findings from individual differences analysis [25], these results from cross-sectional data suggest process-specific but orientation-invariant routes.
These results point to a potential orientation-invariance principle for common variance structures that could be validated by individual differences data. There is growing interest in this approach for resolving controversial issues in the face processing literature, since it offers access to patterns of associations and dissociations among different face processing abilities, which are usually ignored in experimental studies which focus on group-level analysis [27]. Predicting face recognition, measured with representative tests like the Cambridge Face Memory Test (CFMT) [28,29], by cognitive and perceptual abilities has raised considerable attention in the past years [27,30,31]. Face recognition is a highly heritable [29,32], long-lasting developing [33] and specific ability, which is almost independent of general intelligence [31,34,35], non-face memory ability [29], and general object recognition ability [36,37]. It is also a complex ability, comprising face perception and encoding, face memory and cognitive capabilities to infer face identity from unknown views, distorted images, varying external features or age-related changes [28,38,39]. Accordingly, it was found that face recognition was associated with face perception and long-term face memory [29,39,40]. Surprisingly, it appeared to be just weakly associated [41,42], or even dissociated from holistic face perception, indexed by derived measures used in varieties of the composite face paradigm [43,44]. Difference measures of part-whole integration showed modest correlation with face recognition, while larger correlations were obtained with regression-based methods [45]. However, results of a recent study [46] showed that part and whole performance correlated nearly perfectly, casting doubts on the construct validity of the part-whole test. The suitability of different holistic processing measures for predicting face recognition ability is currently under debate (e.g. [47]).^2^
In the present study, we focused on the associations and dissociations among face recognition, face perception and object-related attention to faces (face-selective attention) as three key domains of human face processing. Albeit the literature on individual differences in the face recognition ability is steadily growing [31], there were hitherto no attempts to reveal how correlations with face recognition depend on stimulus orientation. This study is a first attempt to contribute to this question. To assess face perception ability, we used a paradigm that allowed us to measure how precisely observers were able to judge identity of just the inner face parts, ignoring the external facial surround (context congruency paradigm, [12]).^3^ Object-related attention tasks were administered with faces and non-face objects to study whether the potential link between object-related attention and face recognition is face-specific. We administered a comprehensive test battery, including only tests with reasonably good internal consistency, to a broad sample of 314 participants, and measured all test conditions with upright and inverted stimuli. Principal component and multiple regression analysis revealed distinct process-specific pathways for face-related tasks, two of them shared with the face recognition ability. Surprisingly, the patterns of associations and dissociations within and across tests were the same for upright and inverted stimuli, indicating orientation-invariance of correlation structures. These findings indicate that orientation-general mechanisms underlie stimulus processing in shared processing routes for the studied face-related tasks.
Three experimental tests were one for face recognition, one for face perception and one for selective attention. All tests were administered in one 90 min session. The order of the tests was counterbalanced across participants. Test were separated by a 10 min break. Three hundred and fourteen observers participated in the study. All participants were undergraduate students of the Johannes Gutenberg University Mainz. The age range was 17–37 years, mean age = 22.8 years, standard deviation = 3.4 years, 223 were female. The participants had normal or corrected to normal vision, using corrective lenses in the latter case. All participants received a global information about the tests and the abilities being assessed with the test battery. They were given a gratification of €30 for participation.
All experimental tests were computer-administrated, executed with Inquisit runtime units. Stimuli were displayed on NEC Spectra View 2090 TFT displays in 1600 × 1200 resolution at a refresh rate of 60 Hz. Screen mean luminance was 100 cd m^−2^ at a Michelson contrast of = 0.98. No gamma correction was used. The room was darkened so that the ambient illumination matched that of the screen. Stimuli were viewed binocularly at a distance of 70 cm. Participants used a distance marker but no chin rest throughout all experiments. Participants responded by pressing a button on an external key-pad. No feedback about responses was provided in any of the three tests.
The Cambridge Face Memory Test (CFMT) [28] was developed to study face identity recognition, including generalization over viewpoint and image distortion. Six different target faces and 46 distracter faces are used. The participant is asked to memorize six target faces for 20 s (encoding). Afterwards, sequences of six face images are presented, one of them a target face. In the first 18 sequences, the target item is identical with an encoded item. Over the following 30 sequences, the identity of the target and the encoded item stays the same, but the target item shows the encoded item in a different view (left or right profile). The final 24 items show the target items in a different view and distorted by noise. All test sequences were conducted with upright and inverted presentation of the stimuli, organized in coherent blocks. Block order was counterbalanced across participants. The total test score is the number of correctly identified target items in all of the 72 sequences. With an additional learning phase, the test takes about 15 min, adding up to a total of 30 min for both upright and inverted conditions. Detailed information on test construction can be found in [28]. The authors provide free access to the test material for scientific purposes.
The context congruency paradigm (CCP) was developed to measure the interaction among external and internal facial features in face perception, thus indicating the degree of joint processing of inner and outer face parts [12,49]. High internal consistency was verified for all test conditions (see Methods). In the variant used here, participants judged the identity of the internal facial features of two successively shown faces, one in left-hand sided and one in right-hand sided 34 view. The complete design [50] was used to vary the congruency relation of internal and external features (figure 1). Congruent contexts (CC) were defined as total agreement of faces in ‘same’ trials and total disagreement in ‘different’ trials. Incongruent contexts (IC) were defined as part-based agreement/disagreement. Specifically, in incongruent-same trials the internal features of a face pair agreed, while the external features disagreed. In incongruent-different trials, the inner face parts disagreed while the external parts agreed. Responding to whole face agreement/disagreement is beneficial in congruent trials, but leads to chance performance in incongruent trials (see [12] for detailed descriptions).
Figure 1.The context congruency paradigm (a) and the superposition paradigm (b). (a) Illustrates congruent (total sameness and difference) and incongruent (part-based sameness and difference) trials of the CCP in the framework of the complete design. The congruency relation refers to inner face part matching in the context of external feature surround. The letters a–d refer to face identities, the first index denotes internal features, the second external features. (b) Shows stimulus examples for the superposition paradigm for attending faces while ignoring scenes, SPPF, and, vice versa, SPPO. The non-attended stimuli were realized with high (65%, HO) and low (35%, LO) values of opacity to construct conditions with high and low attentional competition among faces and scenes.
Photographs of four male face models taken in a professional photo studio were used as templates for stimulus construction. The photographs were converted to 8 bit greyscale pictures. Face models had similar haircut, face geometry and overall type appearance. As a scheme for stimulus construction, we used a 4 × 4 composite face matrix with line index referring to internal features and column index referring to external features, such that an entry Sij denotes a face with internal features of face i and external features of face j. Before placing the internal features of face j onto face i, mean grey level and contrast of the internal features of face j were adjusted to the values of the internal features of face i. As a result, internal features were smoothly integrated into the external feature surround. Thus, we obtained 16 natural looking composite faces, to be used for same-congruent trials. For each composite it is possible to combine in nine different ways to form complete sets of trials within the complete design [50]. Hence, using 16 ‘same’ and 16 ‘different’ trials warranted that there was no repetition of the same stimulus pair, and that each observer responded to an individual choice of trial selections. Stimulus size was 300 × 400 pixels (width × height), which corresponded to 7.0 × 9.7 cm, or 5.7° × 7.9° visual angle. For each stimulus, an individual mask was constructed from randomly ordered 5 × 5 pixel blocks of the stimulus image. Masks subtended 450 × 600 pixels (width × height). (See [12] for more details.)
Participants were informed that two face images would be presented successively, and that the identity of the inner face parts had to be judged. They were also told that the external face parts could vary or be identical and were irrelevant for the identity judgement. The structure of a trial fixation mark (300 ms)–blank (100 ms)–first face stimulus (433 ms)–mask (350 ms)–blank (200 ms)–second face stimulus (433 ms)–mask (350 ms)–blank frame until response. The centre positions of each of the two face images were shifted by 20 pixels away from the centre in random direction to preclude the same image parts being focused. In half of the trials, the first face image was in left-hand sided view, and in the other half in right-hand sided view. Further, both face images of a trial sequence were presented upright, or rotated by 180° (inverted). The two congruency conditions (congruent, incongruent), two orientations (upright, inverted), two trial types (same, different) and 16 replications of condition resulted in 128 trials, which were ordered randomly. The proportion of correct judgements, calculated from both correct ‘same’ and correct ‘different’ judgements, was measured. Participants were made familiar with the task with 16 randomly selected practice trials. The experimental test took about 20 min.
The superposition paradigm (SPP) was developed to study attentional competition among coherent sensory inputs (objects) presented simultaneously by spatial overlap [51,52]. As in [52], we studied attentional competition among faces and scenes in two categorization tasks. In the face categorization task (SPPF), a face image was overlayed in transparency with a scene image of the same size, and the participants judged the gender of the face. In the scene categorization task (SPPO), they judged whether the scene image depicted was an interior or exterior scene. Two levels of opacity of the irrelevant, to be ignored image were 35% (low opacity, LO) and 65% (high opacity, HO), (figure 1). The LO condition warrants high salience of target objects and low levels of competition, while there is no salience of target objects and a high level of attentional competition in the HO condition. Composite face+scene images were created by combining pictures from a database with 18 male and 18 female faces and 18 interior and 18 exterior scenes. Stimulus picture dimensions were the same as in the CCP. Gender and scene category were varied orthogonally to create composites, i.e. a face of a given gender was paired with both interior and exterior scenes, and vice versa. The design comprised two tasks (gender/scene categorization), two stimulus orientations (upright/inverted) and two levels of opacity (LO/HO). Each of these eight conditions was executed with 32 replications, resulting in 128 trials for each categorization task. The proportion of correct classifications, calculated from both response alternatives in each test, was measured.
Participants were made familiar with each task by going through five randomly selected probe trials. A trial started with presentation of a central fixation mark for 1 s, which was replaced by the compound image for 2.5 s. Response triggered presentation of the next trial, with an inter-trial period of 2 s. The order of the two categorization tasks was counterbalanced across observers. Each categorization task took about 15 min testing time.
In total, 14 performance indices, seven for upright and seven for inverted presentation, were obtained from the three experimental tests. All indicated the proportion of correct judgements. The CFMT yielded two sumscore indices, one for each orientation. The CCP yielded four indices, reflecting performance in congruent and incongruent condition for each stimulus orientation. Each categorization task of SPP again yielded four indices, reflecting performance for both opacity levels for upright and inverted presentation. To ensure that the correlation analyses were not biased by outliers, we conducted multivariate outlier analysis based on the squared Mahalanobis distance measure, Δ^2^ ([53, pp. 177]). To clear for multivariate outliers, we calculated the distribution of Δ^2^ values, and removed all cases with values beyond the third quartile plus 1.5 units of inter-quartile range (Tukey’s method, [54]). This method identified 16 outliers, which were excluded from data analysis. Thus, N = 298 of the 314 cases remained in the final sample. Inspection of the (142)=91 bivariate regression plots verified that correlations were not biased by single observations. The proportion correct measures, Pc, were analysed with ANOVA methods for group-level analyses. Additionally, inversion effects (IEs), congruency effects for the CCP and opacity effects for the SPP were calculated with paired t-tests. Cohen’s d was calculated for all tests to provide the same effect size measure for all effects. Statistica 13.0 software was used for all ANOVA-based testing, principal component analysis and multiple regression methods.
Internal consistencies for all tests were assessed by computing Cronbach’s α, which is based on inter-item correlations [55]. Test trials were treated as items. Overall test reliability was calculated by applying the Spearman–Brown test length factor formula to the single test parts. Table 1 shows the results for the CFMT and table 2 summarizes the results for the CCP and the SPP. The CFMT showed high overall internal consistency, exactly in line with the estimate reported by the test authors from a sample of 389 participants [29]. Internal consistency was reduced in inverted face presentation, mostly due to a noticeable loss of homogeneity in the third test part with external noise. Similarly, the CCP exhibited high overall internal consistency, which was larger for congruent (full face agreement/disagreement) than for incongruent trials (partial agreement/disagreement). Internal consistency of the SPP was acceptable, higher for SPPO compared with SPPF, which reflected stronger item-heterogeneity in the gender categorization task. Since only one test suffered from reduced reliability we calculated across-test correlations without corrections for attenuation.
Table 1.Internal consistency of the CFMT. The table shows Cronbach’s α calculated from the inter-trial correlations in the three test parts, and Spearman–Brown based reliability estimate for the complete test, rtt.uprightinvertedT1T2T3T1T2T3α0.690.810.780.580.560.25rtt.uprightinvertedCCPCCICCCICsamediffsamediffsamediffsamediffα0.800.800.700.680.750.680.600.66r~tt0.880.70
Table 2.Internal consistency of the CCP and the SPP. The table shows Cronbach’s α and Spearman–Brown reliability estimates for the test conditions, rtt0.890.820.830.78SPPfacescenefacesceneHOLOHOLOHOLOHOLOα0.330.290.610.650.390.440.550.49rtt~0.470.770.580.69
Figure 2 shows the mean accuracy data for all three tests, and table 3 shows inversion effects.
Figure 2.Mean proportion correct rates, Pc, for all test conditions. The error bars denote the 95% confidence intervals of the means.
Table 3.Inversion effects in the CFMT, the CCP and the SPP. The table shows the difference measure, IE, its standard error, t − statistic, significance level, p, and Cohen’s d. All t-tests had 297 degrees of freedom.testconditionIEsetpdCFMTall0.1690.00725.7<0.0011.49CCPcongruent0.1120.00619.7<0.0011.14incongruent0.0420.0075.7<0.0010.33SPPface-LO0.1130.00522.6<0.0011.31face-HO0.1480.00626.5<0.0011.53scene-LO0.0010.0020.10.8940.01scene-HO0.0130.0025.7<0.0010.33
Figure 3.Factor scree plot for the principal component decomposition of all tests (open circles). Grey diamonds indicate the Eigenvalues of normally distributed and independent random variables with the same sample size as the dataset.
In the CFMT, an average accuracy based on the total test score of Pc = 0.746 was achieved in upright and Pc = 0.577 in inverted presentation, which fairly well agreed with the data of the test authors, who obtained rates of Pc = 0.804 (upright) and Pc = 0.584 (inverted) in a sample of 50 college students [28, p. 580]. The inversion effect was large, and reached an effect size of d = 1.5 (table 3). In the CCP, there were significant effects of Congruency [F1,297 = 925.9, p < 0.001, ηp = 0.76], Orientation [F1,297 = 259.9, p < 0.001, ηp = 0.47] and a significant Orientation × Congruency interaction [F1,297 = 59.5, p < 0.001, ηp = 0.17]. This interaction indicated a larger congruency effect in upright compared to inverted presentation. However, congruency effects in both orientations were substantial, reaching large effect sizes (upright: Δ = 0.26, t297 = 30.6, p < 0.001, d = 1.77; Δ = 0.19, t297 = 21.3, p < 0.001, d = 1.24), while inversion effects were large in congruent trials, but had modest effect size in incongruent trials (table 3).
In the SPP, there were significant effects of Stimulus [F1,297 = 1166, p < 0.001, ηp = 0.80], Opacity [F1,297 = 790.1, p < 0.001, ηp = 0.73] and Orientation [F1,297 = 1039, p < 0.001, ηp = 0.78]. Further, all 2-way interactions were highly significant [Opacity × Orientation: F1,297 = 40.2, p < 0.001, ηp = 0.12; Stimulus × Opacity: F1,297 = 68.2, p < 0.001, ηp = 0.19; Stimulus × Orientation: F1,297 = 896.3, p < 0.001, ηp = 0.75], and also the Stimulus × Opacity × Orientation interaction was significant [F1,297 = 7.9, p < 0.01, ηp = 0.03]. The 2-way interactions involving opacity reflected significantly larger opacity effects in inverted compared to upright presentation, and significantly larger opacity effects for attending scenes (d = 1.06) than for attending faces (d = 0.82).^4^ Inversion effects with large effect sizes were reached for attending faces in both opacity conditions. For attending scenes, an inversion effect was absent in the LO condition, but there was a modest inversion effect in the HO condition (table 3).
The CFMT, the CCP in congruent conditions and the SPP for attending faces all yielded strong inversion effects (d > 1) (table 3). On the other hand, the CCP in incongruent condition and the SPP for attending scenes exhibited moderate (HO), or even lacking (LO) inversion effects, which were clearly distinguished from the aforementioned large inversion effects in terms of effect size. The expected strong effects of congruency (CCP) and opacity (SPP) at the group level verified the experimental paradigms. Larger opacity effects for scenes compared to faces showed that faces were more robust than scenes in object-based attentional competition [52].
Performance in the congruent condition of the CCP suffered notably from inversion. However, the congruency effect in inverted presentation was still quite substantial (d > 1, see figure 2. See also [23] for similar observations in the composite paradigm). This shows that contextual interference from external features was diminished but not abolished for inverted faces. Likewise, we found substantial opacity effects of the non-face masker for the SPPF in both orientations, which indicates that the nature of attentional competition was not altered by inversion.
Several lines of evidence suggest a common route for upright and inverted stimuli, while there is also evidence for orientation-dependent processing (see Introduction). A strong claim of orientation-invariance would consider only orientation-general mechanisms. This claim has two implications. First, it implies that performance in the same test condition is solely explained by a common mechanism for upright and inverted stimuli, up to spurious error variance. Second, it implies that the correlation structures of a test battery are the same for either orientation.
A weaker claim of orientation-invariance would consider that both orientation-general and orientation-specific mechanisms handle a given test condition, but would postulate that only orientation-general mechanisms underlie shared processing within and across tests. This would imply (i) a strong orientation-general variance component in each test condition and (ii) same correlation structures of a test battery for either orientation.
The strong claim of orientation-invariance can be tested using the bivariate (zero-order) correlations among both orientations for the same test conditions. If the correlation among both orientations, r, is close to the maximum possible correlation, which is limited by the geometric mean of the reliabilities for either orientation, rmax=rttrt′t′, this would indicate just a common route for both orientations, since there is no reliable unexplained variance left.
A redundant test strategy, which analyses the data from different angles, is recommended for testing whether the correlation structure of the test battery is orientation-invariant. Since face recognition is a complex ability (see Introduction), we may predict the CFMT by the remainder tests, and probe whether the results for regression weights and explained variance depend on stimulus orientation. Orientation-invariance would be concluded if the best predictor battery is the same, irrespective of whether the predictors have same or different orientation than the criterion. Another way of testing orientation-invariance of correlations is decomposing the correlation matrix of all test conditions into principal components. In a factor solution for the common variance components,^5^ we should observe that, if a test has significant factor loadings, the loadings should be equally strong for either orientation. The results of multiple regression analysis are validated by the factor solution if it is shown that the CFMT and its valid predictors load on the same factors, and equally strong for upright and inverted stimuli.^6^
We calculated bivariate product-moment correlations among all test conditions. Table 4 shows the results. With N = 298 cases, the critical correlations for significance levels of α=0.1%, α=1% and α=5%, respectively, are r(0.1%)=0.19, r(1%)=0.15 and r(5%)=0.11. In the table, the correlations larger than r(0.1%) are printed boldface.
Table 4.Pearson product-moment correlations of the CFMT, the CCP, the SPPF and the SPPO for all seven test conditions and both orientations. Correlation coefficients larger than the critical correlation r(0.1%)=0.19 are printed boldface. Coefficients for the same test condition but across orientations are underlined.testCFMTCCPSPPFSPPOCond——CCCCICICLOLOHOHOLOLOHOorientuprinvuprinvuprinvuprinvuprinvuprinvuprCFMTinv0.50CCPCCupr0.28****0.35CCPCCinv0.270.270.52CCPICupr0.100.130.23****0.19CCPICinv0.130.120.260.250.58SPPFLOupr0.190.12−0.010.050.070.17SPPFLOinv0.160.270.150.14−0.040.080.22SPPFHOupr0.22****0.240.150.160.050.120.23****0.44SPPFHOinv0.21****0.270.140.120.130.200.170.40****0.33SPPOLOupr0.040.060.050.100.060.080.080.130.140.02SPPOLOinv0.110.030.100.170.120.170.130.110.200.130.44SPPOHOupr0.110.170.100.080.150.150.260.250.250.250.48****0.50SPPOHOinv0.040.130.110.080.110.080.110.200.200.200.410.44****0.50
There are several crucial observations concerning the pattern of bivariate correlations. First, there were strong and significant correlations among conditions within the same tests (diagonal quadrants of table 4). These correlations were similar for upright and inverted presentation. They tended to be highest for the same condition, measured across orientations (see underlined values in table 4). Second, there were selective correlations between tests. Whenever between-test conditions correlated significantly, the correlation magnitude was alike in both orientations, and across them, reflected by quite homogeneous blocks of four correlations in table 4. Looking at the bivariate correlations of the CFMT shows that there were significant correlations with the CCP in congruent condition (both orientations), but not in the incongruent condition. Further, the CFMT correlated most substantially with the SPPF in the high opacity condition (both orientations), and weaker for low opacity of the masker. With just one exception, there were no highly significant correlations with SPPO conditions in the low opacity condition, while there were correlations among SPPF and SPPO for high opacity. This indicates that correctly perceiving the target when it is strongly masked reflects some common ability across faces and non-face object categories (see last two lines of table 4).
Next we used the bivariate correlations among both orientations of the same test condition to test whether performance is solely explained by their common variance. Decomposing the proportion of reliable variance into a predicted and a residual component, rmax2=r2+rres2, we tested the residual component with an F- statistic, i.e. F=rres2(N−2)/(1−rres2) [57]. Results (table 5) revealed that a significant proportion of reliable residual variance remained in each test condition, whereby the proportions of explained and residual variance were nearly alike. Proportions of residual variance tended to be larger for the tests with higher reliability (CFMT and CCP), while they were smaller for the less reliable SPP.
Table 5.Pearson product-moment correlations of upright and inverted orientation for each test condition. The table shows the correlation coefficient, r, the theoretical maximum correlation based on test condition reliability in either orientation, rmax, proportion of common variance, r^2^, theoretical maximum proportion of common variance, rmax2, proportion of residual variance, rres2, its F-statistic and significance level, p.testconditionr**rmaxr^2^rmax2rres2F1,296pCFMTall0.500.780.250.610.36169.5<0.001CCPcongruent0.520.860.270.740.47265.3<0.001incongruent0.580.800.330.640.31130.8<0.001SPPface-LO0.330.360.110.130.025.42<0.05face-HO0.220.360.050.130.0825.9<0.001scene-LO0.500.580.250.330.0827.1<0.001scene-HO0.440.560.200.320.1240.9<0.001
Hence, results showed that not all of the reliable variance for upright stimuli could be predicted by the reliable variance for inverted stimuli in the same test condition. The strong correlations across orientation clearly suggest a common route for both orientations. However, significant proportions of residual variance indicate that additional orientation-specific components enter.
In the next steps, we analysed whether common variance across test conditions was orientation-invariant.
Multiple regression was used to predict the CFMT by the CCP, the SPPF and the SPPO. To prove whether equal predictability results when criterion and predictors have same or opposite orientation, we ran separate analyses for each combination of predictor and criterion orientations, resulting in four analyses. Table 6 summarizes the overall results. All four analyses revealed highly significant predictability of the CFMT by the test batteries. The multiple correlation reached by the predictor batteries was R¯=0.379 on average, obtained by using Fisher Z transforms of the multiple correlation coefficients. Predictability in the four orientation configurations was alike, showing only marginal deviation from the average and largely overlapping confidence intervals. The largest multiple correlation was observed for predicting the CFMT in inverted presentation from CCP and SPP predictors in upright configuration.
Table 6.Multiple regression results for estimating the CFMT from test batteries formed by CCP, SPPF and SPPO conditions in definite orientations. The table shows the orientation conditions for criterion and predictors, multiple correlation coefficient, R, model and residual sums of squares and degrees of freedom, SSQ and d.f., F ratio, significance level, p and the 95% confidence interval of the multiple correlation coefficient, CI.criterionpredictorsRSSQ~md.f.mSSQed.f.e~F**pCI(R)uprightupright0.370.6464.112917.56<0.001[0.27,0.46]uprightinverted0.330.5264.232915.96<0.001[0.23,0.43]invertedupright0.420.4662.2229110.07<0.001[0.32,0.51]invertedinverted0.400.4462.242919.48<0.001[0.31,0.50]
Table 7 shows the detailed results for the four test batteries, allowing identification of the most valid predictors for the CFMT. Across all four analyses results were practically coincident. The CCP in congruent condition was the best predictor of the CFMT, followed by the SPPF in the high opacity condition. Only in one case did the low opacity condition of the SPPF reach significant contributions to the CFMT criterion. The CCP in incongruent condition and all conditions of the SPPO clearly failed to contribute to valid CFMT predictions. Importantly, whether a test condition was a valid predictor or not did not depend on the orientation of criterion or predictors. Hence, the results from multiple regression analysis strongly indicate that just the nature of the task underlying the predictors defined the common variance with the CFMT, but not their orientation.
Table 7.Multiple regression results for estimating the CFMT from test batteries formed by CCP, SPPF and SPPO conditions in definite orientations. The table shows predictor with condition, standardized regression coefficient, b, its standard error, sb, product moment correlation of predictor and criterion, rc, t-statistic with degrees of freedom and significance level, p. Significant regression coefficients are printed boldface.PredCondb**sbrct291pbsbrct291pupright criterion and upright predictorsupright criterion and inverted predictorsCCPCC0.260.0570.284.51<0.0010.220.0580.273.85<0.001CCPIC0.020.0570.100.380.710.030.0580.130.580.562SPPFLO0.150.0580.192.60<0.010.070.0610.161.060.292SPPFHO0.150.0580.222.50<0.010.150.0620.212.39<0.02SPPOLO−0.010.0620.04−0.160.8710.060.0630.110.970.334SPPOHO0.010.0660.110.180.854−0.050.0630.04−0.740.461inverted criterion and upright predictorsinverted criterion and inverted predictorsCCPCC0.310.0560.355.60<0.0010.230.0560.274.08<0.001CCPIC0.030.0550.130.560.5780.0240.0570.120.430.668SPPFLO0.060.0560.121.100.2740.170.0590.272.83<0.01SPPFHO0.160.0570.242.83<0.010.170.0600.272.75<0.01SPPOLO−0.040.0610.06−0.590.556−0.090.0610.03−1.420.158SPPOHO0.090.0640.171.460.1440.080.0610.131.330.183
To further validate this result, we used bivariate regression to predict the scores for upright from the scores for inverted stimuli for each test condition, and analysed the predicted scores, z^, as well as the residual scores, ze, with multiple regression. Note that z^ are the scores expected from orientation-invariant processing, since they capture the variance which is shared by both orientations. The ze scores are the scores which remain unexplained by this assumption, thus they capture the orientation-specific variance. If all common test variance is orientation-unspecific, then predicting the CFMT z^ scores by a multiple regression battery of z^ scores from the other tests identifies the significant predictors, while predicting CFMT ze scores should result in a non-significant prediction battery. Multiple regression results exactly complied with this prediction. For z^ scores we obtained a highly significant multiple regression battery (R = 0.40, F6,291 = 9.48, p < 0.001), while the battery for ze scores was not significant (R = 0.18, F6,291 = 9.48, p = 0.148). Detailed results (table 8) substantiated that the CCP in congruent condition was the best predictor of the CFMT, followed by both opacity conditions of the SPPF.
Table 8.Multiple regression results for estimating the CFMT from test batteries formed by CCP, SPPF and SPPO conditions, using scores predicted from common orientation processing in each test (z^, left columns), and the residual scores that are unexplained by this assumption (ze, right columns). The table shows predictor with condition, standardized regression coefficient, b, its standard error, sb, product moment correlation of predictor and criterion, rc, t-statistic with degrees of freedom, and significance level, p. Significant regression coefficients are printed boldface.z^ scoresze scoresPredCondb**sbrct291pbsbrct291pCCPCC0.230.0560.274.08<0.0010.060.0580.051.000.314CCPIC0.020.0570.120.430.668−0.010.058−0.01−0.150.881SPPFLO0.170.0590.272.83<0.010.130.0590.142.25<0.05SPPFHO0.160.0600.272.75<0.010.070.0590.091.260.207SPPOLO−0.090.0610.03−1.420.158−0.040.059−0.03−0.750.453SPPOHO0.080.0610.131.330.1830.020.0600.050.350.724
Principal component decomposition was calculated to reveal the factor structure of all seven test conditions in the two orientations (14 variables). This analysis revealed a typical exponential decay of the Eigenvalues, and identified four factors with Eigenvalue above 1 (figure 3). To verify a four factors solution for the true common variance components, we additionally used Horn’s modification of the factor-scree test [58]. Generating 14 normally distributed and independent random variables with the same set size as the dataset (N = 298) and applying PCA yielded a linear Eigenvalue course (see grey values and straight line in figure 3). Only the first four factors of the dataset had Eigenvalues above the Eigenvalues of the random variables (Horn criterion). Together, these four factors extracted 58.5% of test variance from the correlation matrix. We rotated with a Varimax standardized procedure to achieve a simple structure in the factor loadings pattern. The results of this solution are shown in table 9. A clear-cut simple structure was obtained, featuring four factors which accounted for approximately 15% of the total test variance each. The first factor represented shared variance of the CFMT (40.4%) and the CCP in congruent condition (52.9%). The second factor was specific for the SPPO (96.3%) and the third factor was specific for the CCP in incongruent condition (88.6%). Neither of them had common variance with the CFMT nor with other tests. The fourth factor was specific for the SPPF (82.6%), but also represented common variance with the CFMT (9.9%). These results strongly correspond to the result obtained in the multiple regression analyses. The CCP in congruent condition and, to minor degrees, the SPPF shared distinct and independent (orthogonal) common variance components with the CFMT. The CCP in incongruent condition and the SPPO did not share common variance with the CFMT, nor with other tests,^7^ representing specific abilities with no link to others. A further very striking result of the principal component analysis is that, if there were strong factor loadings, they had equal magnitudes in upright and in inverted orientations. The factor loadings uniquely reflect orientation-invariance of specific test variance and of common test variance. The pattern of factor loadings consistently showed that both orientations of the same test conditions always loaded together on the same factors. There was no sign of orientation-specific variance in the factor solution. The orientation-invariant factor solution was validated against potential metric quality constraints by analysing Spearman rank correlations, which yielded practically the same factor loadings pattern (see appendix B).
Table 9.Factor loadings of all experimental conditions in both orientations for a four factor solution (λ > 1), after varimax standardized rotation. The table shows the factor loadings, explained variance in Eigenvalue units, proportion of explained variance of each factor (%), proportion of test-specific variance of the test with dominant loadings on the factor (%), proportion of variance of the CFMT on the factor (%) and the sum of test-specific and CFMT variance on the factor. Factor loadings larger than 0.5 are printed boldface.testconditionorientationF1F2F3F4CFMT—upr0.63−0.01−0.020.29CFMT—inv0.670.00−0.050.35CCPCCupr0.770.080.21−0.04CCPCCinv0.720.120.20−0.05CCPICupr0.130.080.850.01CCPICinv0.150.060.850.15SPPFLOupr−0.100.090.200.56SPPFLOinv0.170.12−0.130.71SPPFHOupr0.180.17−0.030.66SPPFHOinv0.140.050.140.68SPPOLOupr0.050.78−0.02−0.03SPPOLOinv0.070.760.110.06SPPOHOupr0.020.750.100.31SPPOHOinv0.050.730.010.17explained Var2.092.381.642.08total (%)15.017.011.714.9test-specific (%)52.996.388.682.6CFMT (%)40.4< 0.10.29.9test+CFMT (%)93.396.388.892.5
Both regression and principal component analysis showed that the CFMT shared common variance with the CCP in congruent condition and with the SPPF, while the latter two did not share common variance. Thus, performance in the CCP (congruent), measuring face identity perception across swapped 34 views and in the SPPF, measuring object-related attention to faces, likely concern different and independent abilities, both contributing to the face recognition ability. In either orientation, the congruent condition of the CCP predicted face recognition, while the incongruent condition failed to do so. This is clear evidence that task success in congruent and incongruent condition of the CCP rest on distinct attentional strategies, thus validating the assumptions of the paradigm.^8^ Precise focus on just the inner face parts, which warrants success in the incongruent condition, is unrelated to CFMT performance, while attending the whole facial appearance including its external features [59,60] improves face recognition.
These results correspond to recent findings reported for the composite face paradigm [45]. For this paradigm, authors reported the largest correlation with the CFMT in the aligned-congruent condition (r = 0.5) and the smallest correlation in the aligned-incongruent condition (r = 0.21, see [45], table 1). These results might suggest that the ability to precisely keep focus on facial target parts is less important for face recognition than the ability to assess face identity using a more global attentional strategy, since this allows for a richer and deeper face encoding particularly for changing views [28,61–63]. However, it was pointed out [37,44] that the CFMT with its constant repetition of the six target faces has a clear learning format, and tests which use repetition of faces or face parts could correlate with the CFMT due to similar learning formats. Indeed, this potential confounding concerns the CCP, since the composites in the CCP stem from only four base faces.^9^
Second, we found that face-selective attention under conditions of visual competition (SPPF) explains face recognition, while scene-selective attention SPPO does not. This finding is in line with the strong category specificity of face recognition [29,36,37]. Since the gender categorization task favoured usage of whole face-based strategies [64], this corresponds to the finding that only the congruent condition of the CCP shared variance with the CFMT. In the SPP, the same face appeared eight times (four times as a distracter and four times as the target), and presentation time was quite long (2.5 s). This could have enabled face learning. Indeed, as a result of repetition, there were adaptation effects in the FFA and the PPA (parahippocampal place area), which were shown to correlate with object recognition tasks administered after the SPP [51]. These findings indicated a relationship between intact early attentional selection mechanisms, selective and more strongly tuned activation of stimulus category-specific extrastriate brain areas [52], and better encoding. In line with these results, our findings corroborate that observers who better attended faces and more efficiently suppressed scenes also showed better face recognition performance. The independence of SPPF and the CCP (both conditions) showed that this object-related attentional ability is clearly distinguished from the ability to attend faces with different spatial foci.
Testing face recognition, face perception and object-based attention to faces and scenes with a comprehensive test battery revealed consistent orientation-invariance in all patterns of associations and dissociations. If test conditions shared common variance, they shared it within the same and across different orientations. Factor loadings of test conditions on common or unique factors, which accounted for 58.5% of the total test variance, were always equivalent in magnitude for either orientation. Prediction of the CFMT succeeded equally well both within same orientations and across them, and the best CFMT predictor battery was validated by the scores predicted from a common mechanism for upright and inverted stimuli. Thus, individual difference analysis indicated the absence of orientation-specific correlation structures. There was no evidence for task-general processing specific for either orientation, and therefore, there was also no support for the claim that inverted faces rely on a task-general, part-driven processing mode, while upright stimuli are handled by globally face-tuned and ‘holistic’ mechanisms [2,9].
On the other hand, an orientation-general route could not completely explain all reliable test condition variance, which indeed indicated involvement of orientation-specific components. This finding is in agreement with earlier observations from a smaller sample with N = 77 participants [25].^10^ Since we found that the correlations within and between tests were strictly orientation-invariant, orientation-specific processing is limited to the single test conditions. Therefore, our findings support a weaker claim of orientation-invariance, which permits involvement of orientation-specific processes, but assumes that all shared processes within and across tests are orientation-invariant. Particularly, we found evidence that only orientation-invariant processing links face recognition to face perception and face-selective attention. These results are plausible in the light of current findings, which corroborate that the same face-selective brain areas are involved in different face processing tasks [13,18,65], while inverted faces engage mechanisms in face- and non-face object selective areas [4,14,66]. Results of a recent study [24] corroborated quantitative orientation-based modulation in mechanisms of the basic face network (early visual cortex, OFA, FFA and inferior frontal gyrus), which are sensitive to faces in either orientation, while only upright faces induced inhibition in the general object-tuned lateral occipital complex (LOC), and only inverted faces caused activation in the intraparietal sulcus (IPS), which is associated with visual working memory [67]. These results provide evidence both for quantitative changes of activity in common, orientation-general mechanisms along the central face processing route and broader involvement of generally object-tuned areas for inverted faces.
What are the implications for the debate about potential qualitative changes of face processing due to picture-plane inversion? Assuming that performance in a given test condition relies on task-related and condition-specific processes, our findings imply only quantitative changes in the more general, task-related processes and putative qualitative changes in the components that are added to handle a specific test condition. Individual differences analysis cannot further resolve the latter, because it rests on common variance. Hence, the condition-specific processes which contingently change with stimulus orientation could be strong unique components, or fall into several scattered and inconsiderable ones.
The single test results of this study, however, do not support major qualitative changes due to picture-plane inversion. There were strong interactions of congruency (CCP), and opacity (SPP), with orientation. However, closer analysis showed that the effect of external context was diminished for inverted faces, but it remained substantial. Likewise, we found that attentional competition among target object and masker was stronger for inverted presentation, indicating that upright stimuli yielded more robust object representations. Yet, opacity effects were strong in either orientation, indicating that inversion did not affect the kind of attentional interaction. Hence, the experimental results of the tests used here add to a line of evidence showing that, albeit with reduced efficiency, the way observers handle task demands does not change qualitatively with stimulus inversion [20–22].^11^
The finding of orientation-invariant correlation structures, though, could derive from the specific test selection. We tested face recognition, face identity matching with congruent and incongruent external features, and object-based attention to faces and scenes, while sensitivity to facial feature spacing [25,70,71], matching face parts and wholes [10] and face identity matching with internal feature composites [11,72] were not tested. Since the discussion about potential orientation-specific processing frequently referred to experimental results from spacing and composite face tasks [2], it would be highly desirable to have individual difference results from these paradigms, based on a large sample. This could clarify whether there is indeed orientation-invariance of common variance in a wide range of tasks for key aspects of face processing. A further limitation is that only one test for non-face objects was included. Recent research showed the importance of using several different object categories, since the distinctiveness of faces is not necessarily greater than the distinctiveness of other object categories (e.g. cars, [36]), which could be uncovered only by using objects from many different classes, as in the Vanderbilt Expertise Test [37]. Using a wide range of object classes and controlling for expertise, a domain-general object ability could be revealed, which proved to be rather independent from general intelligence and other cognitive and perceptual skills [73,74]. With such a comprehensive approach, it would be challenging to show that domain-general abilities are independent of stimulus orientation. The present study was a first attempt to reveal whether and how stimulus orientation modifies the pattern of associations and dissociations in face processing tasks. Further experimentation in the framework of the individual differences approach is necessary to gauge whether orientation-invariance in the shared variance components of face- and non-face related tasks can claim generality.
Studying face recognition, face perception and face-selective attention with a comprehensive test battery has revealed orientation-invariance of common variance within and among these domains of face-processing. Face inversion impaired performance seriously, but did not change the specific patterns of associations and dissociations within and across tests. This suggests that process-specific, but orientation-general mechanisms drove common variance in the face related tasks. Our findings add to a line of research which shows that, after abstracting from the performance decline for inverted presentation, key features of face processing, like spatial frequency tuning or cue usage in face image classification, are orientation-invariant.