A recent study published in Cell Reports revealed the neural code for incomplete faces in the macaque face patches. This work was performed by researchers in Dr. CHANG Le’s Lab at the Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology of the Chinese Academy of Sciences. This work systematically investigated how faces are represented in the primate brain when only a part of the face is shown, by conducting electrophysiological recordings in face-selective areas of the macaque inferotemporal cortex, paving the way for understanding the code of faces under natural conditions. 

Face recognition is an important capability that is critical to our daily lives and social interactions. Brain regions located in the inferotemporal cortex of the primate brain, called “face patches”, are dedicated to processing faces, and a series of studies have revealed the neural code of faces in these regions. While these studies typically used complete faces as visual stimuli, incomplete faces are often encountered in real life, such as when wearing face masks is required. In this context, how is facial information encoded in face-selective neurons, or “face cells”? For example, how do we distinguish between complete and incomplete faces? How is facial identity encoded when faces are incomplete? Previous studies have suggested that individual face cells encode local face parts, but this seems to be at odds with our ability to recognize faces as a whole. 

In this study, incomplete faces were synthesized in a quantitative manner by controlling the following parameters: facial features related to identity, the way the faces were rendered incomplete (as fragments or as occluded faces), and the location of the fragment or occluder relative to the complete face (Figure A). Using fMRI-guided electrophysiology (Figure B), the responses of face cells to the incomplete faces were recorded. 

When analyzing the responses of individual neurons, quite diverse patterns were observed. In particular, the critical regions identified in two different conditions, fragments and occluded faces, are largely dissociated in many face cells (Figure C). This inconsistency between two stimulus conditions is strongly correlated with the nonlinear spatial integration of different face parts (Figure D). At the population level, this nonlinearity was manifested as a curved manifold representing how complete the face is (Figure E). Such a curving representation makes it easy to distinguish between faces with varying degrees of completeness, such as between incomplete and complete faces. Furthermore, facial identity is represented in a condition general way, supporting an easy readout of identity information in downstream areas (Figure F). Finally, the dissociation between critical regions was also observed when comparing two different aspects of face processing: face detection and face discrimination.  

This study has systematically revealed the code for incomplete faces in macaque face patches, demonstrating that individual face cells do not simply encode local parts, but instead integrate information from multiple face parts. Although this coding scheme sounds complex, the diversity in single-neuron responses can be viewed as projections of a curved manifold in the state space onto the axes of single neurons. This study also sheds light on the understanding of artificial intelligent systems operating under natural conditions. 

This paper entitled “Representational geometry of incomplete faces in macaque face patches” was published online in Cell Reports on June 20, 2023. LI Dongyuan is the first author. This work was supported by grants from the Ministry of Science and Technology,  Chinese Academy of Sciences, Shanghai Municipality, and National Natural Science Foundation of China.