Schizophrenia is marked by aberrant processing of complex speech and gesture, which may contribute functionally to its impaired social communication. To date, extant neuroscientific studies of schizophrenia have largely investigated dysfunctional speech and gesture in isolation, and no prior research has examined how the two communicative channels may interact in more natural contexts. Here, we tested if patients with schizophrenia show aberrant neural processing of semantically complex story segments, and if speech-associated gestures (co-speech gestures) might modulate this effect. In a functional MRI study, we presented to 34 participants (16 patients and 18 matched-controls) an ecologically-valid retelling of a continuous story, performed via speech and spontaneous gestures. We split the entire story into ten-word segments, and measured the semantic complexity for each segment with idea density, a linguistic measure that is commonly used clinically to evaluate aberrant language dysfunction at the semantic level. Per segment, the presence of numbers of gestures varied (n = 0, 1, +2). Our results suggest that, in comparison to controls, patients showed reduced activation for more complex segments in the bilateral middle frontal and inferior parietal regions. Importantly, this neural aberrance was normalized in segments presented with gestures. Thus, for the first time with a naturalistic multimodal stimulation paradigm, we show that gestures reduced group differences when processing a natural story, probably by facilitating the processing of semantically complex segments of the story in schizophrenia.

The current study investigated the neuro mechanisms of emoji processing as sentence predicate in written context. In the hybrid textuality which is more cognitively engaging, emojis in sentential intermediate positions were designed as either congruent or incongruent to the context. The results showed that incongruent words led to a robust N400 effect, while incongruent emojis only elicited the P600 effect. It implies that semantics and syntax of words can be separated while those of emojis seem to be integrated together. That is, when the meaning of the emoji is violated to the sentential context, its grammatical role cannot be well interpreted, especially when it is used as a key grammatical component in a sentence, such as the predicate. Thus, it shows that even though the meaning of emojis can be interpreted by readers, their syntactic and semantic functions cannot be clearly separated. In comparison with word processing, the larger amplitude with emojis in the time window of 350-500 ms shows more cognitive efforts in emoji semantic processing, possibly arising from the switch of modalities within the visual channel, that is, the multimodal cognitive load.

Humans can read and comprehend text rapidly, implying that readers might process multiple words per fixation. However, the extent to which parafoveal words are previewed and integrated into the evolving sentence context remains disputed. We investigated parafoveal processing during natural reading by recording brain activity and eye movements using MEG and an eye tracker while participants silently read one-line sentences. The sentences contained an unpredictable target word that was either congruent or incongruent with the sentence context. To measure parafoveal processing, we flickered the target words at 60 Hz and measured the resulting brain responses (i.e. Rapid Invisible Frequency Tagging, RIFT) during fixations on the pre-target words. Our results revealed a significantly weaker tagging response for target words that were incongruent with the previous context compared to congruent ones, even within 100ms of fixating the word immediately preceding the target. This reduction in the RIFT response was also found to be predictive of individual reading speed. We conclude that semantic information is not only extracted from the parafovea but can also be integrated with the previous context before the word is fixated. This early and extensive parafoveal processing supports the rapid word processing required for natural reading. Our study suggests that theoretical frameworks of natural reading should incorporate the concept of deep parafoveal processing.

The way we establish meaning has been a profound question not only in language research but in developmental science as well. The relation between linguistic form and content has been loosened up in recent pragmatic approaches to communication, showing that code-based models of language comprehension must be augmented by context-sensitive, pragmatic-inferential mechanisms to recover the speaker\'s intended meaning. Language acquisition has traditionally been thought to involve building a mental lexicon and extracting syntactic rules from noisy linguistic input, while communicative-pragmatic inferences have also been argued to be indispensable. Recent research findings exploring the electrophysiological indicator of semantic processing, the N400, have raised serious questions about the traditional separation between semantic decoding and pragmatic inferential processes. The N400 appears to be sensitive to mentalization-the ability to attribute beliefs to social partners-already from its developmental onset. This finding raises the possibility that mentalization may not simply contribute to pragmatic inferences that enrich linguistic decoding processes but that the semantic system may be functioning in a fundamentally mentalistic manner. The present review first summarizes the key contributions of pragmatic models of communication to language comprehension. Then, it provides an overview of how communicative intentions are interpreted in developmental theories of communication, with a special emphasis on mentalization. Next, it discusses the sensitivity of infants to the information-transmitting potential of language, their ability to pick up its code-like features, and their capacity to track language comprehension of social partners using mentalization. In conclusion, I argue that the recovery of meaning during linguistic communication is not adequately modeled as a process of code-based semantic retrieval complemented by pragmatic inferences. Instead, the semantic system may establish meaning, as intended, during language comprehension and acquisition through mentalistic attribution of content to communicative partners.

This study examined lexical-semantic processing in children with Developmental Language Disorder (DLD) during visually situated comprehension of real-time spoken words. Existing evidence suggests that children with DLD may experience challenges in lexical access and retrieval, as well as greater lexical competition compared to their peers with Typical Development (TD). However, the specific nature of these difficulties remains unclear. Using eye-tracking methodology, the study investigated the real-time comprehension of semantic relationships in children with DLD and their age-matched peers. The results revealed that, for relatively frequent nouns, both groups demonstrated similar comprehension of semantic relationships. Both groups favored the semantic competitor when it appeared with an unrelated visual referent. In turn, when the semantic competitor appeared with the visual referent of the spoken word, both groups disregarded the competitor. This finding shows that, although children with DLD usually present a relatively impoverished vocabulary, frequent nouns may not pose greater difficulties for them. While the temporal course of preference for the competitor or the referent was similar between the two groups, numerical, though non-significant, differences in the extension of the clusters were observed. In summary, this research demonstrates that monolingual preschoolers with DLD exhibit similar lexical access to frequent words compared to their peers with TD. Future studies should investigate the performance of children with DLD on less frequent words to provide a comprehensive understanding of their lexical-semantic abilities.

We investigated whether, during visual word recognition, semantic processing is modulated by attentional control mechanisms directed at matching semantic information with task-relevant goals. In previous research, we analyzed the semantic Stroop interference as a function of response latency (delta-plot analyses) and found that this phenomenon mainly occurs in the slowest responses. Here, we investigated whether this pattern is due to reduced ability to proactively maintain the task goal in these slowest trials. In two pairs of experiments, participants completed two semantic Stroop tasks: a classic semantic Stroop task (Experiment 1A and 2A) and a semantic Stroop task combined with an n-back task (Experiment 1B and 2B). The two pairs of experiments only differed in the trial pace, which was slightly faster in Experiments 2A and 2B than in Experiments 1A and 1B. By taxing the executive control system, the n-back task was expected to hinder proactive control. Delta-plot analyses of the semantic Stroop task replicated the enhanced effect in the slowest responses, but only under sufficient time pressure. Combining the semantic Stroop task with the n-back task produced a change in the distributional profile of semantic Stroop interference, which we ascribe to a general difficulty in the use of proactive control. Our findings suggest that semantic Stroop interference is, to some extent, dependent on the available executive resources, while also being sensitive to subtle variations in task conditions.

Pupil dilation has been associated with the effort required to perform various cognitive tasks. At the lexical level, some studies suggest that this neurophysiological measure would provide objective, real-time information during word processing and lexical access. However, due to the scarcity and incipient advancement of this line of research, its applicability, use, and sensitivity are not entirely clear. This scoping review aims to determine the applicability and usefulness of pupillometry in the study of lexical access by providing an up-to-date overview of research in this area. Following the PRISMA protocol, 16 articles were included in this review. The results show that pupillometry is a highly applicable, useful, and sensitive method for assessing lexical skills of word recognition, word retrieval, and semantic activation. Moreover, it easily fits into traditional research paradigms and methods in the field. Because it is a non-invasive, objective, and automated procedure, it can be applied to any population or age group. However, the emerging development of this specific area of research and the methodological diversity observed in the included studies do not yet allow for definitive conclusions in this area, which in turn does not allow for meta-analyses or fully conclusive statements about what the pupil response actually reflects when processing words. Standardized pupillary recording and analysis methods need to be defined to generate more accurate, replicable research designs with more reliable results to strengthen this line of research.

The hub-and-spoke theory of semantic representation fractionates the neural underpinning of semantic knowledge into two essential components: the sensorimotor modality-specific regions and a crucially important semantic hub region. Our previous study in patients with semantic dementia has found that the hub region is located in the left fusiform gyrus. However, because this region is located within the brain damage in patients with semantic dementia, it is not clear whether the semantic deficit is caused by structural damage to the hub region itself or by its disconnection from other brain regions. Stroke patients do not have any damage to the left fusiform gyrus, but exhibit amodal and modality-specific deficits in semantic processing. Therefore, in this study, we validated the semantic hub region from a brain network perspective in 79 stroke patients and explored the white matter connections associated with it. First, we collected data of diffusion-weighted imaging and behavioural performance on general semantic tasks and modality-specific semantic tasks (assessing object knowledge on form, colour, motion, sound, manipulation and function). We then used correlation and regression analyses to examine the association between the nodal degree values of brain regions in the whole-brain structural network and general semantic performance in the stroke patients. The results revealed that the connectivity of the left fusiform gyrus significantly predicted general semantic performance, indicating that this region is the semantic hub. To identify the semantic-relevant connections of the semantic hub, we then correlated the white matter integrity values of each tract connected to the left fusiform gyrus separately with performance on general and modality-specific semantic processing. We found that the hub region accomplished general semantic processing through white matter connections with the left superior temporal pole, middle temporal gyrus, inferior temporal gyrus and hippocampus. The connectivity between the hub region and the left hippocampus, superior temporal pole, middle temporal gyrus, inferior temporal gyrus and parahippocampal gyrus was differentially involved in object form, colour, motion, sound, manipulation and function processing. After statistically removing the effects of potential confounding variables (i.e. whole-brain lesion volume, lesion volume of regions of interest and performance on non-semantic control tasks), the observed effects remained significant. Together, our findings support the role of the left fusiform gyrus as a semantic hub region in stroke patients and reveal its crucial connectivity in the network. This study provides new insights and evidence for the neuroanatomical organization of semantic memory in the human brain.

UNASSIGNED: Posterior Cortical Atrophy (PCA) is a syndrome characterized by a progressive decline in higher-order visuospatial processing, leading to symptoms such as space perception deficit, simultanagnosia, and object perception impairment. While PCA is primarily known for its impact on visuospatial abilities, recent studies have documented language abnormalities in PCA patients. This study aims to delineate the nature and origin of language impairments in PCA, hypothesizing that language deficits reflect the visuospatial processing impairments of the disease.
UNASSIGNED: We compared the language samples of 25 patients with PCA with age-matched cognitively normal (CN) individuals across two distinct tasks: a visually-dependent picture description and a visually-independent job description task. We extracted word frequency, word utterance latency, and spatial relational words for this comparison. We then conducted an in-depth analysis of the language used in the picture description task to identify specific linguistic indicators that reflect the visuospatial processing deficits of PCA.
UNASSIGNED: Patients with PCA showed significant language deficits in the visually-dependent task, characterized by higher word frequency, prolonged utterance latency, and fewer spatial relational words, but not in the visually-independent task. An in-depth analysis of the picture description task further showed that PCA patients struggled to identify certain visual elements as well as the overall theme of the picture. A predictive model based on these language features distinguished PCA patients from CN individuals with high classification accuracy.
UNASSIGNED: The findings indicate that language is a sensitive behavioral construct to detect visuospatial processing abnormalities of PCA. These insights offer theoretical and clinical avenues for understanding and managing PCA, underscoring language as a crucial marker for the visuospatial deficits of this atypical variant of Alzheimer\'s disease.

There is a misconception that pictures are easy to comprehend, which is problematic in pedagogical practices that include pictures. For example, if a child has difficulties with verbal narration to picture sequences, it may be interpreted as specific to spoken language even though the child may have additional difficulties with comprehension of visual narratives in the form of picture sequences. The purpose of the present study was therefore to increase our understanding of semantic processing in the pictorial domain in relation to semantic processing in the verbal domain, focusing on 9-13 years-old children with typical language development. To this end, we measured electrical brain responses (event related potentials, ERPs) in 17 children to (i) pictures (panels) that were predicted versus unpredicted in sequences of panels that conveyed visual narratives and (ii) words that were predicted versus unpredicted in sentences that conveyed verbal narratives. Results demonstrated similarities as there were no significant difference in the magnitude of the N400 effect across domains. The only difference between domains was the predicted difference in distribution, that is, a more posterior N400 effect in the verbal domain than in the pictorial domain. The study contributes to an increased understanding of the complexity of processing of visual narratives and its shared features with processing of verbal narratives, which should be considered in pedagogical practices.