Recently, the validity of
combining EEG with VR in studying language processing in naturalistic
environments has been confirmed. By combining VR and EEG, strictly controlled
experiments in a more naturalistic environment would be comfortably in your
hands to get an explicit understanding of how we process language. EEG combined
with VR would, make it possible to correlate humans’ physiological signals with
their every single movement in the designed environment. Thus, the successful
combination of the two has been used to study users’ driving behavior, spatial
navigation, spatial presence and more. Researchers conducted an experiment to
validate the combined use of VR and EEG as a tool to study neurophysiological
mechanisms of language processing and comprehension. They decided to prove the
validity by showing that the N400 response happens similarly in a virtual
environment. The N400 refers to an event-related potential (ERP) component that
peaks around 400ms after the critical stimuli; the previous study in a
traditional setting have found that incongruence between the spoken and visual
stimuli will cause enhanced N400. Therefore, the research team set up the
situation containing mismatches of verbal and visual stimuli and analyzed
brainwave to observe N400 response. In the experiment, total 25 people were put
into the virtual environment where eight tables are in a row with a virtual
guest sitting at each table in a virtual restaurant. The participants were
moved from a table to table following the pre-programmed procedure.
The materials consisted of 80 objects
and 96 sentences (80 experimental sentences and 16 filler ones). Both of them
were relevant with restaurant setting, but only half of the object and sentence
pairs were semantically matched. Each of the participants went through equal
rate of match and mismatch situations and made 12 rounds through the restaurant
during the entire experiment. At the end of the trial, they were asked two
questions to assess whether the participants had paid attention during the
trial and their perceptions of the virtual agents. The EEG was recorded from 59
active electrodes during the entire rounds of the experiment. Epochs from 100ms
preceding the onset of the critical noun to 1200ms after it was selected and
the ERPs were further calculated and analyzed per participant and condition in
three time windows: N400 window (350–600ms), an earlier window (250–350ms) and
a later window (600–800ms). Finally, repeated measures of analyses of variance
(ANOVAs) were performed, three variables were predetermined time windows, and
the factors included condition (match, mismatch), region (vertical midline,
left anterior, right anterior, left posterior, left interior), and the
electrode. It was revealed that ERPs seem more negative for the mismatch
condition than for the match condition in all time windows and the difference
was particularly significant during the N400 window. That is to say, the N400
response was observed in line with predictions, while leading to the conviction
that VR and EEG combined can be used to study language comprehension.
More information: