Decoding visual working memory in a dynamic context

Abstract

Visual working memory (VWM) is the brain’s mechanism for briefly retaining visual information for imminent goal-directed behavior. Existing research has pointed to different brain areas for VWM storage, prompting a discussion. We propose that these conflicting results arise from the static approach with which VWM maintenance has previously been examined. VWM is used to guide behavior in an inherently dynamic visual environment and might therefore be best considered as a dynamic system itself. For instance, task-related goals can change from a moment-to-moment basis, temporarily making some visual stimuli more task-relevant than others. Moreover, brain activity is known to be inherently dynamic, posing the question of whether VWM representations are maintained in a dynamic or stable manner. To address the existing conflict, we have investigated VWM maintenance in two ways: (1) by examining the maintenance of VWM representations over time to adhere to the brain’s inherent dynamics and (2) by exploring the differences between VWM representations with different task-relevance states to take the dynamic visual environment into account. Relevancy states were initiated by task-related goals where items could either be currently task-relevant or prospectively task-relevant. Using a 7-Tesla fMRI dataset and inverted encoding models we have decoded VWM information throughout the human cortex (N=3). We have found that VWM representations are stored in occipital and parietal areas in the cortex, confirming earlier findings. Within these areas, VWM information is maintained in a stable representational format over the course of the 8 seconds retention period. Furthermore, differences in storage location between currently- and prospectively-relevant representations were revealed: currently-relevant representations were present in occipital and parietal areas, while prospectively-relevant representations were present in frontal and parietal areas. Additionally, we found that currently- and prospectively-relevant representations could be stored in similar and even in opposite representational formats in the parietal lobe. These findings show that VWM representations and their storage location can differ between relevancy states, emphasizing the importance of taking dynamic changes into account.