5,710
edits
Changes
→Conclusions and Future Directions
This piece has a few obvious limitations, but also many strengths, pointing to future directions of work as the technical infrastructure develops. In aestheticizing “the look” of the Nishimoto video reconstructions or “the sound” of the Pasley audio reconstructions, we are dwelling on artifacts of the data collection process/training dataset/reconstruction algorithm more than anything specific to the mind of the subject. Ideally we could image some higher-level responses in the viewing subject (beyond primary visual areas). I’m sure many neuroscientists would also like to do this. Decoding higher-level processing would give us more meat for the project, more of a cognitive response as opposed to an evoked perceptual one.
Another shortcoming of the project is its essential theatricality. The videos are produced from averaged, highly correlated 10 second clips of source materials. Similar aesthetic effects to the ghostlike compositing could be achieved with simple algorithmic processes.So in essence, the project relies on the presence of the MRI magnet and the gesture of “peering inside the brain” of another as central vehicles for meaning.
There is much positive potential in this project. Each audio-visual piece produced is uniquely tied to the person of the test subject, the particular of the session, stimulus material, and context that produced it. There is great opportunity for dissonance between source material and filtered signal. Rather than settling for samples extracted from Apple movie trailers and Youtube, you could for instance evoke reactions to video of gory monkey-brain surgery and train/reconstruct the imagery from a database of national geographic videos.
Future directions to explore include the modeling and reconstruction of perceptions by non-human subjects (simulating what your dog sees), implementing audio reconstruction with non-invasive methods (i.e. not ECoG), attempting to evoke and reconstruct responses in other sensory modalities. Additionally with the improvements in the portability of MRI technology or the development of over sensing modalities (EEG) there could be new opportunities for reconstructing internal, sensory experience.