A single spatial transform improves predictions of neural responses by deep neural network models

Encoding models are a powerful tool for predicting neural responses on a per-image basis using the features of deep neural networks (DNNs). Efforts to improve prediction performance have largely focused on changing aspects of DNN training or model architecture. Here, we take a pre-trained DNN and explore whether a fixed, spatial reweighting of features can improve neural predictions without the need for retraining the neural network. We find that spatially distinct areas of visual input (center versus periphery) uniquely contribute to the temporal dynamics of human EEG recordings. These dynamics are unified when transforming feature maps based on ganglion cell sampling (GCS). The same GCS transform improved predictions of both monkey electrophysiology and human fMRI recordings.