Multi-Level Adaptive Separable Convolution for Large-Motion Video Frame Interpolation

Current state-of-the-art methods within Video Frame Interpolation (VI) fail at synthesizing interpolated frames in certain problem areas, such as when the video contains large motion. This work aims at improving performance on frame sequences containing large displacements by extending the Adaptive Separable Convolution model in two ways. First of all, we increase the receptive field of the model by utilizing spatial pyramids, which efficiently increase the interpolation kernel size. We additionally adapt the network to accommodate for four frames, as opposed to just two, which should give it the ability to learn more complex motion patterns. This work also introduces the Large-Motion Video Interpolation Dataset (LMD), which contains extracted frames from videos containing large displacements and highly non-linear movements. Our analysis shows that applying the model changes, together with the use of our new dataset, does indeed result in improved performance on large displacement videos. We also show that the increase in performance generalizes to frame sequences of all sorts by outperforming other models in our benchmark on most tasks, and almost setting the new state-of-the-art on the Vimeo-90K dataset.