Skip to content

A deep convolutional neural network for multi-frame video interpolation.

License

Notifications You must be signed in to change notification settings

susomena/DeepSlowMotion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

6fda6ba · Jul 31, 2019

History

26 Commits
Jul 11, 2019
Aug 8, 2018
Jul 10, 2019
Jul 19, 2019
Jul 10, 2019
Jul 31, 2019
Jul 18, 2019
Jul 31, 2019
Jul 10, 2019
Jul 10, 2019

Repository files navigation

DeepSlowMotion

A deep convolutional neural network for multi-frame video interpolation. Based on the work of Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller and Jan Kautz. To see the original work, please see:

H. Jiang, D. Sun, V. Jampani, M.-H. Yang, E. Learned-Miller and J. Kautz, "Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation," Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 2018, pp. 9000-9008.

or go to: arXiv:1712.00080

The script download_dataset.sh downloads two datasets, the Adobe 240-fps dataset and the Need for Speed dataset, that can be used for training the neural network. For more information on these datasets, please see:

S. Su, M. Delbracio, J. Wang, G. Sapiro, W. Heidrich and O. Wang, "Deep Video Deblurring for Hand-Held Cameras," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 237-246.

or go to: arXiv:1611.08387

H. K. Galoogahi, A. Fagg, C. Huang, D. Ramanan and Simon Lucey, "Need for Speed: A Benchmark for Higher Frame Rate Object Tracking," Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 1125-1134.

or go to: arXiv:1703.05884

In order to download both datasets with the script, just type the following command in a terminal:

./download_dataset.sh path_to_folder

where path_to_folder is the directory where the dataset should be downloaded. This directory is created if it does not exist.

The loss function includes a term that depends on the activations of a convolutional layer of the VGG16 neural network. The pretrained weights of the VGG16 neural networks are stored in the vgg16_weights_no_fc.npz file. This file is a modified version of the file that can be found in this link. In order to reduce the size of the file, the weights of the fully connected layers have been removed. For more information on the VGG16 neural network, please see:

K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2015, pp. 1-14.

or go to: arXiv:1409.1556

About

A deep convolutional neural network for multi-frame video interpolation.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published