ML Research Wiki / Benchmarks / Video Prediction / KTH

KTH

Video Prediction Benchmark

Performance Over Time

📊 Showing 28 results | 📏 Metric: FVD

Top Performing Models

Rank	Model	Paper	FVD	Date	Code
1	WAM	Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Prediction	29.85	2020-02-23	📦 Bei-Jin/STMFANet
2	PredRNN++	PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning	28.47	2018-04-17	📦 thuml/predrnn-pytorch 📦 Yunbo426/predrnn-pp 📦 mindspore-ai/models
3	PredRNN-V2	PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning	28.37	2021-03-17	📦 chengtan9907/simvpv2 📦 thuml/predrnn-pytorch 📦 ksm26/spatiotemporal-predictions
4	MSPred	MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networks	27.81	2022-03-17	📦 AIS-Bonn/MSPred
5	SAVP-VAE	Stochastic Adversarial Video Prediction	27.77	2018-04-04	📦 alexlee-gk/video_prediction 📦 MIT-Omnipush/video-prediction 📦 kamran0153/impact-of-data-freshness-in-learning 📦 Bonennult/video_prediction
6	Conv-TT-LSTM	Convolutional Tensor-Train LSTM for Spatio-temporal Learning	27.62	2020-02-21	📦 NVlabs/conv-tt-lstm 📦 jerrywn121/TianChi_AIEarth
7	LMC	Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning	27.50	2021-04-02	📦 sangmin-git/LMC-Memory
8	DFN	Dynamic Filter Networks	27.26	2016-05-31	📦 dbbert/dfn
9	Grid-keypoints	Accurate Grid Keypoint Learning for Efficient Video Prediction	27.11	2021-07-28	📦 xjgaocs/Grid-Keypoint-Learning
10	MSNET	Mutual Suppression Network for Video Prediction using Disentangled Features	27.08	2018-04-13	-

All Papers (28)

Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Prediction

2020

WAM

Bei-Jin/STMFANet

PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning

2018

PredRNN++

thuml/predrnn-pytorch Yunbo426/predrnn-pp

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

2021

PredRNN-V2

chengtan9907/simvpv2 thuml/predrnn-pytorch ksm26/spatiotemporal-predictions

MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networks

2022

MSPred

AIS-Bonn/MSPred

Stochastic Adversarial Video Prediction

2018

SAVP-VAE

alexlee-gk/video_prediction MIT-Omnipush/video-prediction

Convolutional Tensor-Train LSTM for Spatio-temporal Learning

2020

Conv-TT-LSTM

NVlabs/conv-tt-lstm jerrywn121/TianChi_AIEarth

Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning

2021

LMC

sangmin-git/LMC-Memory

Dynamic Filter Networks

2016

DFN

dbbert/dfn

Accurate Grid Keypoint Learning for Efficient Video Prediction

2021

Grid-keypoints

xjgaocs/Grid-Keypoint-Learning

Mutual Suppression Network for Video Prediction using Disentangled Features

2018

MSNET

Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model

2017

TrajGRU

Hzzone/Precipitation-Nowcasting sxjscience/HKO-7

Decomposing Motion and Content for Natural Video Sequence Prediction

2017

MCnet + Residual

rubenvillegas/iclr2017mcnet

Folded Recurrent Neural Networks for Future Video Prediction

2017

fRNN

moliusimon/frnn

Stochastic Adversarial Video Prediction

2018

SAVP-VAE (from Grid-keypoints)

alexlee-gk/video_prediction MIT-Omnipush/video-prediction

Decomposing Motion and Content for Natural Video Sequence Prediction

2017

MCnet

rubenvillegas/iclr2017mcnet

Stochastic Variational Video Prediction

2017

SV2P time-invariant (from Grid-keypoints)

StanfordVL/roboturk_real_dataset RoboTurk-Platform/roboturk_real_dataset suraj-nair-1/google-research

Stochastic Variational Video Prediction

2017

SV2P time-invariant (from Grid-keypoints)

StanfordVL/roboturk_real_dataset RoboTurk-Platform/roboturk_real_dataset suraj-nair-1/google-research

Unsupervised Learning of Object Structure and Dynamics from Videos

2019

Struct-VRNN (from Grid-keypoints)

google-research/google-research

Stochastic Video Generation with a Learned Prior

2018

SVG-LP (from Grid-keypoints)

edenton/svg joelouismarino/amortized-variational-filtering MIT-Omnipush/video-prediction

Stochastic Adversarial Video Prediction

2018

SAVP (from Grid-keypoints)

alexlee-gk/video_prediction MIT-Omnipush/video-prediction

Video Pixel Networks

2016

VPN

3ammor/Video-Pixel-Networks

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

2015

ConvLSTM

ndrplz/ConvLSTM_pytorch chengtan9907/simvpv2

Stochastic Latent Residual Video Prediction

2020

SRVP

edouardelasalles/srvp

SLAMP: Stochastic Latent Appearance and Motion Prediction

2021

SLAMP

kaanakan/slamp

Stochastic Adversarial Video Prediction

2018

SAVP (from SRVP)

alexlee-gk/video_prediction MIT-Omnipush/video-prediction

Stochastic Video Generation with a Learned Prior

2018

SVG-LP (from SRVP)

edenton/svg joelouismarino/amortized-variational-filtering MIT-Omnipush/video-prediction

Stochastic Variational Video Prediction

2017

SV2P (from SRVP)

StanfordVL/roboturk_real_dataset RoboTurk-Platform/roboturk_real_dataset suraj-nair-1/google-research

Diverse Video Generation using a Gaussian Process Trigger

2021

DVG

shgaurav1/DVG

KTH

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (28)

Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Prediction

PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networks

Stochastic Adversarial Video Prediction

Convolutional Tensor-Train LSTM for Spatio-temporal Learning

Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning

Dynamic Filter Networks

Accurate Grid Keypoint Learning for Efficient Video Prediction

Mutual Suppression Network for Video Prediction using Disentangled Features

Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model

Decomposing Motion and Content for Natural Video Sequence Prediction

Folded Recurrent Neural Networks for Future Video Prediction

Stochastic Adversarial Video Prediction

Decomposing Motion and Content for Natural Video Sequence Prediction

Stochastic Variational Video Prediction

Stochastic Variational Video Prediction

Unsupervised Learning of Object Structure and Dynamics from Videos

Stochastic Video Generation with a Learned Prior

Stochastic Adversarial Video Prediction

Video Pixel Networks

Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Stochastic Latent Residual Video Prediction

SLAMP: Stochastic Latent Appearance and Motion Prediction

Stochastic Adversarial Video Prediction

Stochastic Video Generation with a Learned Prior

Stochastic Variational Video Prediction

Diverse Video Generation using a Gaussian Process Trigger

Model	Paper	FVD	Date
WAM	Exploring Spatial-Temporal Multi-Frequency Analys…	29.85	2020-02-23
PredRNN++	PredRNN++: Towards A Resolution of the Deep-in-Ti…	28.47	2018-04-17
PredRNN-V2	PredRNN: A Recurrent Neural Network for Spatiotem…	28.37	2021-03-17
MSPred	MSPred: Video Prediction at Multiple Spatio-Tempo…	27.81	2022-03-17
SAVP-VAE	Stochastic Adversarial Video Prediction	27.77	2018-04-04
Conv-TT-LSTM	Convolutional Tensor-Train LSTM for Spatio-tempor…	27.62	2020-02-21
LMC	Video Prediction Recalling Long-term Motion Conte…	27.50	2021-04-02
DFN	Dynamic Filter Networks	27.26	2016-05-31
Grid-keypoints	Accurate Grid Keypoint Learning for Efficient Vid…	27.11	2021-07-28
MSNET	Mutual Suppression Network for Video Prediction u…	27.08	2018-04-13
TrajGRU	Deep Learning for Precipitation Nowcasting: A Ben…	26.97	2017-06-12
MCnet + Residual	Decomposing Motion and Content for Natural Video …	26.29	2017-06-25
fRNN	Folded Recurrent Neural Networks for Future Video…	26.12	2017-12-01
SAVP-VAE (from Grid-keypoints)	Stochastic Adversarial Video Prediction	26.00	2018-04-04
MCnet	Decomposing Motion and Content for Natural Video …	25.95	2017-06-25
SV2P time-invariant (from Grid-keypoints)	Stochastic Variational Video Prediction	25.87	2017-10-30
SV2P time-invariant (from Grid-keypoints)	Stochastic Variational Video Prediction	25.70	2017-10-30
Struct-VRNN (from Grid-keypoints)	Unsupervised Learning of Object Structure and Dyn…	24.29	2019-06-19
SVG-LP (from Grid-keypoints)	Stochastic Video Generation with a Learned Prior	23.91	2018-02-21
SAVP (from Grid-keypoints)	Stochastic Adversarial Video Prediction	23.79	2018-04-04
VPN	Video Pixel Networks	23.76	2016-10-03
ConvLSTM	Convolutional LSTM Network: A Machine Learning Ap…	23.58	2015-06-13
SRVP	Stochastic Latent Residual Video Prediction	10.00	2020-02-21
SLAMP	SLAMP: Stochastic Latent Appearance and Motion Pr…	10.00	2021-08-05
SAVP (from SRVP)	Stochastic Adversarial Video Prediction	10.00	2018-04-04
SVG-LP (from SRVP)	Stochastic Video Generation with a Learned Prior	10.00	2018-02-21
SV2P (from SRVP)	Stochastic Variational Video Prediction	10.00	2017-10-30
DVG	Diverse Video Generation using a Gaussian Process…	0.48	2021-07-09