ML Research Wiki / Benchmarks / Zero-Shot Action Recognition / HMDB51

HMDB51

Zero-Shot Action Recognition Benchmark

Performance Over Time

📊 Showing 24 results | 📏 Metric: Top-1 Accuracy

Top Performing Models

Rank	Model	Paper	Top-1 Accuracy	Date	Code
1	MSQNet	Actor-agnostic Multi-label Action Recognition with Multi-modal Query	69.43	2023-07-20	📦 mondalanindya/msqnet
2	MOV (ViT-L/14)	Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models	64.70	2022-07-15	-
3	OTI(ViT-L/14)	Orthogonal Temporal Interpolation for Zero-Shot Video Recognition	64.00	2023-08-14	📦 sweetorangezhuyan/mm2023_oti
4	BIKE	Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models	61.40	2022-12-31	📦 whwu95/Cap4Video 📦 whwu95/text4vis 📦 whwu95/GPT4Vis 📦 whwu95/BIKE 📦 whwu95/ATM
5	MOV (ViT-B/16)	Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models	60.80	2022-07-15	-
6	IMP-MoE-L 📚	Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception	59.10	2023-05-10	-
7	VideoCoCa 📚	VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners	58.70	2022-12-09	-
8	Text4Vis	Revisiting Classifier: Transferring Vision-Language Models for Video Recognition	58.40	2022-07-04	📦 whwu95/Cap4Video 📦 whwu95/text4vis 📦 whwu95/GPT4Vis 📦 whwu95/BIKE 📦 whwu95/ATM
9	TC-CLIP	Leveraging Temporal Contextualization for Video Action Recognition	56.00	2024-04-15	📦 naver-ai/tc-clip 📦 naver-ai/dawin
10	OST	OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition	55.90	2023-11-30	📦 tomchen-ctj/OST

All Papers (24)

Actor-agnostic Multi-label Action Recognition with Multi-modal Query

2023

MSQNet

mondalanindya/msqnet

Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models

2022

MOV (ViT-L/14)

Orthogonal Temporal Interpolation for Zero-Shot Video Recognition

2023

OTI(ViT-L/14)

sweetorangezhuyan/mm2023_oti

Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models

2022

BIKE

whwu95/Cap4Video whwu95/text4vis

Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models

2022

MOV (ViT-B/16)

Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception

2023

IMP-MoE-L

VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners

2022

VideoCoCa

Revisiting Classifier: Transferring Vision-Language Models for Video Recognition

2022

Text4Vis

whwu95/Cap4Video whwu95/text4vis

Leveraging Temporal Contextualization for Video Action Recognition

2024

TC-CLIP

naver-ai/tc-clip naver-ai/dawin

OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition

2023

OST

tomchen-ctj/OST

MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge

2023

MAXI

wlin-at/maxi

VicTR: Video-conditioned Text Representations for Activity Recognition

2023

VicTR (ViT-B/16)

Expanding Language-Image Pretrained Models for General Video Recognition

2022

X-CLIP

microsoft/videox microsoft/VideoX

CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition

2021

CLASTER

Cross-modal Representation Learning for Zero-shot Action Recognition

2022

ResT

Alignment-Uniformity aware Representation Learning for Zero-shot Video Classification

2022

AURL

ShipuLoveMili/CVPR2022-AURL

Synthetic Sample Selection for Generalized Zero-Shot Learning

2023

SPOT

Elaborative Rehearsal for Zero-shot Action Recognition

2021

ER-ZSAR

DeLightCMU/ElaborativeRehearsal

Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications

2020

E2E

bbrattoli/ZeroShotVideoClassification

Towards Universal Representation for Unseen Action Recognition

2018

UR

Alternative Semantic Representations for Zero-Shot Human Action Recognition

2017

ASR

Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation

2016

MTE

Objects2action: Classifying and localizing actions without any video example

2015

O2A

Evaluation of Output Embeddings for Fine-Grained Image Classification

2014

SJE(word embedding)

mvp18/Popular-ZSL-Algorithms inars/developing_mc_for_zsl

Model	Paper	Top-1 Accuracy	Date
MSQNet	Actor-agnostic Multi-label Action Recognition wit…	69.43	2023-07-20
MOV (ViT-L/14)	Multimodal Open-Vocabulary Video Classification v…	64.70	2022-07-15
OTI(ViT-L/14)	Orthogonal Temporal Interpolation for Zero-Shot V…	64.00	2023-08-14
BIKE	Bidirectional Cross-Modal Knowledge Exploration f…	61.40	2022-12-31
MOV (ViT-B/16)	Multimodal Open-Vocabulary Video Classification v…	60.80	2022-07-15
IMP-MoE-L	Alternating Gradient Descent and Mixture-of-Exper…	59.10	2023-05-10
VideoCoCa	VideoCoCa: Video-Text Modeling with Zero-Shot Tra…	58.70	2022-12-09
Text4Vis	Revisiting Classifier: Transferring Vision-Langua…	58.40	2022-07-04
TC-CLIP	Leveraging Temporal Contextualization for Video A…	56.00	2024-04-15
OST	OST: Refining Text Knowledge with Optimal Spatio-…	55.90	2023-11-30
MAXI	MAtch, eXpand and Improve: Unsupervised Finetunin…	52.30	2023-03-15
VicTR (ViT-B/16)	VicTR: Video-conditioned Text Representations for…	51.00	2023-04-05
X-CLIP	Expanding Language-Image Pretrained Models for Ge…	44.60	2022-08-04
CLASTER	CLASTER: Clustering with Reinforcement Learning f…	43.20	2021-01-18
ResT	Cross-modal Representation Learning for Zero-shot…	41.10	2022-05-03
AURL	Alignment-Uniformity aware Representation Learnin…	39.00	2022-03-29
SPOT	Synthetic Sample Selection for Generalized Zero-S…	35.90	2023-04-06
ER-ZSAR	Elaborative Rehearsal for Zero-shot Action Recogn…	35.30	2021-08-05
E2E	Rethinking Zero-shot Video Classification: End-to…	32.70	2020-03-03
UR	Towards Universal Representation for Unseen Actio…	24.40	2018-03-22
ASR	Alternative Semantic Representations for Zero-Sho…	21.80	2017-06-28
MTE	Multi-Task Zero-Shot Action Recognition with Prio…	19.70	2016-11-26
O2A	Objects2action: Classifying and localizing action…	15.60	2015-10-23
SJE(word embedding)	Evaluation of Output Embeddings for Fine-Grained …	13.30	2014-09-30

HMDB51

Performance Over Time

Edit Benchmark Results

Edit Result

Top Performing Models

All Papers (24)

Actor-agnostic Multi-label Action Recognition with Multi-modal Query

Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models

Orthogonal Temporal Interpolation for Zero-Shot Video Recognition

Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models

Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models

Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception

VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners

Revisiting Classifier: Transferring Vision-Language Models for Video Recognition

Leveraging Temporal Contextualization for Video Action Recognition

OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition

MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge

VicTR: Video-conditioned Text Representations for Activity Recognition

Expanding Language-Image Pretrained Models for General Video Recognition

CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition

Cross-modal Representation Learning for Zero-shot Action Recognition

Alignment-Uniformity aware Representation Learning for Zero-shot Video Classification

Synthetic Sample Selection for Generalized Zero-Shot Learning

Elaborative Rehearsal for Zero-shot Action Recognition

Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications

Towards Universal Representation for Unseen Action Recognition

Alternative Semantic Representations for Zero-Shot Human Action Recognition

Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentation

Objects2action: Classifying and localizing actions without any video example

Evaluation of Output Embeddings for Fine-Grained Image Classification