|
ScaleKD (T:BEiT-L S:ViT-B/14)
|
ScaleKD: Strong Vision Transformers Could Be Exce…
|
86.43
|
2024-11-11
|
|
|
ScaleKD (T:Swin-L S:ViT-B/16)
|
ScaleKD: Strong Vision Transformers Could Be Exce…
|
85.53
|
2024-11-11
|
|
|
ScaleKD (T:Swin-L S:ViT-S/16)
|
ScaleKD: Strong Vision Transformers Could Be Exce…
|
83.93
|
2024-11-11
|
|
|
ScaleKD (T:Swin-L S:Swin-T)
|
ScaleKD: Strong Vision Transformers Could Be Exce…
|
83.80
|
2024-11-11
|
|
|
KD++(T: regnety-16GF S:ViT-B)
|
Improving Knowledge Distillation via Regularizing…
|
83.60
|
2023-05-26
|
|
|
VkD (T:RegNety 160 S:DeiT-S)
|
$V_kD:$ Improving Knowledge Distillation using Or…
|
82.90
|
2024-03-10
|
|
|
SpectralKD (T:Swin-S S:Swin-T)
|
SpectralKD: A Unified Framework for Interpreting …
|
82.70
|
2024-12-26
|
|
|
ScaleKD (T:Swin-L S:ResNet-50)
|
ScaleKD: Strong Vision Transformers Could Be Exce…
|
82.55
|
2024-11-11
|
|
|
DiffKD (T:Swin-L S: Swin-T)
|
Knowledge Diffusion for Distillation
|
82.50
|
2023-05-25
|
|
|
DIST (T: Swin-L S: Swin-T)
|
Knowledge Distillation from A Stronger Teacher
|
82.30
|
2022-05-21
|
|
|
SpectralKD (T:Cait-S24 S:DeiT-S)
|
SpectralKD: A Unified Framework for Interpreting …
|
82.20
|
2024-12-26
|
|
|
SRD (T:RegNety 160 S:DeiT-S)
|
Understanding the Role of the Projector in Knowle…
|
82.10
|
2023-03-20
|
|
|
OFA (T: ViT-B S: ResNet-50)
|
One-for-All: Bridge the Gap Between Heterogeneous…
|
81.33
|
2023-10-30
|
|
|
DiffKD (T:Swin-L S: ResNet-50)
|
Knowledge Diffusion for Distillation
|
80.50
|
2023-05-25
|
|
|
VkD (T:RegNety 160 S:DeiT-Ti)
|
$V_kD:$ Improving Knowledge Distillation using Or…
|
79.20
|
2024-03-10
|
|
|
KD++(T:resnet-152 S:resnet-101)
|
Improving Knowledge Distillation via Regularizing…
|
79.15
|
2023-05-26
|
|
|
ADLIK-MO-P25(T:SeNet154, ResNet152b S:ResNet-50-prune25%)
|
Ensemble Knowledge Distillation for Learning Impr…
|
78.79
|
2019-09-17
|
|
|
ADLIK-MO-P375(T:SeNet154, ResNet152b S:ResNet-50-prune37.5)
|
Ensemble Knowledge Distillation for Learning Impr…
|
78.07
|
2019-09-17
|
|
|
KD++(T:resnet-152 S:resnet-50)
|
Improving Knowledge Distillation via Regularizing…
|
77.48
|
2023-05-26
|
|
|
SpectralKD (T:Cait-S24 S:DeiT-T)
|
SpectralKD: A Unified Framework for Interpreting …
|
77.40
|
2024-12-26
|
|
|
SRD (T:RegNety 160 S:DeIT-Ti)
|
Understanding the Role of the Projector in Knowle…
|
77.20
|
2023-03-20
|
|
|
ADLIK-MO(T: ResNet101 S: ResNet50)
|
Distilling the Knowledge in a Neural Network
|
77.14
|
2015-03-09
|
|
|
WTTM (T: DeiT III-Small S:DeiT-Tiny)
|
Knowledge Distillation Based on Transformed Teach…
|
77.03
|
2024-02-17
|
|
|
ADLIK-MO-P50(T:SeNet154, ResNet152b S:ResNet-50-half)
|
Ensemble Knowledge Distillation for Learning Impr…
|
76.38
|
2019-09-17
|
|
|
KD++(T:resnet152 S:resnet34)
|
Improving Knowledge Distillation via Regularizing…
|
75.53
|
2023-05-26
|
|
|
WTTM (T:resnet50, S:mobilenet-v1)
|
Knowledge Distillation Based on Transformed Teach…
|
73.09
|
2024-02-17
|
|
|
ReviewKD++(T:resnet50, S:mobilenet-v1)
|
Improving Knowledge Distillation via Regularizing…
|
72.96
|
2023-05-26
|
|
|
KD++(T:resnet-152 S:resnet18)
|
Improving Knowledge Distillation via Regularizing…
|
72.54
|
2023-05-26
|
|
|
KD++(T:renset101 S:resnet18)
|
Improving Knowledge Distillation via Regularizing…
|
72.54
|
2023-05-26
|
|
|
KD++(T:resnet50 S:resnet18)
|
Improving Knowledge Distillation via Regularizing…
|
72.53
|
2023-05-26
|
|
|
HSAKD (T: ResNet-34 S:ResNet-18)
|
Hierarchical Self-supervised Augmented Knowledge …
|
72.39
|
2021-07-29
|
|
|
WTTM (T: ResNet-34 S:ResNet-18)
|
Knowledge Distillation Based on Transformed Teach…
|
72.19
|
2024-02-17
|
|
|
DIST (T: ResNet-34 S:ResNet-18)
|
Knowledge Distillation from A Stronger Teacher
|
72.07
|
2022-05-21
|
|
|
KD++(T: ResNet-34 S:ResNet-18)
|
Improving Knowledge Distillation via Regularizing…
|
72.07
|
2023-05-26
|
|
|
WSL (T: ResNet-34 S:ResNet-18)
|
Rethinking Soft Labels for Knowledge Distillation…
|
72.04
|
2021-02-01
|
|
|
CRCD (T: ResNet-34 S:ResNet-18)
|
Complementary Relation Contrastive Distillation
|
71.96
|
2021-03-29
|
|
|
SRD (T: ResNet-34 S:ResNet-18)
|
Understanding the Role of the Projector in Knowle…
|
71.87
|
2023-03-20
|
|
|
KD++(T:ViT-B, S:resnet18)
|
Improving Knowledge Distillation via Regularizing…
|
71.84
|
2023-05-26
|
|
|
LSHFM (T: ResNet-34 S:ResNet-18)
|
Distilling Knowledge by Mimicking Features
|
71.72
|
2020-11-03
|
|
|
ITRD (T: ResNet-34 S:ResNet-18)
|
Information Theoretic Representation Distillation
|
71.68
|
2021-12-01
|
|
|
SSKD (T: ResNet-34 S:ResNet-18)
|
Knowledge Distillation Meets Self-Supervision
|
71.62
|
2020-06-12
|
|
|
Knowledge Review (T: ResNet-34 S:ResNet-18)
|
Distilling Knowledge via Knowledge Review
|
71.61
|
2021-04-19
|
|
|
Adaptive (T:ResNet-50 S:ResNet-18)
|
Adaptive Distillation: Aggregating Knowledge from…
|
71.61
|
2021-10-19
|
|
|
KD++(T: ViT-S, S:resnet18)
|
Improving Knowledge Distillation via Regularizing…
|
71.46
|
2023-05-26
|
|
|
AFD (T: ResNet-34 S:ResNet-18)
|
Show, Attend and Distill:Knowledge Distillation v…
|
71.38
|
2021-02-05
|
|
|
CRD (T: ResNet-34 S:ResNet-18)
|
Contrastive Representation Distillation
|
71.38
|
2019-10-23
|
|
|
Overhual (T: ResNet-34 S:ResNet-18)
|
A Comprehensive Overhaul of Feature Distillation
|
70.81
|
2019-04-03
|
|
|
AT (T: ResNet-34 S:ResNet-18)
|
Paying More Attention to Attention: Improving the…
|
70.70
|
2016-12-12
|
|
|
KD (T: ResNet-34 S:ResNet-18)
|
Distilling the Knowledge in a Neural Network
|
70.66
|
2015-03-09
|
|
|
AT (T: ResNet-34 S:ResNet-18)
|
Paying More Attention to Attention: Improving the…
|
|
2016-12-12
|
|