Bridging the Knowledge Gap via Transformer-Based Multi-Layer Correlation Learning

Bridging the Knowledge Gap via Transformer-Based Multi-Layer Correlation Learning

We tackle a multi-layer knowledge distillation problem between deep models with heterogeneous architectures. The main challenges of that are the mismatches of the feature maps in terms of the resolution or semantic levels. To resolve this, we propose a novel transformer-based multi-layer correlation...

Full description

Bibliographic Details
Main Authors: Hun-Beom Bak, Seung-Hwan Bae
Format: Article
Language:English
Published: IEEE 2024-01-01
Series:IEEE Access
Subjects:
Correlation learning
image classification
knowledge distillation
model compression
object detection
transformer-based learning
Online Access:https://ieeexplore.ieee.org/document/10497103/

Internet

https://ieeexplore.ieee.org/document/10497103/

Similar Items