Attention and feature transfer based knowledge distillation
Abstract Existing knowledge distillation (KD) methods are mainly based on features, logic, or attention, where features and logic represent the results of reasoning at different stages of a convolutional neural network, and attention maps symbolize the reasoning process. Because of the continuity of...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-10-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-43986-y |