Heterogeneous Student Knowledge Distillation From BERT Using a Lightweight Ensemble Framework

Heterogeneous Student Knowledge Distillation From BERT Using a Lightweight Ensemble Framework

Deep learning models have demonstrated their effectiveness in capturing complex relationships between input features and target outputs across many different application domains. These models, however, often come with considerable memory and computational demands, posing challenges for deployment on...

Full description

Bibliographic Details
Main Authors: Ching-Sheng Lin, Chung-Nan Tsai, Jung-Sing Jwo, Cheng-Hsiung Lee, Xin Wang
Format: Article
Language:English
Published: IEEE 2024-01-01
Series:IEEE Access
Subjects:
Knowledge distillation
ensemble methods
BERT
LSTM
CNN
contextual word representations
Online Access:https://ieeexplore.ieee.org/document/10458136/

Similar Items

Similar Items