Scalable continual deep learning with computational cost considerations - Prabhu, AP (2025)
Continual Learning (CL) is a emerging field that focuses on developing models capable of learning continuously from a incoming stream of data, as opposed to hundreds of passes over static, curated datasets. These models aim to retain previously acquired knowledge while seamlessly integrating new information, often with constraints like limited storage capacities. To advance this field, we first pinpoint the current research paradigm's limitations. We address these by (i) implementing more realistic constraints, such as optimizing learning within limited computational resources, and (ii) demonstrating the effectiveness of simple, straightforward algorithms. Additionally, we address flaws in existing metrics and enhance data collection methods to improve the efficiency and applicability of continual models in practical scenarios. We detail our contributions below: