Understanding Gaussian noise injections in neural networks - Camuto, A (2021)
Deep Learning (read neural networks) has emerged as one of the most exciting and powerful tools in the statistical machine learning toolbox. Neural networks over the past decade have defeated incumbent state of the art models in computer vision, natural language processing, and reinforcement learning; amongst a myriad of other fields. Though these achievements are impressive, neural networks remain surprisingly brittle, especially in the case of smaller network architectures. They can easily overfit, yielding poor performance on unseen data, and are remarkably susceptible to small perturbations and noise on their inputs. White noise on images has been shown to consistently alter the predictions of neural networks trained for classification. In a more targeted adversarial setting, where a hypothetical adversary has agency on the exact form of the input perturbation, these adverse effects are even more drastic.