Neuron-by-Neuron Quantization for Efficient Low-Bit QNN Training

Neuron-by-Neuron Quantization for Efficient Low-Bit QNN Training

Quantized neural networks (QNNs) are widely used to achieve computationally efficient solutions to recognition problems. Overall, eight-bit QNNs have almost the same accuracy as full-precision networks, but working several times faster. However, the networks with lower quantization levels demonstrat...

Full description

Bibliographic Details
Main Authors:	Artem Sher, Anton Trusov, Elena Limonova, Dmitry Nikolaev, Vladimir V. Arlazarov
Format:	Article
Language:	English
Published:	MDPI AG 2023-04-01
Series:	Mathematics
Subjects:	quantized neural network low-bit quantization layer-by-layer neuron-by-neuron training
Online Access:	https://www.mdpi.com/2227-7390/11/9/2112

Similar Items

4.6-Bit Quantization for Fast and Accurate Neural Network Inference on CPUs
by: Anton Trusov, et al.
Published: (2024-02-01)

Joint high-dimensional soft bit estimation and quantization using deep learning
by: Marius Arvinte, et al.
Published: (2022-06-01)

Latitude-Adaptive Integer Bit Allocation for Quantization of Omnidirectional Images
by: Qian Sima, et al.
Published: (2024-02-01)

Two Novel Non-Uniform Quantizers with Application in Post-Training Quantization
by: Zoran Perić, et al.
Published: (2022-09-01)

Nomograms for comparing the corrective abilities of binary and ternary neurons used in multicriteria testing of the hypothesis of small sample data independence
by: V.I. Volchikhin, et al.
Published: (2023-01-01)

Clipping-Based Post Training 8-Bit Quantization of Convolution Neural Networks for Object Detection
by: Leisheng Chen, et al.
Published: (2022-12-01)

Bit-Weight Adjustment for Bridging Uniform and Non-Uniform Quantization to Build Efficient Image Classifiers
by: Xichuan Zhou, et al.
Published: (2023-12-01)

IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays
by: Mohammed E. Fouda, et al.
Published: (2020-01-01)

Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer
by: Phuoc Pham, et al.
Published: (2021-01-01)

Super-Resolution Model Quantized in Multi-Precision
by: Jingyu Liu, et al.
Published: (2021-09-01)

Lightweight SAR: A Two-Bit Strategy
by: Shiqi Liu, et al.
Published: (2023-01-01)

Gaussian Multiple Access Channels with One-Bit Quantizer at the Receiver †,‡
by: Borzoo Rassouli, et al.
Published: (2018-09-01)

One-Bit Quantization and Distributed Detection with an Unknown Scale Parameter
by: Fei Gao, et al.
Published: (2015-08-01)

State Machine-Based Waveforms for Channels With 1-bit Quantization and Oversampling With Time-Instance Zero-Crossing Modulation
by: Diana Marcela V. Melo, et al.
Published: (2024-01-01)

Robust Quantized Consensus of Discrete Multi-Agent Systems in the Input-to-State Sense
by: Jiayu Chen, et al.
Published: (2019-01-01)

Kernel Quantization for Efficient Network Compression
by: Zhongzhi Yu, et al.
Published: (2022-01-01)

On the achievable rate of bandlimited continuous-time AWGN channels with 1-bit output quantization
by: Sandra Bender, et al.
Published: (2021-03-01)

A Hardware-Friendly Low-Bit Power-of-Two Quantization Method for CNNs and Its FPGA Implementation
by: Xuefu Sui, et al.
Published: (2022-09-01)

Downlink Secrecy Rate of One-Bit Massive MIMO System With Active Eavesdropping
by: M. A. Teeti
Published: (2020-01-01)

Linear CE and 1-bit Quantized Precoding With Optimized Dithering
by: Amodh Kant Saxena, et al.
Published: (2020-01-01)

Entropy-Constrained Scalar Quantization with a Lossy-Compressed Bit
by: Melanie F. Pradier, et al.
Published: (2016-12-01)

Continuous phase modulation with 1-bit quantization and oversampling using iterative detection and decoding
by: Rodrigo R. M. de Alencar, et al.
Published: (2020-11-01)

A Folding Approach for Multiple Antenna Arrays Using Low-Resolution ADCs
by: Jing Guo, et al.
Published: (2022-01-01)

The Quantization of Gravity: Quantization of the Hamilton Equations
by: Claus Gerhardt
Published: (2021-04-01)

1-bit quantization and oversampling at the receiver: Sequence-based communication
by: Lukas T. N. Landau, et al.
Published: (2018-04-01)

A comparative study of hole and electron inversion layer quantization in MOS structures
by: Chaudhry Amit, et al.
Published: (2010-01-01)

Comparison on Color Quantization Techniques
by: Alyaa taqi
Published: (2008-06-01)

O-2A: Outlier-Aware Compression for 8-bit Post-Training Quantization Model
by: Nguyen-Dong Ho, et al.
Published: (2023-01-01)

Geometric formulation of Berezin deformation quantization
by: R. Roknizadeh
Published: (2002-06-01)

An Ultra-Low Power Always-On Keyword Spotting Accelerator Using Quantized Convolutional Neural Network and Voltage-Domain Analog Switching Network-Based Approximate Computing
by: Bo Liu, et al.
Published: (2019-01-01)

A Generalized One-Bit Control System Using a <inline-formula> <tex-math notation="LaTeX">$\Delta\Sigma$ </tex-math></inline-formula>-Quantizer
by: Dhafer J. Almakhles, et al.
Published: (2019-01-01)

Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI
by: Mara Pistellato, et al.
Published: (2023-05-01)

AE-Qdrop: Towards Accurate and Efficient Low-Bit Post-Training Quantization for A Convolutional Neural Network
by: Jixing Li, et al.
Published: (2024-02-01)

Photonics-Assisted Millimeter-Wave Communication System Based on Low-Bit Gaussian Mixture Model Adaptive Vector Quantization
by: Yuancheng Cai, et al.
Published: (2022-01-01)

Forward Adaptive Dual-Mode Quantizer Based on the First-Degree Spline Approximation and Embedded G.711 Codec
by: Z. Peric, et al.
Published: (2019-12-01)

Unmanned ship heading tracking control strategy with state quantization and input quantization
by: Wei LI, et al.
Published: (2024-02-01)

Zero-Crossing Modulation for Wideband Systems Employing 1-Bit Quantization and Temporal Oversampling: Transceiver Design and Performance Evaluation
by: Peter Neuhaus, et al.
Published: (2021-01-01)

Audio Compression using a Modified Vector Quantization algorithm for Mastering Applications
by: Shajin Prince, et al.
Published: (2023-05-01)

CANET: Quantized Neural Network Inference With 8-bit Carry-Aware Accumulator
by: Jingxuan Yang, et al.
Published: (2024-01-01)

Design of a 2-Bit Neural Network Quantizer for Laplacian Source
by: Zoran Perić, et al.
Published: (2021-07-01)