PAPER 7
Bayesian Bits: Unifying Quantization and Pruning
Mart van Baalen
Christos Louizos
Markus Nagel
Rana Ali Amjad
Ying Wang
Tijmen Blankevoort
Max Welling (Qualcomm AI Research)
CVPR 2020
Summary
The Bayesian bits method works a bit like a sequence of gates. Given a dataset, it first determines if there is enough evidence to include a parameter in the computation. If it has decided it is a useful parameter it continues to determine if a second bit is needed to represent it. This process continues until a gate closes, implying that all less significant bits will be removed. Through this idea the method can adaptively determine the necessary precision for a computation and save energy in the process.
Citation
@inproceedings{baalen2020bayesianbits, title={Bayesian Bits: Unifying Quantization and Pruning}, author={Baalen, Mart van and Louizos, Christos and Nagel, Markus and Amjad, Rana Ali and Wang, Ying and Blankevoort, Tijmen and Welling, Max}, booktitle={Advances in Neural Information Processing Systems 33}, year={2020} }
Results
The experiments validate the proposed method on several benchmark datasets and show that it can learn pruned, mixed precision networks that provide a better trade-off between accuracy and efficiency than their static bit width equivalents.
Looking for more papers with code?
* Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.
