[CVPR 2025] APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers [Github] [arXiv]

Zhuguanyu Wu, Jiayi Zhang, Jiaxin Chen^✉, Jinyang Guo, Di Huang, Yunhong Wang^✉

In this paper, we propose an importance estimation criterion based on the average perturbation Hessen (APH) for block reconstruction. Compared with previous methods, APH uses fewer assumptions and approximations, and can make the optimization process more stable. We further propose MLP-Reconstruction (MR), which significantly reduces the activation range during block reconstruction, and replaces the GELU activation function with ReLU to further improve the accuracy.

[CVPR 2025 Highlight] FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation [Github]

Zhuguanyu Wu*, Shihe Wang*, Jiayi Zhang, Jiaxin Chen^✉, Yunhong Wang^✉

In this paper, we conduct a systematic analysis of the widely adopted Hessian-based quantization loss and identify its inherent limitations and inaccuracies. In light of the limitations, we propose a novel quantization loss formulation based on the Fisher Information Matrix (FIM). First, we theoretically establish the relationship between FIM and KL divergence. Leveraging this relationship, we develop four approximation schemes for FIM, where the diagonal plus low-rank (DPLR) approximation of FIM achieves superior performance across various ViT architectures and tasks.

[ECCV 2024] AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer [Github] [arXiv] [bibtex]

Zhuguanyu Wu, Jiaxin Chen^✉, Hanwen Zhong, Di Huang, Yunhong Wang

In this paper, we introduce a hardware-friendly base-adaptive logarithmic quantizer (AdaLog), which adaptively adjusts the logarithmic base to align with the power-law distribution of activations and simultaneously allows hardware-friendly quantization and de-quantization through lookup table and bit-shift operations. By incorporating bias reparameterization techniques, we successfully apply the AdaLog quantizer to post-GELU activation quantization. Additionally, we propose a Fast Progressive Combining Search (FPCS) strategy to accurately determine quantization parameters.