LLMの圧縮・高速化
高速化
NPU
SpQR: ほぼ損失のない LLM 重み圧縮のためのスパース量子化表現より
https://note.com/daichi_mu/n/n32b6bd0ab28d
Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases
https://arxiv.org/abs/2301.12017
A Simple and Effective Pruning Approach for Large Language Models
https://arxiv.org/pdf/2306.11695.pdf
Google Colab で vLLM を試す
https://note.com/npaka/n/ne6fe8ae8aca0?sub_rt=share_h
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems
https://arxiv.org/abs/2312.15234
高速AI推論「Groq」を試す
https://zenn.dev/kun432/scraps/0f70f232a5f27b
深層モデルの高速化
https://speakerdeck.com/joisino/shen-ceng-moderunogao-su-hua