Sheared LLaMA
https://huggingface.co/princeton-nlp/Sheared-LLaMA-1.3B Sheared-LLaMA-1.3B
https://huggingface.co/princeton-nlp/Sheared-LLaMA-2.7B Sheared-LLaMA-2.7B
https://xiamengzhou.github.io/sheared-llama/
https://github.com/princeton-nlp/LLM-Shearingprinceton-nlp/LLM-Shearing
https://arxiv.org/abs/2310.06694Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
構造化刈り込み
Llama-2-7bモデルを1.3Bと2.7Bのパラメータに刈り込みした
https://gyazo.com/a60df91d2fe8987c3232be2a051b263e