MiniMA-3B

Papers using or citing this model 1

• Towards the Law of Capacity Gap in Distilling Language Models 57 citations · 2023 · arXiv

  Language model (LM) distillation aims at distilling the knowledge in a large teacher LM to a small student one. As a critical issue facing LM distillation, a superior student often arises from a teacher of a relatively small scale...