DeepSeek-Coder-V2-Lite-Instruct

Papers using or citing this model 1

• DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models 1.2K citations · 2024 · arXiv (Cornell University) · arXiv

  In the era of large language models, Mixture-of-Experts (MoE) is a promising architecture for managing computational costs when scaling up model parameters. However, conventional MoE architectures like GShard, which activate the top-$K$...