Step-3.5-Flash-int4-AutoRound

Papers using or citing this model 1

• Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs 57 citations · 2023 · arXiv (Cornell University) · arXiv

  Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant challenges due to substantial memory and storage requirements. Weight-only quantization has emerged...