EAGLE3-LLaMA3.1-Instruct-8B

Papers using or citing this model 3

• EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty 696 citations · 2024 · arXiv (Cornell University) · arXiv

  Autoregressive decoding makes the inference of Large Language Models (LLMs) time-consuming. In this paper, we reconsider speculative sampling and derive two key observations. Firstly, autoregression at the feature (second-to-top-layer)...

• EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees 431 citations · 2024 · arXiv (Cornell University) · arXiv

  Inference with modern Large Language Models (LLMs) is expensive and time-consuming, and speculative sampling has proven to be an effective solution. Most speculative sampling methods such as EAGLE use a static draft tree, implicitly...

• EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test 376 citations · 2025 · arXiv (Cornell University) · arXiv

  The sequential nature of modern LLMs makes them expensive and slow, and speculative sampling has proven to be an effective solution to this problem. Methods like EAGLE perform autoregression at the feature level, reusing top-layer...