Hugging Peers

Hunyuan-Large-eval-result

☆ Save
by easyllm
2 downloads 0 likes on ModelScope updated

Model card

模型信息

With the rapid development of artificial intelligence technology, large language models (LLMs) have made significant progress in fields such as natural language processing, computer vision, and scientific tasks. However, as the scale of these models increases, optimizing resource consumption while maintaining high performance has become a key challenge. To address this challenge, we have explored Mixture of Experts (MoE) models. The currently unveiled Hunyuan-Large (Hunyuan-MoE-A52B) model is the largest open-source Transformer-based MoE model in the industry, featuring a total of 389 billion parameters and 52 billion active parameters. This is currently the largest open-source Transformer-based MoE model in the industry, featuring a total of 389 billion parameters and 52 billion active parameters.

By open-sourcing the Hunyuan-Large model and revealing related technical details, we hope to inspire more researchers with innovative ideas and collectively advance the progress and application of AI technology. We welcome you to join our open-source community to explore and optimize future AI models together!

Introduction to Model Technical Advantages

Model

  • High-Quality Synthetic Data: By enhancing training with synthetic data, Hunyuan-Large can learn richer representations, handle long-context inputs, and generalize better to unseen data.

  • KV Cache Compression: Utilizes Grouped Query Attention (GQA) and Cross-Layer Attention (CLA) strategies to significantly reduce memory usage and computational overhead of KV caches, improving inference throughput.

  • Expert-Specific Learning Rate Scaling: Sets different learning rates for different experts to ensure each sub-model effectively learns from the data and contributes to overall performance.

  • Long-Context Processing Capability: The pre-trained model supports text sequences up to 256K, and the Instruct model supports up to 128K, significantly enhancing the ability to handle long-context tasks.

  • Extensive Benchmarking: Conducts extensive experiments across various languages and tasks to validate the practical effectiveness and safety of Hunyuan-Large.

模型下载

点击下载

评测结果

开源模型对比

排名 大模型 机构 size 总分 医疗 教育 金融 法律 行政公务 心理健康 推理与数学计算 语言与指令遵从
1 DeepSeek-R1 深度求索 671B 80.8 81.9 89.2 82.9 73.8 83.0 61.5 89.2 84.8
2 hunyuan-large 腾讯 389B 80.2 88.6 85.5 84.6 82.4 70.4 73.2 76.8 80.1
3 deepseek-chat-v3-0324(new) 深度求索 671B 77.7 78.0 87.0 76.1 61.7 81.2 64.6 87.6 85.4
4 qwq-32b 阿里巴巴 32B 77.5 76.5 87.2 78.3 60.9 82.2 63.0 87.2 85.2
5 GLM-Z1-32B-0414(new) 智谱AI 32B 75.6 72.6 81.9 74.8 62.2 80.0 63.3 88.5 81.3
6 DeepSeek-R1-Distill-Qwen-32B 深度求索 32B 71.8 69.4 85.1 73.8 51.8 76.0 53.8 83.6 81.4
7 GLM-4-32B-0414(new) 智谱AI 32B 71.8 71.7 76.5 72.3 54.5 76.0 60.9 81.2 81.3
8 Llama-4-Maverick-17B-128E-Instruct-FP8(new) Meta 400B 71.3 75.8 83.0 71.1 48.1 69.0 59.0 82.7 81.6
9 qwen2.5-72b-instruct 阿里巴巴 72B 71.2 74.1 80.0 75.6 51.6 67.0 59.5 79.7 81.7
10 MiniMax-Text-01 minimax 456B 69.7 70.6 79.8 70.0 50.7 71.0 57.8 76.0 81.3
11 qwen2.5-32b-instruct 阿里巴巴 32B 69.1 71.6 82.0 71.8 50.8 64.5 57.8 73.5 80.7
12 DeepSeek-R1-Distill-Qwen-14B 深度求索 14B 66.5 62.8 79.3 67.5 40.2 66.5 55.6 81.0 78.7
13 qwen2.5-14b-instruct 阿里巴巴 14B 65.9 67.7 78.6 68.0 45.4 61.0 56.1 70.4 79.9
14 GLM-Z1-9B-0414(new) 智谱AI 9B 63.2 58.0 66.9 63.8 31.5 75.5 48.8 84.8 76.2
15 qwen2.5-7b-instruct 阿里巴巴 7B 62.5 64.5 74.8 66.5 41.8 53.0 56.0 67.6 76.1
16 Llama-4-Scout-17B-16E-Instruct(new) Meta 109B 62.4 65.4 78.0 62.1 31.8 55.5 54.0 76.0 76.2
17 DeepSeek-R1-Distill-Llama-70B 深度求索 70B 62.2 54.1 72.9 60.6 34.7 70.0 46.2 81.8 77.0
18 internlm2_5-20b-chat 上海人工智能实验室 20B 61.6 65.4 73.9 62.9 44.7 53.5 52.7 62.4 77.2
19 Meta-Llama-3.1-405B-Instruct Meta 405B 61.0 66.1 64.1 59.3 34.7 59.0 53.9 73.2 77.9
20 internlm2_5-7b-chat 上海人工智能实验室 7B 60.6 64.8 66.1 64.1 43.1 55.5 51.0 64.3 76.3
21 GLM-4-9B-0414(new) 智谱AI 9B 60.3 59.3 72.0 62.1 40.8 53.5 49.2 68.3 76.8
22 Llama-3.3-70B-Instruct Meta 70B 59.0 59.2 63.8 56.9 29.9 60.5 49.6 74.2 78.0
23 Mistral-Small-3.1-24B-Instruct-2503(new) Mistral 24B 58.7 53.1 68.0 57.6 33.3 57.0 47.1 76.3 77.1
24 Llama-3.3-70B-Instruct-fp8 Meta 70B 58.4 58.7 63.0 56.0 29.2 59.0 48.5 74.7 78.1
25 GLM-Z1-Rumination-32B-0414(new) 智谱AI 32B 57.5 47.9 69.6 54.3 38.9 56.7 44.1 78.3 70.2
26 Hermes-3-Llama-3.1-405B NousResearch 405B 57.4 55.6 62.3 59.2 29.8 52.5 48.9 72.7 78.0
27 glm-4-9b-chat 智谱AI 9B 56.8 56.0 73.4 57.3 36.5 52.0 47.1 60.1 71.7
28 gemma-3-27b-it Google 27B 56.4 48.2 64.3 56.1 21.3 66.5 44.5 77.4 73.1
29 phi-4 微软 14B 55.3 43.4 59.7 56.0 24.9 64.5 43.2 77.3 73.7
30 Mistral-Small-24B-Instruct-2501 Mistral 24B 53.2 45.8 60.5 53.1 26.3 53.5 40.2 71.4 74.4
31 gemma-3-12b-it Google 12B 52.2 43.6 58.2 46.6 20.4 59.0 41.1 77.3 71.6
32 qwen2.5-3b-instruct 阿里巴巴 3B 50.6 49.8 62.4 53.8 26.9 42.0 43.8 59.1 67.6
33 DeepSeek-R1-Distill-Qwen-7B 深度求索 7B 45.4 28.8 52.8 45.8 19.2 54.0 30.4 70.9 61.2
34 DeepSeek-R1-Distill-Llama-8B 深度求索 8B 45.2 29.7 51.4 42.6 20.3 52.0 31.9 68.5 64.8
35 Llama-3.1-8B-Instruct Meta 8B 44.0 39.9 44.3 40.9 21.3 43.0 37.2 59.9 65.8
36 Meta-Llama-3.1-8B-Instruct-fp8 Meta 8B 42.9 39.2 43.0 40.8 19.5 37.5 34.0 63.1 65.7
37 qwen2.5-1.5b-instruct 阿里巴巴 2B 42.3 43.8 56.0 48.6 25.6 35.0 39.6 38.5 51.5
38 gemma-3-4b-it Google 4B 41.1 31.8 46.6 39.7 16.6 39.5 29.2 67.6 58.0
39 Llama-3.2-3B-Instruct Meta 3B 38.5 34.6 37.5 35.0 17.1 34.0 29.6 57.6 62.7
40 DeepSeek-R1-Distill-Qwen-1.5B 深度求索 2B 35.5 23.8 42.8 34.5 15.4 34.5 23.9 61.8 47.1
41 Llama-3.2-1B-Instruct Meta 1B 30.1 25.4 28.3 28.8 14.6 29.0 21.1 41.6 51.9
42 qwen2.5-0.5b-instruct 阿里巴巴 1B 30.0 29.2 36.8 33.9 18.4 22.5 24.5 35.2 39.2
43 gemma-3-1b-it Google 1B 28.6 23.6 31.9 28.7 15.4 29.0 20.6 31.5 48.2

精益评测：更细分的维度

  • 医疗
    • 医师
    • 护理
    • 药师
    • 医技
    • 医学基础知识
    • 医学考研
  • 教育
    • 小学学科
    • 初中学科
    • 中考
    • 高中学科
    • 高考
    • 高等教育
    • 考研
    • 教师资格
  • 金融排行榜
    • 财务
    • 银行
    • 保险
    • 证券
    • 其他金融资格考试
    • 金融基础知识
    • 金融应用
  • 法律排行榜
    • 律师资格考试
  • 行政公务排行榜
    • 公务员考试
  • 心理健康排行榜
  • 推理与数学计算排行榜
    • 演绎推理
    • 常识推理
    • 符号推理BBH
    • 算术能力
    • 表格问答
    • 高中奥数
    • 初中奥数
    • 小学奥数
  • 语言与指令遵从排行榜
    • 成语理解
    • 情感分析
    • 文本蕴含
    • 分类能力
    • 信息抽取
    • 阅读理解
    • C3中文阅读理解
    • 代词理解CLUEWSC
    • 诗词匹配CCPM
    • 中文指令遵从

以上细分能力评测结果，详见github