Hugging Peers

Lin, Haibin

☆ Save

3 papers 84 citations h-index 2 2019–2025

Robust LLM Training Infrastructure at ByteDance

Borui Wan, Gaohong Liu, Zuquan Song, Wang, Jun, Zhang, Yun, Guangming Sheng
+29 more show fewer
Shuguang Wang, Wei, Houmin, Wang, Chenyuan, Lou, Weiqiang, Yang, Xi, Zhang, Mofan, Kaihua Jiang, Ren, Cheng, Zhi, Xiaoyun, Menghan Yu, Nan, Zhe, Zheng, Zhuolin, Baoquan Zhong, Qinlong Wang, Yu, Huan, Chi, Jinxin, Zhang, Wang, Li, Yuhan, Du, Zixian, Zhao, Sida, Zhang, Yongqiang, Tang, Jingzhe, Zherui Liu, Chuan Wu, Yanghua Peng, Lin, Haibin, Wencong Xiao, Liu, Xin, Xiang, Liang

The training scale of large language models (LLMs) has reached tens of thousands of GPUs and is still continuously expanding, enabling faster learning of larger models. Accompanying the expansion...

35 citations 2025-09 Symposium on Operating Systems Principles

ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development

Borui Wan, Mingji Han, Yiyao Sheng, Yanghua Peng, Lin, Haibin, Zhang, Mofan
+6 more show fewer
Zhichao Lai, Yu, Menghan, Zhang, Junda, Song, Zuquan, Xin Liu, Chuan Wu

Checkpointing to preserve training states is crucial during the development of Large Foundation Models (LFMs), for training resumption upon various failures or changes in GPU resources and...

49 citations 2024-07 arXiv (Cornell University)

Dive into Deep Learning for Natural Language Processing

Lin, Haibin, Shi, Xingjian, Lausen, Leonard, Zhang, Aston, He, He, Zha, Sheng
+1 more show fewer
Smola, Alexander
2019-01 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): Tutorial Abstracts