RankMe_FinInd

基于Qwen2.5-7B-Instruct微调的金融行业大模型

训练数据使用IndustryInstruction_Finance-Economics_RankMe_Filter_Regen · 数据集

文件目录结构

.
├── adapter_config.json // 模型文件
├── adapter_model.safetensors // 模型文件
├── inference.py // 推理代码
├── RankMe_FinInd_12081411.jsonl // 推理结果
├── RankMe_FinInd_report.pdf // 报告
├── README.md
├── run_log.jsonl // 推理日志
└── train_config.yaml // 训练配置
└── trainer_log.jsonl // 训练日志

训练过程

超参数

learning_rate: 1e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 8
total_train_batch_size: 32
total_eval_batch_size: 4
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 2

结果

Training Loss	Epoch	Step	Validation Loss
0.8448	0.1021	36	0.8503
0.7601	0.2043	72	0.8113
0.7435	0.3064	108	0.7970
0.7562	0.4085	144	0.7891
0.7248	0.5106	180	0.7842
0.7566	0.6128	216	0.7815
0.7442	0.7149	252	0.7784
0.7513	0.8170	288	0.7759
0.7469	0.9191	324	0.7747
0.7998	1.0213	360	0.7730
0.7227	1.1234	396	0.7717
0.7466	1.2255	432	0.7709
0.7677	1.3277	468	0.7701
0.7146	1.4298	504	0.7696
0.7407	1.5319	540	0.7691
0.7524	1.6340	576	0.7689
0.7149	1.7362	612	0.7687
0.7456	1.8383	648	0.7687
0.7433	1.9404	684	0.7688

Framework versions

PEFT 0.12.0
Transformers 4.46.1
Pytorch 2.3.1+cu121
Datasets 3.1.0
Tokenizers 0.20.2

RankMe_FinInd

Model card

RankMe_FinInd

文件目录结构

训练过程

超参数

结果

Framework versions