精度：Qwen2

0.5B -72B

SFT+DPO

1.Qwen 2 dense

GQA

Chunk attention

YARN

2.Qwen 2 MoE

MoE layers incorporate multiple FFNs

data expension

distribution improvement

高质量多任务数据

long-context training:4096-32768