HF_Trainer. HF_Trainer is a Huggingface-based framework for training LLMs, supporting pre-training, fine-tuning, reward modeling, and DPO training.