llama-factory微调工具使用入门

一、定义

1. 环境配置
2. 案例： https://zhuanlan.zhihu.com/p/695287607
3. chatglm3 案例
4. 多卡训练deepspeed
5. llama factory 案例Qwen1.5
6. 报错

二、实现

1. 环境配置

git clone https://github.com/hiyouga/LLaMA-Factory.git
conda create -n llama_factory python=3.10
conda activate llama_factory
cd LLaMA-Factory
pip install -e '.[torch,metrics]'
如果发生冲突：    pip install --no-deps -e .  

同时对本库的基础安装做一下校验，输入以下命令获取训练相关的参数指导, 否则说明库还没有安装成功

llamafactory-cli train -h

模型下载与可用性校对

git clone https://www.modelscope.cn/LLM-Research/Meta-Llama-3-8B-Instruct.git

import transformers
import torch# 切换为你下载的模型文件目录, 这里的demo是Llama-3-8B-Instruct
# 如果是其他模型，比如qwen，chatglm，请使用其对应的官方demo
model_id = "/home/Meta-Llama-3-8B-Instruct"pipeline = transformers.pipeline("text-generation",model=model_id,model_kwargs={"torch_dtype": torch.bfloat16},device_map="auto",
)messages = [{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},{"role": "user", "content": "Who are you?"},
]prompt = pipeline.tokenizer.apply_chat_template(messages,tokenize=False,add_generation_prompt=True
)terminators = [pipeline.tokenizer.eos_token_id,pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]outputs = pipeline(prompt,max_new_tokens=256,eos_token_id=terminators,do_sample=True,temperature=0.6,top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])

2. 案例： https://zhuanlan.zhihu.com/p/695287607

2.1 数据准备
将该自定义数据集放到我们的系统中使用，则需要进行如下两步操作
a 复制该数据集到 data目录下
b 修改 data/dataset_info.json 新加内容完成注册, 该注册同时完成了3件事
b1 自定义数据集的名称为adgen_local，后续训练的时候就使用这个名称来找到该数据集
b2 指定了数据集具体文件位置
b3 定义了原数据集的输入输出和我们所需要的格式之间的映射关系

2. 微调：
下载模型
>> git clone https://www.modelscope.cn/LLM-Research/Meta-Llama-3-8B-Instruct.git
微调

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \--stage sft \--do_train \--model_name_or_path /home/Meta-Llama-3-8B-Instruct \--dataset alpaca_gpt4_zh,identity,adgen_local \--dataset_dir ./data \--template llama3 \--finetuning_type lora \--output_dir ./saves/LLaMA3-8B/lora/sft \--overwrite_cache \--overwrite_output_dir \--cutoff_len 1024 \--preprocessing_num_workers 16 \--per_device_train_batch_size 2 \--per_device_eval_batch_size 1 \--gradient_accumulation_steps 8 \--lr_scheduler_type cosine \--logging_steps 50 \--warmup_steps 20 \--save_steps 100 \--eval_steps 50 \--evaluation_strategy steps \--load_best_model_at_end \--learning_rate 5e-5 \--num_train_epochs 5.0 \--max_samples 1000 \--val_size 0.1 \--plot_loss \--fp16

或者：

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train ./examples/train_lora/llama3_lora_sft.yaml

3. 推理

CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat ./examples/inferce/llama3_lora_sft.yaml

或

CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat \--model_name_or_path /home/Meta-Llama-3-8B-Instruct \--adapter_name_or_path ./saves/LLaMA3-8B/lora/sft  \--template llama3 \--finetuning_type lora

4. 批量预测与训练效果评估

CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat ./examples/train/llama3_lora_predict.yaml

或

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train \--stage sft \--do_predict \--model_name_or_path /media/codingma/LLM/llama3/Meta-Llama-3-8B-Instruct \--adapter_name_or_path ./saves/LLaMA3-8B/lora/sft  \--dataset alpaca_gpt4_zh,identity,adgen_local \--dataset_dir ./data \--template llama3 \--finetuning_type lora \--output_dir ./saves/LLaMA3-8B/lora/predict \--overwrite_cache \--overwrite_output_dir \--cutoff_len 1024 \--preprocessing_num_workers 16 \--per_device_eval_batch_size 1 \--max_samples 20 \--predict_with_generate

5. LoRA模型合并导出

CUDA_VISIBLE_DEVICES=0 llamafactory-cli export \--model_name_or_path /home/Meta-Llama-3-8B-Instruct \--adapter_name_or_path ./saves/LLaMA3-8B/lora/sft  \--template llama3 \--finetuning_type lora \--export_dir megred-model-path \--export_size 2 \--export_device cpu \--export_legacy_format False

CUDA_VISIBLE_DEVICES=0 llamafactory-cli export ./examples/merge_lora/llama3_lora_sft.yaml

6. api 调用

CUDA_VISIBLE_DEVICES=0 API_PORT=8000 nohup llamafactory-cli api \--model_name_or_path /media/codingma/LLM/llama3/Meta-Llama-3-8B-Instruct \--adapter_name_or_path ./saves/LLaMA3-8B/lora/sft \--template llama3 \--finetuning_type lora

项目也支持了基于vllm 的推理后端，但是这里由于一些限制，需要提前将LoRA 模型进行merge，使用merge后的完整版模型目录或者训练前的模型原始目录都可。

CUDA_VISIBLE_DEVICES=0 API_PORT=8000 nohup llamafactory-cli api \--model_name_or_path megred-model-path \--template llama3 \--infer_backend vllm \--vllm_enforce_eager>output.log 2>&1 &

import os
from openai import OpenAI
from transformers.utils.versions import require_versionrequire_version("openai>=1.5.0", "To fix: pip install openai>=1.5.0")if __name__ == '__main__':# change to your custom portport = 8000client = OpenAI(api_key="0",base_url="http://localhost:{}/v1".format(os.environ.get("API_PORT", 8000)),)messages = []messages.append({"role": "user", "content": "hello, where is USA"})result = client.chat.completions.create(messages=messages, model="test")print(result.choices[0].message)

7. 测试

CUDA_VISIBLE_DEVICES=0 llamafactory-cli eval ./examples/train/llama3_lora_eval.yaml

或

CUDA_VISIBLE_DEVICES=0 llamafactory-cli eval \
--model_name_or_path /media/codingma/LLM/llama3/Meta-Llama-3-8B-Instruct \
--template llama3 \
--task mmlu \
--split validation \
--lang en \
--n_shot 5 \
--batch_size 1

3. chatglm3 案例
   见专题模块

4. 多卡训练deepspeed
   多卡看llama3_lora_sft_ds0.yaml

5. 报错

   1，RuntimeError: Failed to import trl.trainer.dpo_trainer because of the following error (look up to see its traceback):
   ‘FieldInfo’ object has no attribute ‘required’
   解决：换干净的环境，重新安装。