LLama-Factory 实现大模型LoRA-SFT微调指南

LLaMA Factory 是一款开源、低代码、一站式大语言与多模态模型微调框架，用于降低大模型的微调落地门槛。框架兼容 Qwen、LLaMA、ChatGLM、LLaVA 等上百款主流开源模型，支持增量预训练、SFT 监督微调、DPO、KTO、ORPO 等多种训练对齐方案，原生集成 LoRA、QLoRA 量化微调技术，可在消费级显卡完成轻量级模型微调。框架支持可视化网页端和命令行双操作模式，完整覆盖模型训练、评估、权重合并、推理部署全流程，适配科研实验、行业轻量化模型定制等场景。

• 项目地址：https://github.com/hiyouga/LlamaFactory/

本次实践基于 Ubuntu 22.04 系统、RTX 4090 24G 显卡、CUDA12.8 环境，全程使用国内镜像加速，解决外网下载慢、超时问题。

安装与配置

1、备份原有源文件，替换为阿里云镜像源，提升系统包下载速度。

root@localhost:~# cp /etc/apt/sources.list /etc/apt/sources.list.bak
root@localhost:~# bash -c 'cat > /etc/apt/sources.list <<EOF
deb http://mirrors.aliyun.com/ubuntu/ jammy main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ jammy-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ jammy-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ jammy-backports main restricted universe multiverse
EOF'
root@localhost:~# apt update

2、安装显卡检测工具，校验 NVIDIA 显卡识别状态，确保硬件环境正常。

root@localhost:~# apt install -y pciutils kmod

root@localhost:~# lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
25:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
25:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
41:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
41:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
61:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
61:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
81:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
81:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
a1:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
a1:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
c1:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
c1:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)
e1:00.0 VGA compatible controller: NVIDIA Corporation Device 2684 (rev a1)
e1:00.1 Audio device: NVIDIA Corporation Device 22ba (rev a1)

root@localhost:~# nvidia-smi 
Wed Sep  9 14:28:46 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.10              Driver Version: 570.86.10      CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:C1:00.0 Off |                  Off |
| 32%   31C    P8             22W /  405W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------------------------------------------------------+

root@localhost:~# ls /dev/nvidia*
/dev/nvidia-uvm  /dev/nvidia-uvm-tools  /dev/nvidia5  /dev/nvidiactl
/dev/nvidia-caps:
nvidia-cap1  nvidia-cap2

3、若系统未安装显卡驱动，执行以下命令安装官方推荐驱动，并禁用开源 nouveau 驱动。

禁用系统默认驱动

root@localhost:~# tee /etc/modprobe.d/blacklist-nouveau.conf <<'EOF'
blacklist nouveau
options nouveau modeset=0
EOF

添加显卡驱动PPA源，并安装 NVIDIA 驱动

root@localhost:~# apt install -y alsa-utils software-properties-common
root@localhost:~# add-apt-repository ppa:graphics-drivers/ppa
root@localhost:~# apt upgrade -y

# 查看推荐驱动
root@localhost:~# ubuntu-drivers devices

# 手动指定版本 recommended 标签则为推荐版本
root@localhost:~# sudo apt install -y nvidia-driver-570

# 重启验证显卡驱动
root@localhost:~# sudo reboot

4、搭建独立虚拟环境，避免依赖版本冲突。

root@localhost:~# apt install -y software-properties-common
root@localhost:~# add-apt-repository -y ppa:deadsnakes/ppa

root@localhost:~# apt install -y python3.12 python3.12-venv python3.12-dev git vim
root@localhost:~# python3.12 -m venv myvenv
root@localhost:~# source myvenv/bin/activate

5、根据显卡 CUDA 版本适配 PyTorch 版本，本次 CUDA12.8 对应安装 Torch2.8.0，使用清华、上交双镜像加速。

其他的版本对应预览表：

CUDA 编译包	Linux 最低驱动	Windows 最低驱动	PyTorch 版本
CUDA 13.0	≥ 580.30.02	≥ 581.06	2.9 / 2.12
CUDA 12.9	≥ 575.51.03	≥ 576.02	2.8
CUDA 12.8	≥ 570.26	≥ 570.65	2.7/ 2.8 / 2.9 / 2.10 / 2.11
CUDA 12.6	≥ 560.28.03	≥ 561.17	2.6 / 2.7 ~ 2.12
CUDA 12.4	≥ 550.54.14	≥ 551.23	2.5 / 2.6
CUDA 12.1	≥ 525.60.13	≥ 527.41	2.2 / 2.3 / 2.4
CUDA 11.8	≥ 450.80.02	≥ 452.39	2.0 ~ 2.7

依次执行命令安装，该过程较慢

root@localhost:~# pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 \
-i https://pypi.tuna.tsinghua.edu.cn/simple \
--extra-index-url https://mirror.sjtu.edu.cn/pytorch-wheels/cu128

root@localhost:~# pip install transformers==4.48.2 accelerate -i https://pypi.tuna.tsinghua.edu.cn/simple
root@localhost:~# pip list
Package                  Version
------------------------ ------------
accelerate               1.15.0
certifi                  2026.7.22
charset-normalizer       3.5.1
filelock                 3.32.6
fsspec                   2026.7.0
hf-xet                   1.6.0
huggingface_hub          0.36.2
idna                     3.19
Jinja2                   3.1.6
MarkupSafe               3.0.3
mpmath                   1.3.0
networkx                 3.6.1
numpy                    2.5.3
nvidia-cublas-cu12       12.8.4.1
nvidia-cuda-cupti-cu12   12.8.90
nvidia-cuda-nvrtc-cu12   12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12        9.10.2.21
nvidia-cufft-cu12        11.3.3.83
nvidia-cufile-cu12       1.13.1.3
nvidia-curand-cu12       10.3.9.90
nvidia-cusolver-cu12     11.7.3.90
nvidia-cusparse-cu12     12.5.8.93
nvidia-cusparselt-cu12   0.7.1
nvidia-nccl-cu12         2.27.3
nvidia-nvjitlink-cu12    12.8.93
nvidia-nvtx-cu12         12.8.90
packaging                26.3
pillow                   12.3.0
pip                      25.0.1
psutil                   7.2.2
PyYAML                   6.0.3
regex                    2026.9.3
requests                 2.34.2
safetensors              0.8.0
setuptools               84.0.0
sympy                    1.14.0
tokenizers               0.21.4
torch                    2.8.0+cu128
torchaudio               2.8.0+cu128
torchvision              0.23.0+cu128
tqdm                     4.70.0
transformers             4.48.2
triton                   3.4.0
typing_extensions        4.16.0
urllib3                  2.7.0

6、执行 Python 代码校验 PyTorch、CUDA、显卡识别状态。

import torch
print("torch版本:", torch.__version__)
print("torch编译用的CUDA版本:", torch.version.cuda)
print("CUDA是否可用:", torch.cuda.is_available())
print("GPU数量:", torch.cuda.device_count())
if torch.cuda.is_available():
    print("GPU名称:", torch.cuda.get_device_name(0))

# -----------------------------------------
torch版本: 2.8.0+cu128
torch编译用的CUDA版本: 12.8
CUDA是否可用: True
GPU数量: 1
GPU名称: NVIDIA GeForce RTX 4090

7、编译 Llama-Factory 镜像并安装。

root@localhost:~# git clone https://gitee.com/lyshark/LLaMA-Factory.git
root@localhost:~# cd LLaMA-Factory
root@localhost:~# pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -e .

root@localhost:~# pip list
Package                  Version
------------------------ ------------
accelerate               1.11.0
aiofiles                 24.1.0
aiohappyeyeballs         2.7.1
aiohttp                  3.14.3
aiosignal                1.4.0
annotated-doc            0.0.5
annotated-types          0.8.0
antlr4-python3-runtime   4.9.3
anyio                    4.15.1
attrs                    26.1.0
av                       16.0.0
brotli                   1.2.0
certifi                  2026.7.22
cffi                     2.1.1
charset-normalizer       3.5.1
click                    8.5.0
contourpy                1.3.3
cryptography             50.0.1
cycler                   0.12.1
datasets                 4.0.0
dill                     0.3.8
docstring_parser         0.18.0
einops                   0.8.2
fastapi                  0.141.1
ffmpy                    1.0.0
filelock                 3.32.6
fire                     0.7.1
fonttools                4.64.0
frozenlist               1.8.0
fsspec                   2025.3.0
gradio                   5.50.0
gradio_client            1.14.0
groovy                   0.1.2
h11                      0.16.0
hf_transfer              0.1.9
hf-xet                   1.6.0
httpcore                 1.0.9
httpx                    0.28.1
huggingface_hub          1.30.0
idna                     3.19
Jinja2                   3.1.6
kiwisolver               1.5.1
llamafactory             0.9.6.dev0
markdown-it-py           4.2.0
MarkupSafe               3.0.3
matplotlib               3.11.1
mdurl                    0.1.2
modelscope               1.40.0
modelscope-hub           0.4.1
mpmath                   1.3.0
multidict                6.8.0
multiprocess             0.70.16
networkx                 3.6.1
numpy                    2.5.3
nvidia-cublas-cu12       12.8.4.1
nvidia-cuda-cupti-cu12   12.8.90
nvidia-cuda-nvrtc-cu12   12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12        9.10.2.21
nvidia-cufft-cu12        11.3.3.83
nvidia-cufile-cu12       1.13.1.3
nvidia-curand-cu12       10.3.9.90
nvidia-cusolver-cu12     11.7.3.90
nvidia-cusparse-cu12     12.5.8.93
nvidia-cusparselt-cu12   0.7.1
nvidia-nccl-cu12         2.27.3
nvidia-nvjitlink-cu12    12.8.93
nvidia-nvtx-cu12         12.8.90
omegaconf                2.3.1
orjson                   3.12.0
packaging                26.3
pandas                   2.3.3
peft                     0.18.1
pillow                   11.3.0
pip                      25.0.1
propcache                0.5.2
protobuf                 7.36.1
psutil                   7.2.2
pyarrow                  25.0.1
pycparser                3.0
pydantic                 2.12.3
pydantic_core            2.41.4
pydub                    0.25.1
Pygments                 2.21.0
pyparsing                3.3.2
python-dateutil          2.9.0.post0
python-multipart         0.0.32
pytz                     2026.3.post1
PyYAML                   6.0.3
regex                    2026.9.3
requests                 2.34.2
rich                     15.0.0
ruff                     0.16.6
safehttpx                0.1.7
safetensors              0.8.0
scipy                    1.18.1
semantic-version         2.10.0
sentencepiece            0.2.2
setuptools               84.0.0
shellingham              1.5.4
shtab                    1.12.1
six                      1.17.0
sse-starlette            3.4.11
starlette                0.52.1
sympy                    1.14.0
termcolor                3.3.0
tiktoken                 0.14.0
tokenizers               0.22.2
tomlkit                  0.13.3
torch                    2.8.0+cu128
torchaudio               2.8.0+cu128
torchdata                0.11.0
torchvision              0.23.0+cu128
tqdm                     4.70.0
transformers             5.8.0
triton                   3.4.0
trl                      0.24.0
typer                    0.27.2
typing_extensions        4.16.0
typing-inspection        0.4.4
tyro                     0.8.14
tzdata                   2026.3
urllib3                  2.7.0
uvicorn                  0.52.4
websockets               15.0.1
xxhash                   4.0.1
yarl                     1.24.5

root@localhost:~# llamafactory-cli version
----------------------------------------------------------
| Welcome to LLaMA Factory, version 0.9.6.dev0           |
|                                                        |
| Project page: https://github.com/hiyouga/LLaMA-Factory |
----------------------------------------------------------

监督微调

本次微调采用问答数据集，原始数据为 jsonl 格式，需转换为 LlamaFactory 标准的 Alpaca 训练格式。

1、通过 ModelScope 下载 Qwen3.5-0.8B-Instruct 轻量化对话模型，适合消费级显卡微调。

root@localhost:~/# mkdir /data
root@localhost:~/# cd /data
root@localhost:~/# modelscope download --model icyfenix/Qwen3.5-0.8B-Instruct --local_dir /data/qwen3.5\-0.8B\-Instruct
root@localhost:~/# mv qwen3.5‑0.8B‑Instruct qwen3.5

2、准备训练材料，数据包含 question、answer 字段，文件名称叫做train.json，放入到LlamaFactory/data 目录下，以下结构是标准训练集结构。

[
  {
    "instruction": "你是一个助手",
    "input": "用户问题",
    "output": "回答"
  },
  {
    "instruction": "你是一个助手",
    "input": "第二个问题",
    "output": "对应的回答"
  }
]

3、继续使用《千问大模型完整RLHF全参数微调指南》文章中的医疗数据集，并将其做清洗处理。

下载数据集

root@localhost:~/qwen# wget https://modelscope.cn/datasets/krisfu/delicate_medical_r1_data/resolve/master/r1_data_example.jsonl
root@localhost:~/qwen# ls -lh

直接使用脚本将其转换为符合规范的格式，读取 r1_data_example.jsonl 文件，把每条的 question → input、answer → output 进行关联，并固定 instruction 为特定提示词，输出标准 json 数组格式。

import json

def convert_jsonl_to_json(jsonl_file_path, out_json_path):
    output_data = []
    with open(jsonl_file_path, "r", encoding="utf-8") as f:
        for line in f:
            line = line.strip()
            if not line:
                continue
            item = json.loads(line)
            new_sample = {
                "instruction": "你是一个医疗问答助手，请规范回答用户提问。",
                "input": item["question"],
                "output": item["answer"]
            }
            output_data.append(new_sample)

    with open(out_json_path, "w", encoding="utf-8") as fw:
        json.dump(output_data, fw, ensure_ascii=False, indent=4)
    print(f"转换完成，输出文件: {out_json_path}")

if __name__ == "__main__":
    jsonl_path = "/data/r1_data_example.jsonl"
    save_json_path = "/data/train.json"
    convert_jsonl_to_json(jsonl_path, save_json_path)

输出成train.json文件，并放入到/data目录下

root@localhost:~/# cd /data/
root@localhost:~/qwen# ls -lh
total 12M
drwxr-xr-x 2 root root 4.0K Sep  9 04:32 qwen3.5
-rw-r--r-- 1 root root 8.8M Apr 22  2025 r1_data_example.jsonl
-rw-r--r-- 1 root root 2.4M Sep  9 04:47 train.json

4、覆盖写入自定义数据集，修改 LlamaFactory/data/dataset_info.json 直接覆盖。

"mydata" 就是数据集名称

root@localhost:~/# cat <<EOF | tee data/dataset_info.json
{
  "mydata": {
    "file_name": "/data/train.json",
    "format": "alpaca"
  }
}
EOF

5、在 LlamaFactory 根目录新建 sft.yaml，配置 LoRA 微调核心参数。

root@localhost:~/# pip install tiktoken
root@localhost:~/# cat <<EOF | tee sft.yaml
model_name_or_path: /data/qwen3.5
dataset: mydata
template: qwen
finetuning_type: lora
lora_target: all
lora_rank: 8
lora_alpha: 16
lora_dropout: 0.05
stage: sft
do_train: true
num_train_epochs: 3
per_device_train_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 5e-5
lr_scheduler_type: cosine
warmup_steps: 50
weight_decay: 0.01
dataloader_num_workers: 0
output_dir: /data/qwen3.5_sft
save_steps: 100
logging_steps: 10
overwrite_output_dir: true
gradient_checkpointing: true
fp16: true
EOF

6、在 LLaMA-Factory 目录下执行命令启动训练。

root@localhost:/data# mkdir /data/qwen3.5_sft
root@localhost:/data# mkdir /data/qwen3.5_lora_merged

root@localhost:~/LLaMA-Factory# llamafactory-cli train sft.yaml
{'train_runtime': '21.48', 'train_samples_per_second': '2.793', 'train_steps_per_second': '0.419', 'train_loss': '2.202', 'epoch': '3'}
100%|███████████████████████████████████████████████████| 9/9 [00:21<00:00,  2.39s/it]

root@localhost:/data# ls -lh
total 49M
drwxr-xr-x 2 root root 4.0K Sep  9 07:40 checkpoint-9
drwxr-xr-x 2 root root 4.0K Sep  9 07:26 qwen3.5
drwxr-xr-x 2 root root   10 Sep  9 07:44 qwen3.5_lora_merged
drwxr-xr-x 3 root root 4.0K Sep  9 07:42 qwen3.5_sft
-rw-r--r-- 1 root root 8.8M Apr 27 00:54 r1_data_example.jsonl
-rw-r--r-- 1 root root  19K Sep  9 07:38 train.json

7、训练结束后通过执行 merge_lora.yaml 脚本实现合并模型权重。

root@localhost:~/LLaMA-Factory# cat <<EOF | tee merge_lora.yaml
model_name_or_path: /data/qwen3.5
adapter_name_or_path: /data/qwen3.5_sft
template: qwen
finetuning_type: lora
export_dir: /data/qwen3.5_lora_merged
export_legacy_format: false
EOF

root@localhost:~/LLaMA-Factory# llamafactory-cli export merge_lora.yaml
Loading weights: 100%|███████████████████████████████████████████| 473/473 [00:00<00:00, 6414.93it/s]

root@localhost:/data# ls -lh
total 49M
drwxr-xr-x 2 root root 4.0K Sep  9 07:26 qwen3.5
drwxr-xr-x 2 root root   10 Sep  9 07:44 qwen3.5_lora_merged
drwxr-xr-x 3 root root 4.0K Sep  9 07:42 qwen3.5_sft
-rw-r--r-- 1 root root 8.8M Apr 27 00:54 r1_data_example.jsonl
-rw-r--r-- 1 root root  19K Sep  9 07:38 train.json

root@localhost:/data/qwen3.5_lora_merged# ls -lh
total 1.7G
-rw-r--r-- 1 root root  464 Sep  9 07:46 Modelfile
-rw-r--r-- 1 root root 7.6K Sep  9 07:46 chat_template.jinja
-rw-r--r-- 1 root root 2.7K Sep  9 07:46 config.json
-rw-r--r-- 1 root root  115 Sep  9 07:46 generation_config.json
-rw------- 1 root root 1.6G Sep  9 07:46 model.safetensors
-rw-r--r-- 1 root root 1.2K Sep  9 07:46 processor_config.json
-rw-r--r-- 1 root root  20M Sep  9 07:46 tokenizer.json
-rw-r--r-- 1 root root 1.2K Sep  9 07:46 tokenizer_config.json

模型测试

编写推理配置文件，启动命令行交互式对话，测试医疗微调效果。

1、新建 infer_lora.yaml 放在 LLaMA-Factory 根目录

root@localhost:~/LLaMA-Factory# cat <<EOF | tee infer_lora.yaml
model_name_or_path: /data/qwen3.5_lora_merged
template: qwen
temperature: 0.4
top_p: 0.8
max_new_tokens: 512
EOF

2、启动终端对话

root@localhost:~/LLaMA-Factory# llamafactory-cli chat infer_lora.yaml

Loading weights: 100%|███████████████████████████████████| 473/473 [00:00<00:00, 1092.90it/s]
Welcome to the CLI application, use `clear` to remove the history, use `exit` to exit the application.

User: 你好
Assistant: <think>
用户打招呼，这是一个简单的问候。我应该用友好的方式回应，保持亲切和友好的语气。
</think>

你好！很高兴见到你。有什么我可以帮你的吗？