从零实现大模型-GPT2任务微调原创

"Is the following text 'spam'? Answer with 'yes' or 'no':"
    " 'You are a winner you have been specially"
    " selected to receive $1000 cash or a $2000 award.'"1.
2.
3.

以下是GPT4-o给出的答案，不仅准确识别出了垃圾邮件，还遵循了人类指令输出了“yes”。

从零实现大模型-GPT2任务微调-AI.x社区

如果只使用预训练模型，没有经过指令微调，前面我们也测试过，模型虽然有输出，但输出只是简单的拷贝输入。

输入：
Is the following text 'spam'? Answer with 'yes' or 'no': 'You are a winner you have been specially selected to receive $1000 cash or a $2000 award.'
输出：
The following text 'spam'? Answer with 'yes' or 'no': 'You are a winner1.
2.
3.
4.

2.任务微调

本文完整代码如下，建议结合代码阅读文本。

https://github.com/AIDajiangtang/LLM-from-scratch/blob/main/GPT2_fine-tune_spam_classifier_from_scratch.ipynb1.

在通过Embedding实现文本分类时，Embedding是通过调用openAI的API得到的，其实，这个Embedding也可以看作是GPT模型输出隐状态的一部分。

除了用Embedding计算余弦相似度来实现文分类外，还可以基于隐状态实现一个分类模型，也就是在模型的输出端加一个分类头。

从零实现大模型-GPT2任务微调-AI.x社区

准备数据

训练数据来自公开的垃圾邮件数据集，包括文本和标签两列，标签列中spam代表是垃圾邮件。

从零实现大模型-GPT2任务微调-AI.x社区

因为数据集中垃圾邮件数量少于正常邮件，所以要平衡正负样本数量。

# Examine class distributions
print(df["Label"].value_counts())1.
2.

Label
ham     4825
spam     747
Name: count, dtype: int641.
2.
3.
4.

def create_balanced_dataset(df):
    num_spam = df[df["Label"] == "spam"].shape[0]
    ham_subset = df[df["Label"] == "ham"].sample(num_spam, random_state=123)
    balanced_df = pd.concat([ham_subset, df[df["Label"] == "spam"]])
    return balanced_df


balanced_df = create_balanced_dataset(df)
print(balanced_df["Label"].value_counts())1.
2.
3.
4.
5.
6.
7.
8.
9.

Label
ham     747
spam    747
Name: count, dtype: int641.
2.
3.
4.

接下来划分训练集，验证机和测试集。

接下来是tokenization，padding或者截断到最大长度。

最后构造Dataloader。

加载预训练模型

加载gpt2预训练模型。

CHOOSE_MODEL = "gpt2-small (124M)"
INPUT_PROMPT = "Every effort moves"
BASE_CONFIG = {
    "vocab_size": 50257,     # Vocabulary size
    "context_length": 1024,  # Context length
    "drop_rate": 0.0,        # Dropout rate
    "qkv_bias": True         # Query-key-value bias
}
model_configs = {
    "gpt2-small (124M)": {"emb_dim": 768, "n_layers": 12, "n_heads": 12},
    "gpt2-medium (355M)": {"emb_dim": 1024, "n_layers": 24, "n_heads": 16},
    "gpt2-large (774M)": {"emb_dim": 1280, "n_layers": 36, "n_heads": 20},
    "gpt2-xl (1558M)": {"emb_dim": 1600, "n_layers": 48, "n_heads": 25},
}
BASE_CONFIG.update(model_configs[CHOOSE_MODEL])


assert train_dataset.max_length <= BASE_CONFIG["context_length"], (
    f"Dataset length {train_dataset.max_length} exceeds model's context "
    f"length {BASE_CONFIG['context_length']}. Reinitialize data sets with "
    f"`max_length={BASE_CONFIG['context_length']}`"
)1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.

from gpt_download import download_and_load_gpt2


model_size = CHOOSE_MODEL.split(" ")[-1].lstrip("(").rstrip(")")
settings, params = download_and_load_gpt2(model_size=model_size, models_dir="gpt2")


model = GPTModel(BASE_CONFIG)
load_weights_into_gpt(model, params)
model.eval()1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

再微调过程中，可以冻结大部分预训练模型参数。

# Freeze all model layers first
for param in model.parameters():
  param.requires_grad = False1.
2.
3.

添加分类头

在预训练模型的输出端加一个二分类分类头。

# Add a classification head
torch.manual_seed(123)
num_classes = 2
model.out_head = torch.nn.Linear(
    in_features=BASE_CONFIG["emb_dim"], # 768
    out_features=num_classes,           # 2 (spam or not spam)
)1.
2.
3.
4.
5.
6.
7.

构造损失函数

因为是二分类，所以构造一个交叉熵损失函数。

def calculate_loss_batch(input_batch, target_batch, model, device):
  input_batch, target_batch = input_batch.to(device), target_batch.to(device)
  logits = model(input_batch)[:, -1, :] # Grab logits of last output token only!
  loss = torch.nn.functional.cross_entropy(logits, target_batch)
  return loss1.
2.
3.
4.
5.

然后开始训练，训练过程与指令微调过程基本一致。

import time


start_time = time.time()
torch.manual_seed(123)


# Create optimizer
optimizer = torch.optim.AdamW(model.parameters(), lr=5e-5, weight_decay=0.1)


# Set training epochs
num_epochs = 5


# Train the model
train_losses, val_losses, train_accs, val_accs, examples_seen = train_classifier(
    model=model,
    train_loader=train_loader,
    val_loader=val_loader,
    optimizer=optimizer,
    device=device,
    num_epochs=num_epochs,
    eval_freq=50,
    eval_iter=5,
    tokenizer=tokenizer,
)


end_time = time.time()
execution_time_minutes = (end_time - start_time) / 60
print(f"Training completed in {execution_time_minutes:.2f} minutes.")1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.