用开源模型合并利器MergeKit开发自己的定制模型

slices:
  - sources:
      - model: OpenPipe/mistral-ft-optimized-1218
        layer_range: [0, 32]
      - model: mlabonne/NeuralHermes-2.5-Mistral-7B
        layer_range: [0, 32]
merge_method: slerp
base_model: OpenPipe/mistral-ft-optimized-1218
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

models:
  - model: mistralai/Mistral-7B-v0.1
    # 基本模型不需要任何参数
  - model: OpenPipe/mistral-ft-optimized-1218
    parameters:
      density: 0.5
      weight: 0.5
  - model: mlabonne/NeuralHermes-2.5-Mistral-7B
    parameters:
      density: 0.5
      weight: 0.3
merge_method: ties
base_model: mistralai/Mistral-7B-v0.1
parameters:
  normalize: true
dtype: float16

models:
  - model: mistralai/Mistral-7B-v0.1
    #基本模型不需要任何参数
  - model: samir-fama/SamirGPT-v1
    parameters:
      density: 0.53
      weight: 0.4
  - model: abacusai/Slerp-CM-mist-dpo
    parameters:
      density: 0.53
      weight: 0.3
  - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.2
    parameters:
      density: 0.53
      weight: 0.3
merge_method: dare_ties
base_model: mistralai/Mistral-7B-v0.1
parameters:
  int8_mask: true
dtype: bfloat16

slices:
  - sources:
    - model: OpenPipe/mistral-ft-optimized-1218
      layer_range: [0, 32]
  - sources:
    - model: mlabonne/NeuralHermes-2.5-Mistral-7B
      layer_range: [24, 32]
merge_method: passthrough
dtype: bfloat16

!git clone https://github.com/cg123/mergekit.git
!cd mergekit && pip install -q -e .

import yaml

MODEL_NAME = "Marcoro14-7B-slerp"
yaml_config = """
slices:
 - sources:
 - model: AIDC-ai-business/Marcoroni-7B-v3
 layer_range: [0, 32]
 - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1
 layer_range: [0, 32]
merge_method: slerp
base_model: AIDC-ai-business/Marcoroni-7B-v3
parameters:
 t:
 - filter: self_attn
 value: [0, 0.5, 0.3, 0.7, 1]
 - filter: mlp
 value: [1, 0.5, 0.7, 0.3, 0]
 - value: 0.5
dtype: bfloat16

"""

# 将配置保存为yaml文件
with open('config.yaml', 'w', encoding="utf-8") as f:
 f.write(yaml_config)

# 合并模型
!mergekit-yaml config.yaml merge --copy-tokenizer --allow-crimes --out-shard-size 1B --lazy-unpickl

!pip install -qU huggingface_hub
 
from huggingface_hub import ModelCard, ModelCardData
from jinja2 import Template
 
username = "mlabonne"
 
template_text = """
---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
{%- for model in models %}
- {{ model }}
{%- endfor %}
---
 
# {{ model_name }}
 
{{ model_name }} is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
 
{%- for model in models %}
* [{{ model }}](https://huggingface.co/{{ model }})
{%- endfor %}
 
## �� Configuration
 
```yaml
{{- yaml_config -}}
```
"""
 
#创建一个Jinja 模板对象
jinja_template = Template(template_text.strip())
 
# 从配置文件中取得一个模型列表
data = yaml.safe_load(yaml_config)
if "models" in data:
 models = [data["models"][i]["model"] for i in range(len(data["models"])) if "parameters" in data["models"][i]]
elif "parameters" in data:
 models = [data["slices"][0]["sources"][i]["model"] for i in range(len(data["slices"][0]["sources"]))]
elif "slices" in data:
 models = [data["slices"][i]["sources"][0]["model"] for i in range(len(data["slices"]))]
else:
 raise Exception("No models or slices found in yaml config")
 
# 填充模板
content = jinja_template.render(
 model_name=MODEL_NAME,
 models=models,
 yaml_cnotallow=yaml_config,
 username=username,
)
 
# 保存模型卡
card = ModelCard(content)
card.save('merge/README.md')
现在，我们已经创建了模型卡ModelCard。接下来，我们可以将整个文件夹推送到Hugging Face Hub上。
from google.colab import userdata
from huggingface_hub import HfApi
 
username = "mlabonne"
 
#在Google Colab网页的secrets面板上定义
api = HfApi(token=userdata.get("HF_TOKEN"))
 
api.create_repo(
 repo_id=f"{username}/{MODEL_NAME}",
 repo_type="model"
)
api.upload_folder(
 repo_id=f"{username}/{MODEL_NAME}",
 folder_path="merge",
)

!pip install -qU transformers accelerate
 
from transformers import AutoTokenizer
import transformers
import torch
 
model = "mlabonne/Marcoro14-7B-slerp"
messages = [{"role": "user", "content": "What is a large language model?"}]
 
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(
 messages,
 tokenize=False,
 add_generation_prompt=True
)
pipeline = transformers.pipeline(
 "text-generation",
 model=model,
 torch_dtype=torch.float16,
 device_map="auto",
)
 
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)

“A large language model is a type of artificial intelligence (AI) system that has been trained on vast amounts of text data. It’s designed to understand and generate human-like language, making predictions on what words or phrases might come next in a sentence or document. These models use complex algorithms and neural network architectures to learn from the data and improve their performance over time. Some well-known large language models include GPT-3 from OpenAI and BERT from Google.”