Model Merging

Creator

Creator

Seonglae Cho

Created

Created

2024 Mar 30 17:30

Editor

Editor

Seonglae Cho

Edited

Edited

2024 May 2 15:20

Refs

Refs

Combining multiple pretrained models into one model, giving it the combined abilities of each individual model without any additional training

Model Merging Notion

Model Stacking

Model Blending

Weighted Averaging

Model Merging Usages

Model soups

Task Arithmetic

Merge Large Language Models with mergekit

A Blog post by Maxime Labonne on Hugging Face

Merge Large Language Models with mergekit

https://huggingface.co/blog/mlabonne/merge-models

Merge Large Language Models with mergekit

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

https://huggingface.co/docs/peft/developer_guides/model_merging

Model merging

Recommendations

//////