#LLM Fusion

Simple LLM Merging Surprises

Direct weighted averaging of LLMs with dimensional adaptation proves surprisingly effective, but ratio control is paramount to avoid capability collapse.