task-arithmetic

[ICLR 2026] OptMerge: data-free merging of multimodal LLM experts (VQA, Geometry, Chart, OCR, Grounding) and modalities, plus SWUDI / ASWUDI closed-form spectral merging for CLIP-ViT, Flan-T5, Llama-3.2 and MLLMs.

Official repository of "Localizing Task Information for Improved Model Merging and Compression" [ICML 2024]

[NeurIPS'24] Official PyTorch implementation for paper "Knowledge Composition using Task Vectors with Learned Anisotropic Scaling"

SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation

Official repository of "Task Vector Quantization for Memory-Efficient Model Merging" [ICCV 2025]

SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery. Arxiv, 2024.

Official repository of our work "Efficient Model Editing with Task-Localized Sparse Fine-tuning" accepted at ICLR 2025

Official code for "Zero-Shot Quantization via Weight-Space Arithmetic" (NeurIPS 2026). Transfer quantization robustness across tasks with quantization vectors, no receiver-side QAT.

Codebase for "MASS: MoErging through Adaptive Subspace Selection."

Official version of the PyTorch implementation of MonoSoup method for model merging.

A comprehensive evaluation of Machine Unlearning via Task Arithmetic on CNN architectures (ResNet-18, VGG-11, MobileNetV2). Includes SOTA metrics like ZRF, MIA, and the novel Anamnesis Index (AIN) for Rebound Effect analysis.

MEng thesis: calibrated merging of LoRA adapters in a 3B Speech LLM recovers single-task performance on average across 7 speech tasks, using 16× less training data than joint multi-task training.

Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. ACM Computing Surveys, 2026.

Multi-Task Learning via Model Merging for Toxicity and Spam Detection. Explores Task Arithmetic and TIES-Merging on DistilBERT, evaluated against a BART-Large Zero-Shot baseline.

Code and measurements for when rank-one knowledge edits merge. Numerical archive on Zenodo.

This repository contain a project which goal is to find new parameter efficient fine tuning framework in order to improve performance of Deep Artificial Neural Network onto "out of distribution" data (OOD). In this specific case you can find Multi-task Learning problem.

MergeForge: world-class model merging / task-arithmetic benchmark (pure NumPy, bitwise deterministic). Author: 晨星