Improving Model Merging with Natural Niches

This repository contains code for the paper: Improving Model Merging with Natural Niches.

Using this repository you can reproduce the results in the image below, which shows that Natural Niches can achieve comparable results to CMA-ES when evolving MNIST classifiers from scratch.

Table of Contents

• Installation
• Running Experiments
• Ablation Studies
• Displaying Results
• Citation

Installation

conda env create -f environment.yml

This will create a new Conda environment named natural_niches with all required packages.

Running experiments

Activate the conda environment

conda activate natural_niches

Running the different methods

You can run different methods by specifying the --method parameter. Replace with one of the following options: natural_niches, map_elites, cma_es, or ga.

python main.py --method <method>

Example: Run the ga without crossover:

python main.py --method ga --no_crossover

Ablation Studies

• Without Crossover:

python main.py --method natural_niches --no_crossover

• Without Matchmaker:

python main.py --method natural_niches --no_matchmaker

• Without Split-point:

python main.py --method natural_niches --no_splitpoint

Displaying results

To visualize the results, open the Jupyter notebook plotting.ipynb and run all the cells.

Citation

If you use this code or the ideas from our paper, please cite our work:

@inproceedings{
  abrantes2024improving,
  title={Improving Model Merging with Natural Niches},
  author={Jo{\~a}o Abrantes and Robert Tjarko Lange and Yujin Tang},
  booktitle={UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models},
  year={2024},
  url={https://openreview.net/forum?id=EavQwYUuJp}
}