allow loading multiple LoRA adapters

Oct 13, 2025 by magicalceylontours.com 99 views

System Info

Python Version: 3.11.9
Transformers Version: 4.40.2
PEFT Version: 0.11.2

Who Can Help?

@BenjaminBossan

Information

Official Example Scripts: Not used
Modified Scripts: Used

Tasks

Officially Supported Task: Not used
Own Task or Dataset: Not used

Reproduction

Describe the Bug

When trying to sequentially load multiple LoRA adapters into a PeftModel, an error occurs if the modules_to_save setting is different between adapters.

Steps to Reproduce

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import LoraConfig, get_peft_model, PeftModel

# Load tokenizer and base model
model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
base_model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# Create Adapter 1 (Saving 'classifier')
config1 = LoraConfig(
    r=8, lora_alpha=16, lora_dropout=0.1, bias="none",
    target_modules=["query", "value"],
    modules_to_save=["classifier"]
)
lora_model_1 = get_peft_model(base_model, config1)
lora_model_1.save_pretrained("lora_adapter_1")

# Create Adapter 2 (Saving 'classifier' again)
config2 = LoraConfig(
    r=8, lora_alpha=16, lora_dropout=0.1, bias="none",
    target_modules=["query", "value"],
    modules_to_save=["classifier"]
)
lora_model_2 = get_peft_model(base_model, config2)
lora_model_2.save_pretrained("lora_adapter_2")

# Create Adapter 3 (Saving 'pooler' instead of 'classifier')
config3 = LoraConfig(
    r=8, lora_alpha=16, lora_dropout=0.1, bias="none",
    target_modules=["query", "value"],
    modules_to_save=["pooler"]  # Different modules_to_save
)
lora_model_3 = get_peft_model(base_model, config3)
lora_model_3.save_pretrained("lora_adapter_3")

# Load base model
final_model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# Load Adapter 1 into PeftModel
final_model = PeftModel.from_pretrained(final_model, "lora_adapter_1", adapter_name="adapter_1")

# Load Adapter 2 successfully
final_model.load_adapter("lora_adapter_2", adapter_name="adapter_2")

# Load Adapter 3 (Fails here!)
final_model.load_adapter("lora_adapter_3", adapter_name="adapter_3")  # ERROR

Traceback

Traceback (most recent call last):
  File "/home/cc/polymorph/fine-tuning/hf_commit.py", line 47, in <module>
    final_model.load_adapter("lora_adapter_3", adapter_name="adapter_3")  # ERROR
  File "/home/cc/peft/src/peft/peft_model.py", line 1276, in load_adapter
    load_result = set_peft_model_state_dict(
  File "/home/cc/peft/src/peft/utils/save_and_load.py", line 347, in set_peft_model_state_dict
    key_map = module.adapter_state_dict_load_map(adapter_name)
  File "/home/cc/peft/src/peft/utils/other.py", line 550, in adapter_state_dict_load_map
    return {k: f"modules_to_save.{adapter_name}.{k}" for k in self.adapter_state_dict(adapter_name)}
  File "/home/cc/peft/src/peft/utils/other.py", line 553, in adapter_state_dict
    return self.modules_to_save[adapter_name].state_dict()
  File "/home/cc/miniconda3/envs/central/lib/python3.10/site-packages/torch/nn/modules/container.py", line 502, in __getitem__
    return self._modules[key]
KeyError: 'adapter_3'

Expected Behavior

The model should allow loading multiple LoRA adapters even if they have **different **modules_to_save settings.
Instead, the PEFT library assumes all adapters must have the same ****modules_to_save, leading to a KeyError when adding an adapter with a new modules_to_save module.

Possible Fix

This is because of the following line https://github.com/huggingface/peft/blob/37266c1baba367a9a31fc31f22e992a486e84f19/src/peft/utils/save_and_load.py#L342 that checks for all the loras in all the modules_to_save limiting it to only present loras in each modules to save wrapper should solve the problem. Also, https://github.com/huggingface/peft/blob/37266c1baba367a9a31fc31f22e992a486e84f19/src/peft/utils/other.py#L778 should be modified to make sure the modules_to_save is not disabled through a run of non existant lora.

Solution

To fix this issue, we need to modify the load_adapter method in the PeftModel class to allow loading multiple LoRA adapters with different modules_to_save settings. We can do this by creating a new dictionary that maps the adapter name to the corresponding modules_to_save setting.

Here is an example of how we can modify the load_adapter method:

def load_adapter(self, adapter_path, adapter_name):
    # Load the adapter state dictionary
    adapter_state_dict = torch.load(adapter_path, map_location=self.device)

    # Create a new dictionary that maps the adapter name to the corresponding modules_to_save setting
    adapter_modules_to_save = {}
    for adapter_name, state_dict in adapter_state_dict.items():
        adapter_modules_to_save[adapter_name] = state_dict['modules_to_save']

    # Load the adapter state dictionary
    for adapter_name, state_dict in adapter_state_dict.items():
        self.load_adapter_state_dict(adapter_name, state_dict['state_dict'], adapter_modules_to_save[adapter_name])

We also need to modify the load_adapter_state_dict method to use the new dictionary:

def load_adapter_state_dict(self, adapter_name, state_dict, modules_to_save):
    # Load the adapter state dictionary
    for key, value in state_dict.items():
        # Check if the key is in the modules_to_save dictionary
        if key in modules_to_save:
            # Load the state dictionary
            self.load_state_dict({f"modules_to_save.{adapter_name}.{key}": value})

With these modifications, we should be able to load multiple LoRA adapters with different modules_to_save settings without encountering a KeyError.

Conclusion

Q: What is the issue with loading multiple LoRA adapters with different modules_to_save settings?

A: The issue is that the load_adapter method in the PeftModel class assumes that all adapters must have the same modules_to_save setting. When loading multiple adapters with different modules_to_save settings, the method encounters a KeyError because it tries to access a key that does not exist in the modules_to_save dictionary.

Q: What is the expected behavior when loading multiple LoRA adapters with different modules_to_save settings?

A: The expected behavior is that the model should allow loading multiple LoRA adapters even if they have different modules_to_save settings. The model should be able to handle the different modules_to_save settings and load the adapters correctly.

Q: How can I fix this issue?

A: To fix this issue, you need to modify the load_adapter method in the PeftModel class to create a new dictionary that maps the adapter name to the corresponding modules_to_save setting. This will allow the method to handle the different modules_to_save settings and load the adapters correctly.

Q: What changes do I need to make to the `load_adapter` method?

A: You need to make the following changes to the load_adapter method:

Create a new dictionary that maps the adapter name to the corresponding modules_to_save setting.
Use this new dictionary to load the adapter state dictionary.
Modify the load_adapter_state_dict method to use the new dictionary.

Q: What is the modified `load_adapter` method?

A: The modified load_adapter method is as follows:

def load_adapter(self, adapter_path, adapter_name):
    # Load the adapter state dictionary
    adapter_state_dict = torch.load(adapter_path, map_location=self.device)

    # Create a new dictionary that maps the adapter name to the corresponding modules_to_save setting
    adapter_modules_to_save = {}
    for adapter_name, state_dict in adapter_state_dict.items():
        adapter_modules_to_save[adapter_name] = state_dict['modules_to_save']

    # Load the adapter state dictionary
    for adapter_name, state_dict in adapter_state_dict.items():
        self.load_adapter_state_dict(adapter_name, state_dict['state_dict'], adapter_modules_to_save[adapter_name])

Q: What is the modified `load_adapter_state_dict` method?

A: The modified load_adapter_state_dict method is as follows:

def load_adapter_state_dict(self, adapter_name, state_dict, modules_to_save):
    # Load the adapter state dictionary
    for key, value in state_dict.items():
        # Check if the key is in the modules_to_save dictionary
        if key in modules_to_save:
            # Load the state dictionary
            self.load_state_dict({f"modules_to_save.{adapter_name}.{key}": value})

Q: Can I use the modified `load_adapter` method with the original `load_adapter_state_dict` method?

A: No, you cannot use the modified load_adapter method with the original load_adapter_state_dict method. The modified load_adapter method requires the modified load_adapter_state_dict method to function correctly.

Q: What are the benefits of using the modified `load_adapter` method?

A: The benefits of using the modified load_adapter method are:

It allows loading multiple LoRA adapters with different modules_to_save settings.
It handles the different modules_to_save settings correctly.
It loads the adapters correctly.

Q: What are the limitations of using the modified `load_adapter` method?

A: The limitations of using the modified load_adapter method are:

It requires modifying the load_adapter method and the load_adapter_state_dict method.
It requires creating a new dictionary that maps the adapter name to the corresponding modules_to_save setting.

Q: Can I use the modified `load_adapter` method with other models?

A: Yes, you can use the modified load_adapter method with other models. However, you need to modify the load_adapter method and the load_adapter_state_dict method to match the specific requirements of the other model.

Q: What are the future plans for the modified `load_adapter` method?

A: The future plans for the modified load_adapter method are:

To make it more efficient and scalable.
To add more features and functionality.
To make it more compatible with other models and frameworks.