Fix Kohya UNet LoRA key conversion for conv_in/conv_out/time_embedding#14006
Draft
dxqb wants to merge 1 commit into
Draft
Fix Kohya UNet LoRA key conversion for conv_in/conv_out/time_embedding#14006dxqb wants to merge 1 commit into
dxqb wants to merge 1 commit into
Conversation
_convert_unet_lora_key() had no mapping for these three top-level UNet submodules, so Kohya-format keys touching them (e.g. lora_unet_conv_in, lora_unet_time_embed_0/2) came out as conv.in/conv.out/time.embed.0/2 instead of conv_in/conv_out/time_embedding.linear_1/2, and were reported as unexpected keys instead of being applied.
Contributor
Author
|
needs a test, I'll mark the PR as ready as soon as that passes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #14005.
What does this PR do?
_convert_unet_lora_key()converts Kohya-format UNet LoRA keys to Diffusers-compatible names by replacing every_with.and then patching specific substrings back (input.blocks→down_blocks,proj.in→proj_in,conv.shortcut→conv_shortcut, etc.). It was missing patches for three top-level UNet submodules:conv.in→conv_inconv.out→conv_outtime.embed.0/time.embed.2→time_embedding.linear_1/time_embedding.linear_2So a Kohya key like
lora_unet_conv_in...came out asconv.in...instead ofconv_in..., didn't match any parameter on the actual model, and got reported as an unexpected key by the PEFT adapter loader instead of being applied.This adds the missing patches. Verified locally that
_convert_unet_lora_keynow produces the correct names forconv_in,conv_out,time_embedding.linear_1, andtime_embedding.linear_2, and that unrelated keys (e.g.emb_layersinside resnet blocks) are unaffected.Who can review?
@yiyixuxu