Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] can not find blah.safetensors or blah.ckpt for lora blah #1348

Open
waltersamson opened this issue Feb 4, 2025 · 3 comments
Open

[BUG] can not find blah.safetensors or blah.ckpt for lora blah #1348

waltersamson opened this issue Feb 4, 2025 · 3 comments

Comments

@waltersamson
Copy link

Describe the Issue
Loading LoRA for flux.1-dev failed with error message: can not find blah.safetensors or blah.ckpt for lora blah
It can only be loaded if the safetensors file is in the same directory as koboldcpp.py, placing it elsewhere will cause it to fail to load.

After briefly inspecting the code, it seems that in void apply_lora(const std::string& lora_name, float multiplier), lora_model_dir might be empty, causing st_file_path to be just blah.safetensors, instead of the full path.

This can be verified by copying the LoRA safetensors to the working directory and it will load correctly.

Also it seems that LoRA is loaded twice, first during startup, and loaded again at the beginning of generation. Is this necessary?

Additional Information:

  • Debian bookworm
  • kobold.cpp 1.82.4

Full debug log:

***
Welcome to KoboldCpp - Version 1.82.4
Loading .kcpps configuration file...
Overriding Config Value: showgui
For command line arguments, please refer to --help
***
Unable to detect VRAM, please set layers manually.
Auto Selected Default Backend...

Initializing dynamic library: koboldcpp_default.so
==========
[2025-02-04T11:10:08] [SD_LOAD_MODEL] {'model_filename': '/storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf', 'vae_filename': '/storage/2/sd/second-state/ae.safetensors', 'lora_filename': '/storage/2/sd/second-state/lora/motokano.safetensors', 't5xxl_filename': '/storage/2/sd/second-state/t5xxl-Q8_0.gguf', 'clipl_filename': '/storage/2/sd/second-state/clip_l-Q8_0.gguf', 'clipg_filename': ''}

ImageGen Init - Load Model: /storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf
With LoRA: /storage/2/sd/second-state/lora/motokano.safetensors at 1.000000 power
With Custom VAE: /storage/2/sd/second-state/ae.safetensors
With Custom T5-XXL Model: /storage/2/sd/second-state/t5xxl-Q8_0.gguf
With Custom Clip-L Model: /storage/2/sd/second-state/clip_l-Q8_0.gguf

Swap to Diffusion Model Path:/storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf
MODEL:
VAE:/storage/2/sd/second-state/ae.safetensors
TAESD:
CNET:
LORA:
EMBD:
VAE_DEC:0
VAE_TILE:0
FREE_PARAM:0
THREADS:2
WTYPE:39
RNGTYPE:1
SCHED:0
CNETCPU:0


Using CPU backend
loading clip_l from '/storage/2/sd/second-state/clip_l-Q8_0.gguf'
load /storage/2/sd/second-state/clip_l-Q8_0.gguf using gguf format
init from '/storage/2/sd/second-state/clip_l-Q8_0.gguf'
loading t5xxl from '/storage/2/sd/second-state/t5xxl-Q8_0.gguf'
load /storage/2/sd/second-state/t5xxl-Q8_0.gguf using gguf format
init from '/storage/2/sd/second-state/t5xxl-Q8_0.gguf'
loading diffusion model from '/storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf'
load /storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf using gguf format
init from '/storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf'
loading vae from '/storage/2/sd/second-state/ae.safetensors'
load /storage/2/sd/second-state/ae.safetensors using safetensors format
init from '/storage/2/sd/second-state/ae.safetensors'
Version: Flux 
Weight type:                 q8_0
Conditioner weight type:     q8_0
Diffusion model weight type: q8_0
VAE weight type:             f32
ggml tensor size = 400 bytes
vocab size: 49408
 trigger word img already in vocab
Flux blocks: 19 double, 38 single
clip params backend buffer size =  125.20 MB(RAM) (196 tensors)
t5 params backend buffer size =  4826.11 MB(RAM) (219 tensors)
flux params backend buffer size =  12125.39 MB(RAM) (780 tensors)
vae params backend buffer size =  160.00 MB(RAM) (244 tensors)
loading weights
loading tensors from /storage/2/sd/second-state/clip_l-Q8_0.gguf
loading tensors from /storage/2/sd/second-state/t5xxl-Q8_0.gguf
unknown tensor 'text_encoders.t5xxl.transformer.encoder.embed_tokens.weight | q8_0 | 2 [4096, 32128, 1, 1, 1]' in model file
loading tensors from /storage/2/sd/second-state/flux1-dev-Q8_0_city96.gguf
loading tensors from /storage/2/sd/second-state/ae.safetensors
total params memory size = 17236.71MB (VRAM 0.00MB, RAM 17236.71MB): clip 4951.31MB(RAM), unet 12125.39MB(RAM), vae 160.00MB(RAM), controlnet 0.00MB(VRAM), pmid 0.00MB(RAM)
loading model from '' completed, taking 282.34s
running in Flux FLOW mode
finished loaded file
Apply LoRA...

load /storage/2/sd/second-state/lora/motokano.safetensors using safetensors format
init from '/storage/2/sd/second-state/lora/motokano.safetensors'
loading LoRA from '/storage/2/sd/second-state/lora/motokano.safetensors'
loading tensors from /storage/2/sd/second-state/lora/motokano.safetensors
lora params backend buffer size =  584.26 MB(RAM) (912 tensors)
loading tensors from /storage/2/sd/second-state/lora/motokano.safetensors
finished loaded lora
(912 / 912) LoRA tensors applied successfully
lora compute buffer size: 756.38 MB(RAM)
(912 / 912) LoRA tensors applied successfully
lora '/storage/2/sd/second-state/lora/motokano.safetensors' applied, taking 168.50sLoad Image Model OK: True
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.
Embedded SDUI loaded.
======
Active Modules: ImageGeneration
Inactive Modules: TextGeneration VoiceRecognition MultimodalVision NetworkMultiplayer ApiKeyPassword WebSearchProxy TextToSpeech
Enabled APIs: KoboldCppApi A1111ForgeApi ComfyUiApi
Starting Kobold API on port 5001 at http://0.0.0.0:5001/api/
Starting OpenAI Compatible API on port 5001 at http://0.0.0.0:5001/v1/
StableUI is available at http://0.0.0.0:5001/sdui/
======
Please connect to custom endpoint at http://0.0.0.0:5001
IPv6 Socket Failed to Bind. IPv6 will be unavailable.
192.168.1.100 - - [04/Feb/2025 11:31:04] "GET /sdapi/v1/samplers HTTP/1.1" 200 -
192.168.1.100 - - [04/Feb/2025 11:31:05] "GET /sdapi/v1/sd-models HTTP/1.1" 200 -

[2025-02-04T11:31:05] Input: {"steps": 30, "n": 1, "sampler_name": "Euler a", "width": 1024, "height": 1024, "cfg_scale": 7, "seed": 1465036203, "denoising_strength": 0.1, "prompt": "<lora:motokano:1> cute teenage girl, black hair, hat, solo, long hair, snow, looking at viewer, winter, blurry, realistic, upper body, outdoors, snowing, teeth, blurry background, bangs, sweater, black eyes, grin, depth of field, winter clothes", "negative_prompt": "", "init_images": []}

[11:31:05] Generating Image (30 steps)

TXT2IMG PROMPT:<lora:motokano:1> cute teenage girl, black hair, hat, solo, long hair, snow, looking at viewer, winter, blurry, realistic, upper body, outdoors, snowing, teeth, blurry background, bangs, sweater, black eyes, grin, depth of field, winter clothes
NPROMPT:
CLPSKP:-1
CFGSCLE:1.000000
W:1024
H:1024
SM:1
STEP:30
SEED:1465036203
BATCH:1
CIMG:(nil)
CSTR:0.900000


txt2img 1024x1024
lora motokano:1.00
prompt after extract and remove lora: " cute teenage girl, black hair, hat, solo, long hair, snow, looking at viewer, winter, blurry, realistic, upper body, outdoors, snowing, teeth, blurry background, bangs, sweater, black eyes, grin, depth of field, winter clothes"
In quantized models when applying LoRA, the images have poor quality.
Attempting to apply 1 LoRAs
can not find motokano.safetensors or motokano.ckpt for lora motokano
apply_loras completed, taking 0.00s
parse ' cute teenage girl, black hair, hat, solo, long hair, snow, looking at viewer, winter, blurry, realistic, upper body, outdoors, snowing, teeth, blurry background, bangs, sweater, black eyes, grin, depth of field, winter clothes' to [[' cute teenage girl, black hair, hat, solo, long hair, snow, looking at viewer, winter, blurry, realistic, upper body, outdoors, snowing, teeth, blurry background, bangs, sweater, black eyes, grin, depth of field, winter clothes', 1], ]
token length: 77
token length: 256
Missing text_projection matrix, assuming identity...
clip compute buffer size: 1.40 MB(RAM)
Missing text_projection matrix, assuming identity...
t5 compute buffer size: 68.25 MB(RAM)192.168.1.100 - - [04/Feb/2025 11:31:16] "GET /sdui HTTP/1.1" 200 -
192.168.1.100 - - [04/Feb/2025 11:31:17] "GET /sdapi/v1/samplers HTTP/1.1" 200 -
@LostRuins
Copy link
Owner

Ah I see the problem. Dynamic lora loading is currently not supposed to be supported, because I don't want runtime swapping of loaded models within the SD context. If you remove the <lora:motokano:1> and simply select the LoRA at load time in the launcher, it will work correctly. I will fix it in future.

@waltersamson
Copy link
Author

Ah I see the problem. Dynamic lora loading is currently not supposed to be supported, because I don't want runtime swapping of loaded models within the SD context. If you remove the <lora:motokano:1> and simply select the LoRA at load time in the launcher, it will work correctly. I will fix it in future.

Thanks! Now I see the quirk is in the LoRA triggerword, I'll try to work around it. Thanks for taking time to look into this!

@LostRuins
Copy link
Owner

New version v1.83 is out, which should fix this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@LostRuins @waltersamson