How to load checkpoints from local model path?

[LKAMING97]

This is the path where I currently store the model weight location. I have been unable to read vqgan.

[LKAMING97]

[LKAMING97]

Some weights of ChameleonForConditionalGeneration were not initialized from the model checkpoint at /root/autodl-tmp/Lumina-mGPT/lumina_mgpt/ckpts/models--Alpha-VLLM--Lumina-mGPT-7B-768-Omni and are newly initialized: ['model.vqmodel.encoder.conv_in.bias', 'model.vqmodel.encoder.conv_in.weight', 'model.vqmodel.encoder.conv_out.bias', 'model.vqmodel.encoder.conv_out.weight', 'model.vqmodel.encoder.down.0.block.0.conv1.bias', 'model.vqmodel.encoder.down.0.block.0.conv1.weight', 'model.vqmodel.encoder.down.0.block.0.conv2.bias', 'model.vqmodel.encoder.down.0.block.0.conv2.weight', 'model.vqmodel.encoder.down.0.block.0.norm1.bias', 'model.vqmodel.encoder.down.0.block.0.norm1.weight', 'model.vqmodel.encoder.down.0.block.0.norm2.bias', 'model.vqmodel.encoder.down.0.block.0.norm2.weight', 'model.vqmodel.encoder.down.0.block.1.conv1.bias', 'model.vqmodel.encoder.down.0.block.1.conv1.weight', 'model.vqmodel.encoder.down.0.block.1.conv2.bias', 'model.vqmodel.encoder.down.0.block.1.conv2.weight', 'model.vqmodel.encoder.down.0.block.1.norm1.bias', 'model.vqmodel.encoder.down.0.block.1.norm1.weight', 'model.vqmodel.encoder.down.0.block.1.norm2.bias', 'model.vqmodel.encoder.down.0.block.1.norm2.weight', 'model.vqmodel.encoder.down.0.downsample.conv.bias', 'model.vqmodel.encoder.down.0.downsample.conv.weight', 'model.vqmodel.encoder.down.1.block.0.conv1.bias', 'model.vqmodel.encoder.down.1.block.0.conv1.weight', 'model.vqmodel.encoder.down.1.block.0.conv2.bias', 'model.vqmodel.encoder.down.1.block.0.conv2.weight', 'model.vqmodel.encoder.down.1.block.0.norm1.bias', 'model.vqmodel.encoder.down.1.block.0.norm1.weight', 'model.vqmodel.encoder.down.1.block.0.norm2.bias', 'model.vqmodel.encoder.down.1.block.0.norm2.weight', 'model.vqmodel.encoder.down.1.block.1.conv1.bias', 'model.vqmodel.encoder.down.1.block.1.conv1.weight', 'model.vqmodel.encoder.down.1.block.1.conv2.bias', 'model.vqmodel.encoder.down.1.block.1.conv2.weight', 'model.vqmodel.encoder.down.1.block.1.norm1.bias', 'model.vqmodel.encoder.down.1.block.1.norm1.weight', 'model.vqmodel.encoder.down.1.block.1.norm2.bias', 'model.vqmodel.encoder.down.1.block.1.norm2.weight', 'model.vqmodel.encoder.down.1.downsample.conv.bias', 'model.vqmodel.encoder.down.1.downsample.conv.weight', 'model.vqmodel.encoder.down.2.block.0.conv1.bias', 'model.vqmodel.encoder.down.2.block.0.conv1.weight', 'model.vqmodel.encoder.down.2.block.0.conv2.bias', 'model.vqmodel.encoder.down.2.block.0.conv2.weight', 'model.vqmodel.encoder.down.2.block.0.nin_shortcut.bias', 'model.vqmodel.encoder.down.2.block.0.nin_shortcut.weight', 'model.vqmodel.encoder.down.2.block.0.norm1.bias', 'model.vqmodel.encoder.down.2.block.0.norm1.weight', 'model.vqmodel.encoder.down.2.block.0.norm2.bias', 'model.vqmodel.encoder.down.2.block.0.norm2.weight', 'model.vqmodel.encoder.down.2.block.1.conv1.bias', 'model.vqmodel.encoder.down.2.block.1.conv1.weight', 'model.vqmodel.encoder.down.2.block.1.conv2.bias', 'model.vqmodel.encoder.down.2.block.1.conv2.weight', 'model.vqmodel.encoder.down.2.block.1.norm1.bias', 'model.vqmodel.encoder.down.2.block.1.norm1.weight', 'model.vqmodel.encoder.down.2.block.1.norm2.bias', 'model.vqmodel.encoder.down.2.block.1.norm2.weight', 'model.vqmodel.encoder.down.2.downsample.conv.bias', 'model.vqmodel.encoder.down.2.downsample.conv.weight', 'model.vqmodel.encoder.down.3.block.0.conv1.bias', 'model.vqmodel.encoder.down.3.block.0.conv1.weight', 'model.vqmodel.encoder.down.3.block.0.conv2.bias', 'model.vqmodel.encoder.down.3.block.0.conv2.weight', 'model.vqmodel.encoder.down.3.block.0.norm1.bias', 'model.vqmodel.encoder.down.3.block.0.norm1.weight', 'model.vqmodel.encoder.down.3.block.0.norm2.bias', 'model.vqmodel.encoder.down.3.block.0.norm2.weight', 'model.vqmodel.encoder.down.3.block.1.conv1.bias', 'model.vqmodel.encoder.down.3.block.1.conv1.weight', 'model.vqmodel.encoder.down.3.block.1.conv2.bias', 'model.vqmodel.encoder.down.3.block.1.conv2.weight', 'model.vqmodel.encoder.down.3.block.1.norm1.bias', 'model.vqmodel.encoder.down.3.block.1.norm1.weight', 'model.vqmodel.encoder.down.3.block.1.norm2.bias', 'model.vqmodel.encoder.down.3.block.1.norm2.weight', 'model.vqmodel.encoder.down.3.downsample.conv.bias', 'model.vqmodel.encoder.down.3.downsample.conv.weight', 'model.vqmodel.encoder.down.4.block.0.conv1.bias', 'model.vqmodel.encoder.down.4.block.0.conv1.weight', 'model.vqmodel.encoder.down.4.block.0.conv2.bias', 'model.vqmodel.encoder.down.4.block.0.conv2.weight', 'model.vqmodel.encoder.down.4.block.0.nin_shortcut.bias', 'model.vqmodel.encoder.down.4.block.0.nin_shortcut.weight', 'model.vqmodel.encoder.down.4.block.0.norm1.bias', 'model.vqmodel.encoder.down.4.block.0.norm1.weight', 'model.vqmodel.encoder.down.4.block.0.norm2.bias', 'model.vqmodel.encoder.down.4.block.0.norm2.weight', 'model.vqmodel.encoder.down.4.block.1.conv1.bias', 'model.vqmodel.encoder.down.4.block.1.conv1.weight', 'model.vqmodel.encoder.down.4.block.1.conv2.bias', 'model.vqmodel.encoder.down.4.block.1.conv2.weight', 'model.vqmodel.encoder.down.4.block.1.norm1.bias', 'model.vqmodel.encoder.down.4.block.1.norm1.weight', 'model.vqmodel.encoder.down.4.block.1.norm2.bias', 'model.vqmodel.encoder.down.4.block.1.norm2.weight', 'model.vqmodel.encoder.mid.attn_1.k.bias', 'model.vqmodel.encoder.mid.attn_1.k.weight', 'model.vqmodel.encoder.mid.attn_1.norm.bias', 'model.vqmodel.encoder.mid.attn_1.norm.weight', 'model.vqmodel.encoder.mid.attn_1.proj_out.bias', 'model.vqmodel.encoder.mid.attn_1.proj_out.weight', 'model.vqmodel.encoder.mid.attn_1.q.bias', 'model.vqmodel.encoder.mid.attn_1.q.weight', 'model.vqmodel.encoder.mid.attn_1.v.bias', 'model.vqmodel.encoder.mid.attn_1.v.weight', 'model.vqmodel.encoder.mid.block_1.conv1.bias', 'model.vqmodel.encoder.mid.block_1.conv1.weight', 'model.vqmodel.encoder.mid.block_1.conv2.bias', 'model.vqmodel.encoder.mid.block_1.conv2.weight', 'model.vqmodel.encoder.mid.block_1.norm1.bias', 'model.vqmodel.encoder.mid.block_1.norm1.weight', 'model.vqmodel.encoder.mid.block_1.norm2.bias', 'model.vqmodel.encoder.mid.block_1.norm2.weight', 'model.vqmodel.encoder.mid.block_2.conv1.bias', 'model.vqmodel.encoder.mid.block_2.conv1.weight', 'model.vqmodel.encoder.mid.block_2.conv2.bias', 'model.vqmodel.encoder.mid.block_2.conv2.weight', 'model.vqmodel.encoder.mid.block_2.norm1.bias', 'model.vqmodel.encoder.mid.block_2.norm1.weight', 'model.vqmodel.encoder.mid.block_2.norm2.bias', 'model.vqmodel.encoder.mid.block_2.norm2.weight', 'model.vqmodel.encoder.norm_out.bias', 'model.vqmodel.encoder.norm_out.weight', 'model.vqmodel.post_quant_conv.bias', 'model.vqmodel.post_quant_conv.weight', 'model.vqmodel.quant_conv.bias', 'model.vqmodel.quant_conv.weight', 'model.vqmodel.quantize.embedding.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

[missTL]

Have you solved the problem?

[vancyland]

i have the same issue