/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch_tensorrt/fx/tracer/acc_tracer/acc_ops.py:895: UserWarning: Unable to import torchvision related libraries.: No module named 'torchvision'. Please install torchvision lib in order to lower stochastic_depth
  warnings.warn(
Unable to import quantization op. Please install modelopt library (https://github.com/NVIDIA/TensorRT-Model-Optimizer?tab=readme-ov-file#installation) to add support for compiling quantized models
[12/19/2024-19:02:20] [TRT] [W] Functionality provided through tensorrt.plugin module is experimental.

Loading pipeline components...:   0%|          | 0/7 [00:00<?, ?it/s]You set `add_prefix_space`. The tokenizer needs to be converted from the slow tokenizers

Loading pipeline components...:  14%|█▍        | 1/7 [00:00<00:01,  3.32it/s]
Loading pipeline components...:  29%|██▊       | 2/7 [00:00<00:01,  4.24it/s]
Loading pipeline components...:  43%|████▎     | 3/7 [00:02<00:04,  1.18s/it]

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]

Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.05it/s]

Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  2.18it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  2.16it/s]

Loading pipeline components...:  57%|█████▋    | 4/7 [00:03<00:03,  1.11s/it]
Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00,  2.26it/s]
Loading pipeline components...: 100%|██████████| 7/7 [00:03<00:00,  1.77it/s]
ep_gen section Start...
Directly use _export because torch.export.export doesn't work
ep_gen section End...
ep_gen section elapsed time: 10.757770538330078 seconds
Generating TRT engine now, dryrun=False...
trt_gen section Start...
DEBUG:torch_tensorrt.dynamo.lowering.passes.remove_detach:Removed 0 detach nodes:
graph():
    %p_module_time_text_embed_timestep_embedder_linear_1_weight : [num_users=1] = placeholder[target=p_module_time_text_embed_timestep_embedder_linear_1_weight]
    %p_module_time_text_embed_timestep_embedder_linear_1_bias : [num_users=1] = placeholder[target=p_module_time_text_embed_timestep_embedder_linear_1_bias]
    %p_module_time_text_embed_timestep_embedder_linear_2_weight : [num_users=1] = placeholder[target=p_module_time_text_embed_timestep_embedder_linear_2_weight]
    %p_module_time_text_embed_timestep_embedder_linear_2_bias : [num_users=1] = placeholder[target=p_module_time_text_embed_timestep_embedder_linear_2_bias]
    %p_module_time_text_embed_text_embedder_linear_1_weight : [num_users=1] = placeholder[target=p_module_time_text_embed_text_embedder_linear_1_weight]
    %p_module_time_text_embed_text_embedder_linear_1_bias : [num_users=1] = placeholder[target=p_module_time_text_embed_text_embedder_linear_1_bias]
    %p_module_time_text_embed_text_embedder_linear_2_weight : [num_users=1] = placeholder[target=p_module_time_text_embed_text_embedder_linear_2_weight]
    %p_module_time_text_embed_text_embedder_linear_2_bias : [num_users=1] = placeholder[target=p_module_time_text_embed_text_embedder_linear_2_bias]
    %p_module_context_embedder_weight : [num_users=1] = placeholder[target=p_module_context_embedder_weight]
    %p_module_context_embedder_bias : [num_users=1] = placeholder[target=p_module_context_embedder_bias]
    %p_module_x_embedder_weight : [num_users=1] = placeholder[target=p_module_x_embedder_weight]
    %p_module_x_embedder_bias : [num_users=1] = placeholder[target=p_module_x_embedder_bias]
    %p_module_transformer_blocks_0_norm1_linear_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_norm1_linear_weight]
    %p_module_transformer_blocks_0_norm1_linear_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_norm1_linear_bias]
    %p_module_transformer_blocks_0_norm1_context_linear_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_norm1_context_linear_weight]
    %p_module_transformer_blocks_0_norm1_context_linear_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_norm1_context_linear_bias]
    %p_module_transformer_blocks_0_attn_norm_q_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_norm_q_weight]
    %p_module_transformer_blocks_0_attn_norm_k_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_norm_k_weight]
    %p_module_transformer_blocks_0_attn_to_q_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_q_weight]
    %p_module_transformer_blocks_0_attn_to_q_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_q_bias]
    %p_module_transformer_blocks_0_attn_to_k_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_k_weight]
    %p_module_transformer_blocks_0_attn_to_k_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_k_bias]
    %p_module_transformer_blocks_0_attn_to_v_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_v_weight]
    %p_module_transformer_blocks_0_attn_to_v_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_v_bias]
    %p_module_transformer_blocks_0_attn_add_k_proj_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_add_k_proj_weight]
    %p_module_transformer_blocks_0_attn_add_k_proj_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_add_k_proj_bias]
    %p_module_transformer_blocks_0_attn_add_v_proj_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_add_v_proj_weight]
    %p_module_transformer_blocks_0_attn_add_v_proj_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_add_v_proj_bias]
    %p_module_transformer_blocks_0_attn_add_q_proj_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_add_q_proj_weight]
    %p_module_transformer_blocks_0_attn_add_q_proj_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_add_q_proj_bias]
    %p_module_transformer_blocks_0_attn_to_out_0_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_out_0_weight]
    %p_module_transformer_blocks_0_attn_to_out_0_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_out_0_bias]
    %p_module_transformer_blocks_0_attn_to_add_out_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_add_out_weight]
    %p_module_transformer_blocks_0_attn_to_add_out_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_to_add_out_bias]
    %p_module_transformer_blocks_0_attn_norm_added_q_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_norm_added_q_weight]
    %p_module_transformer_blocks_0_attn_norm_added_k_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_attn_norm_added_k_weight]
    %p_module_transformer_blocks_0_ff_net_0_proj_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_net_0_proj_weight]
    %p_module_transformer_blocks_0_ff_net_0_proj_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_net_0_proj_bias]
    %p_module_transformer_blocks_0_ff_net_2_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_net_2_weight]
    %p_module_transformer_blocks_0_ff_net_2_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_net_2_bias]
    %p_module_transformer_blocks_0_ff_context_net_0_proj_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_0_ff_context_net_0_proj_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_0_ff_context_net_2_weight : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_context_net_2_weight]
    %p_module_transformer_blocks_0_ff_context_net_2_bias : [num_users=1] = placeholder[target=p_module_transformer_blocks_0_ff_context_net_2_bias]
    %p_module_transformer_blocks_1_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_norm1_linear_weight]
    %p_module_transformer_blocks_1_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_norm1_linear_bias]
    %p_module_transformer_blocks_1_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_norm1_context_linear_weight]
    %p_module_transformer_blocks_1_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_norm1_context_linear_bias]
    %p_module_transformer_blocks_1_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_norm_q_weight]
    %p_module_transformer_blocks_1_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_norm_k_weight]
    %p_module_transformer_blocks_1_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_q_weight]
    %p_module_transformer_blocks_1_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_q_bias]
    %p_module_transformer_blocks_1_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_k_weight]
    %p_module_transformer_blocks_1_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_k_bias]
    %p_module_transformer_blocks_1_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_v_weight]
    %p_module_transformer_blocks_1_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_v_bias]
    %p_module_transformer_blocks_1_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_add_k_proj_weight]
    %p_module_transformer_blocks_1_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_add_k_proj_bias]
    %p_module_transformer_blocks_1_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_add_v_proj_weight]
    %p_module_transformer_blocks_1_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_add_v_proj_bias]
    %p_module_transformer_blocks_1_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_add_q_proj_weight]
    %p_module_transformer_blocks_1_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_add_q_proj_bias]
    %p_module_transformer_blocks_1_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_out_0_weight]
    %p_module_transformer_blocks_1_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_out_0_bias]
    %p_module_transformer_blocks_1_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_add_out_weight]
    %p_module_transformer_blocks_1_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_to_add_out_bias]
    %p_module_transformer_blocks_1_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_norm_added_q_weight]
    %p_module_transformer_blocks_1_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_attn_norm_added_k_weight]
    %p_module_transformer_blocks_1_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_net_0_proj_weight]
    %p_module_transformer_blocks_1_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_net_0_proj_bias]
    %p_module_transformer_blocks_1_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_net_2_weight]
    %p_module_transformer_blocks_1_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_net_2_bias]
    %p_module_transformer_blocks_1_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_1_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_1_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_context_net_2_weight]
    %p_module_transformer_blocks_1_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_1_ff_context_net_2_bias]
    %p_module_transformer_blocks_2_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_norm1_linear_weight]
    %p_module_transformer_blocks_2_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_norm1_linear_bias]
    %p_module_transformer_blocks_2_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_norm1_context_linear_weight]
    %p_module_transformer_blocks_2_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_norm1_context_linear_bias]
    %p_module_transformer_blocks_2_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_norm_q_weight]
    %p_module_transformer_blocks_2_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_norm_k_weight]
    %p_module_transformer_blocks_2_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_q_weight]
    %p_module_transformer_blocks_2_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_q_bias]
    %p_module_transformer_blocks_2_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_k_weight]
    %p_module_transformer_blocks_2_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_k_bias]
    %p_module_transformer_blocks_2_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_v_weight]
    %p_module_transformer_blocks_2_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_v_bias]
    %p_module_transformer_blocks_2_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_add_k_proj_weight]
    %p_module_transformer_blocks_2_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_add_k_proj_bias]
    %p_module_transformer_blocks_2_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_add_v_proj_weight]
    %p_module_transformer_blocks_2_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_add_v_proj_bias]
    %p_module_transformer_blocks_2_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_add_q_proj_weight]
    %p_module_transformer_blocks_2_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_add_q_proj_bias]
    %p_module_transformer_blocks_2_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_out_0_weight]
    %p_module_transformer_blocks_2_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_out_0_bias]
    %p_module_transformer_blocks_2_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_add_out_weight]
    %p_module_transformer_blocks_2_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_to_add_out_bias]
    %p_module_transformer_blocks_2_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_norm_added_q_weight]
    %p_module_transformer_blocks_2_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_attn_norm_added_k_weight]
    %p_module_transformer_blocks_2_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_net_0_proj_weight]
    %p_module_transformer_blocks_2_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_net_0_proj_bias]
    %p_module_transformer_blocks_2_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_net_2_weight]
    %p_module_transformer_blocks_2_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_net_2_bias]
    %p_module_transformer_blocks_2_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_2_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_2_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_context_net_2_weight]
    %p_module_transformer_blocks_2_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_2_ff_context_net_2_bias]
    %p_module_transformer_blocks_3_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_norm1_linear_weight]
    %p_module_transformer_blocks_3_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_norm1_linear_bias]
    %p_module_transformer_blocks_3_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_norm1_context_linear_weight]
    %p_module_transformer_blocks_3_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_norm1_context_linear_bias]
    %p_module_transformer_blocks_3_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_norm_q_weight]
    %p_module_transformer_blocks_3_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_norm_k_weight]
    %p_module_transformer_blocks_3_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_q_weight]
    %p_module_transformer_blocks_3_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_q_bias]
    %p_module_transformer_blocks_3_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_k_weight]
    %p_module_transformer_blocks_3_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_k_bias]
    %p_module_transformer_blocks_3_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_v_weight]
    %p_module_transformer_blocks_3_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_v_bias]
    %p_module_transformer_blocks_3_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_add_k_proj_weight]
    %p_module_transformer_blocks_3_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_add_k_proj_bias]
    %p_module_transformer_blocks_3_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_add_v_proj_weight]
    %p_module_transformer_blocks_3_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_add_v_proj_bias]
    %p_module_transformer_blocks_3_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_add_q_proj_weight]
    %p_module_transformer_blocks_3_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_add_q_proj_bias]
    %p_module_transformer_blocks_3_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_out_0_weight]
    %p_module_transformer_blocks_3_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_out_0_bias]
    %p_module_transformer_blocks_3_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_add_out_weight]
    %p_module_transformer_blocks_3_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_to_add_out_bias]
    %p_module_transformer_blocks_3_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_norm_added_q_weight]
    %p_module_transformer_blocks_3_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_attn_norm_added_k_weight]
    %p_module_transformer_blocks_3_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_net_0_proj_weight]
    %p_module_transformer_blocks_3_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_net_0_proj_bias]
    %p_module_transformer_blocks_3_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_net_2_weight]
    %p_module_transformer_blocks_3_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_net_2_bias]
    %p_module_transformer_blocks_3_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_3_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_3_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_context_net_2_weight]
    %p_module_transformer_blocks_3_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_3_ff_context_net_2_bias]
    %p_module_transformer_blocks_4_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_norm1_linear_weight]
    %p_module_transformer_blocks_4_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_norm1_linear_bias]
    %p_module_transformer_blocks_4_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_norm1_context_linear_weight]
    %p_module_transformer_blocks_4_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_norm1_context_linear_bias]
    %p_module_transformer_blocks_4_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_norm_q_weight]
    %p_module_transformer_blocks_4_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_norm_k_weight]
    %p_module_transformer_blocks_4_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_q_weight]
    %p_module_transformer_blocks_4_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_q_bias]
    %p_module_transformer_blocks_4_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_k_weight]
    %p_module_transformer_blocks_4_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_k_bias]
    %p_module_transformer_blocks_4_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_v_weight]
    %p_module_transformer_blocks_4_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_v_bias]
    %p_module_transformer_blocks_4_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_add_k_proj_weight]
    %p_module_transformer_blocks_4_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_add_k_proj_bias]
    %p_module_transformer_blocks_4_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_add_v_proj_weight]
    %p_module_transformer_blocks_4_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_add_v_proj_bias]
    %p_module_transformer_blocks_4_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_add_q_proj_weight]
    %p_module_transformer_blocks_4_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_add_q_proj_bias]
    %p_module_transformer_blocks_4_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_out_0_weight]
    %p_module_transformer_blocks_4_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_out_0_bias]
    %p_module_transformer_blocks_4_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_add_out_weight]
    %p_module_transformer_blocks_4_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_to_add_out_bias]
    %p_module_transformer_blocks_4_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_norm_added_q_weight]
    %p_module_transformer_blocks_4_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_attn_norm_added_k_weight]
    %p_module_transformer_blocks_4_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_net_0_proj_weight]
    %p_module_transformer_blocks_4_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_net_0_proj_bias]
    %p_module_transformer_blocks_4_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_net_2_weight]
    %p_module_transformer_blocks_4_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_net_2_bias]
    %p_module_transformer_blocks_4_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_4_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_4_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_context_net_2_weight]
    %p_module_transformer_blocks_4_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_4_ff_context_net_2_bias]
    %p_module_transformer_blocks_5_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_norm1_linear_weight]
    %p_module_transformer_blocks_5_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_norm1_linear_bias]
    %p_module_transformer_blocks_5_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_norm1_context_linear_weight]
    %p_module_transformer_blocks_5_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_norm1_context_linear_bias]
    %p_module_transformer_blocks_5_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_norm_q_weight]
    %p_module_transformer_blocks_5_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_norm_k_weight]
    %p_module_transformer_blocks_5_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_q_weight]
    %p_module_transformer_blocks_5_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_q_bias]
    %p_module_transformer_blocks_5_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_k_weight]
    %p_module_transformer_blocks_5_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_k_bias]
    %p_module_transformer_blocks_5_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_v_weight]
    %p_module_transformer_blocks_5_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_v_bias]
    %p_module_transformer_blocks_5_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_add_k_proj_weight]
    %p_module_transformer_blocks_5_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_add_k_proj_bias]
    %p_module_transformer_blocks_5_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_add_v_proj_weight]
    %p_module_transformer_blocks_5_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_add_v_proj_bias]
    %p_module_transformer_blocks_5_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_add_q_proj_weight]
    %p_module_transformer_blocks_5_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_add_q_proj_bias]
    %p_module_transformer_blocks_5_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_out_0_weight]
    %p_module_transformer_blocks_5_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_out_0_bias]
    %p_module_transformer_blocks_5_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_add_out_weight]
    %p_module_transformer_blocks_5_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_to_add_out_bias]
    %p_module_transformer_blocks_5_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_norm_added_q_weight]
    %p_module_transformer_blocks_5_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_attn_norm_added_k_weight]
    %p_module_transformer_blocks_5_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_net_0_proj_weight]
    %p_module_transformer_blocks_5_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_net_0_proj_bias]
    %p_module_transformer_blocks_5_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_net_2_weight]
    %p_module_transformer_blocks_5_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_net_2_bias]
    %p_module_transformer_blocks_5_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_5_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_5_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_context_net_2_weight]
    %p_module_transformer_blocks_5_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_5_ff_context_net_2_bias]
    %p_module_transformer_blocks_6_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_norm1_linear_weight]
    %p_module_transformer_blocks_6_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_norm1_linear_bias]
    %p_module_transformer_blocks_6_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_norm1_context_linear_weight]
    %p_module_transformer_blocks_6_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_norm1_context_linear_bias]
    %p_module_transformer_blocks_6_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_norm_q_weight]
    %p_module_transformer_blocks_6_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_norm_k_weight]
    %p_module_transformer_blocks_6_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_q_weight]
    %p_module_transformer_blocks_6_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_q_bias]
    %p_module_transformer_blocks_6_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_k_weight]
    %p_module_transformer_blocks_6_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_k_bias]
    %p_module_transformer_blocks_6_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_v_weight]
    %p_module_transformer_blocks_6_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_v_bias]
    %p_module_transformer_blocks_6_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_add_k_proj_weight]
    %p_module_transformer_blocks_6_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_add_k_proj_bias]
    %p_module_transformer_blocks_6_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_add_v_proj_weight]
    %p_module_transformer_blocks_6_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_add_v_proj_bias]
    %p_module_transformer_blocks_6_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_add_q_proj_weight]
    %p_module_transformer_blocks_6_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_add_q_proj_bias]
    %p_module_transformer_blocks_6_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_out_0_weight]
    %p_module_transformer_blocks_6_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_out_0_bias]
    %p_module_transformer_blocks_6_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_add_out_weight]
    %p_module_transformer_blocks_6_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_to_add_out_bias]
    %p_module_transformer_blocks_6_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_norm_added_q_weight]
    %p_module_transformer_blocks_6_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_attn_norm_added_k_weight]
    %p_module_transformer_blocks_6_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_net_0_proj_weight]
    %p_module_transformer_blocks_6_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_net_0_proj_bias]
    %p_module_transformer_blocks_6_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_net_2_weight]
    %p_module_transformer_blocks_6_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_net_2_bias]
    %p_module_transformer_blocks_6_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_6_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_6_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_context_net_2_weight]
    %p_module_transformer_blocks_6_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_6_ff_context_net_2_bias]
    %p_module_transformer_blocks_7_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_norm1_linear_weight]
    %p_module_transformer_blocks_7_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_norm1_linear_bias]
    %p_module_transformer_blocks_7_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_norm1_context_linear_weight]
    %p_module_transformer_blocks_7_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_norm1_context_linear_bias]
    %p_module_transformer_blocks_7_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_norm_q_weight]
    %p_module_transformer_blocks_7_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_norm_k_weight]
    %p_module_transformer_blocks_7_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_q_weight]
    %p_module_transformer_blocks_7_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_q_bias]
    %p_module_transformer_blocks_7_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_k_weight]
    %p_module_transformer_blocks_7_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_k_bias]
    %p_module_transformer_blocks_7_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_v_weight]
    %p_module_transformer_blocks_7_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_v_bias]
    %p_module_transformer_blocks_7_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_add_k_proj_weight]
    %p_module_transformer_blocks_7_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_add_k_proj_bias]
    %p_module_transformer_blocks_7_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_add_v_proj_weight]
    %p_module_transformer_blocks_7_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_add_v_proj_bias]
    %p_module_transformer_blocks_7_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_add_q_proj_weight]
    %p_module_transformer_blocks_7_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_add_q_proj_bias]
    %p_module_transformer_blocks_7_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_out_0_weight]
    %p_module_transformer_blocks_7_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_out_0_bias]
    %p_module_transformer_blocks_7_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_add_out_weight]
    %p_module_transformer_blocks_7_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_to_add_out_bias]
    %p_module_transformer_blocks_7_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_norm_added_q_weight]
    %p_module_transformer_blocks_7_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_attn_norm_added_k_weight]
    %p_module_transformer_blocks_7_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_net_0_proj_weight]
    %p_module_transformer_blocks_7_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_net_0_proj_bias]
    %p_module_transformer_blocks_7_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_net_2_weight]
    %p_module_transformer_blocks_7_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_net_2_bias]
    %p_module_transformer_blocks_7_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_7_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_7_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_context_net_2_weight]
    %p_module_transformer_blocks_7_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_7_ff_context_net_2_bias]
    %p_module_transformer_blocks_8_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_norm1_linear_weight]
    %p_module_transformer_blocks_8_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_norm1_linear_bias]
    %p_module_transformer_blocks_8_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_norm1_context_linear_weight]
    %p_module_transformer_blocks_8_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_norm1_context_linear_bias]
    %p_module_transformer_blocks_8_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_norm_q_weight]
    %p_module_transformer_blocks_8_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_norm_k_weight]
    %p_module_transformer_blocks_8_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_q_weight]
    %p_module_transformer_blocks_8_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_q_bias]
    %p_module_transformer_blocks_8_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_k_weight]
    %p_module_transformer_blocks_8_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_k_bias]
    %p_module_transformer_blocks_8_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_v_weight]
    %p_module_transformer_blocks_8_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_v_bias]
    %p_module_transformer_blocks_8_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_add_k_proj_weight]
    %p_module_transformer_blocks_8_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_add_k_proj_bias]
    %p_module_transformer_blocks_8_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_add_v_proj_weight]
    %p_module_transformer_blocks_8_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_add_v_proj_bias]
    %p_module_transformer_blocks_8_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_add_q_proj_weight]
    %p_module_transformer_blocks_8_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_add_q_proj_bias]
    %p_module_transformer_blocks_8_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_out_0_weight]
    %p_module_transformer_blocks_8_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_out_0_bias]
    %p_module_transformer_blocks_8_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_add_out_weight]
    %p_module_transformer_blocks_8_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_to_add_out_bias]
    %p_module_transformer_blocks_8_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_norm_added_q_weight]
    %p_module_transformer_blocks_8_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_attn_norm_added_k_weight]
    %p_module_transformer_blocks_8_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_net_0_proj_weight]
    %p_module_transformer_blocks_8_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_net_0_proj_bias]
    %p_module_transformer_blocks_8_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_net_2_weight]
    %p_module_transformer_blocks_8_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_net_2_bias]
    %p_module_transformer_blocks_8_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_8_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_8_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_context_net_2_weight]
    %p_module_transformer_blocks_8_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_8_ff_context_net_2_bias]
    %p_module_transformer_blocks_9_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_norm1_linear_weight]
    %p_module_transformer_blocks_9_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_norm1_linear_bias]
    %p_module_transformer_blocks_9_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_norm1_context_linear_weight]
    %p_module_transformer_blocks_9_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_norm1_context_linear_bias]
    %p_module_transformer_blocks_9_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_norm_q_weight]
    %p_module_transformer_blocks_9_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_norm_k_weight]
    %p_module_transformer_blocks_9_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_q_weight]
    %p_module_transformer_blocks_9_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_q_bias]
    %p_module_transformer_blocks_9_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_k_weight]
    %p_module_transformer_blocks_9_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_k_bias]
    %p_module_transformer_blocks_9_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_v_weight]
    %p_module_transformer_blocks_9_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_v_bias]
    %p_module_transformer_blocks_9_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_add_k_proj_weight]
    %p_module_transformer_blocks_9_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_add_k_proj_bias]
    %p_module_transformer_blocks_9_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_add_v_proj_weight]
    %p_module_transformer_blocks_9_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_add_v_proj_bias]
    %p_module_transformer_blocks_9_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_add_q_proj_weight]
    %p_module_transformer_blocks_9_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_add_q_proj_bias]
    %p_module_transformer_blocks_9_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_out_0_weight]
    %p_module_transformer_blocks_9_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_out_0_bias]
    %p_module_transformer_blocks_9_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_add_out_weight]
    %p_module_transformer_blocks_9_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_to_add_out_bias]
    %p_module_transformer_blocks_9_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_norm_added_q_weight]
    %p_module_transformer_blocks_9_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_attn_norm_added_k_weight]
    %p_module_transformer_blocks_9_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_net_0_proj_weight]
    %p_module_transformer_blocks_9_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_net_0_proj_bias]
    %p_module_transformer_blocks_9_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_net_2_weight]
    %p_module_transformer_blocks_9_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_net_2_bias]
    %p_module_transformer_blocks_9_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_9_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_9_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_context_net_2_weight]
    %p_module_transformer_blocks_9_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_9_ff_context_net_2_bias]
    %p_module_transformer_blocks_10_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_norm1_linear_weight]
    %p_module_transformer_blocks_10_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_norm1_linear_bias]
    %p_module_transformer_blocks_10_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_norm1_context_linear_weight]
    %p_module_transformer_blocks_10_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_norm1_context_linear_bias]
    %p_module_transformer_blocks_10_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_norm_q_weight]
    %p_module_transformer_blocks_10_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_norm_k_weight]
    %p_module_transformer_blocks_10_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_q_weight]
    %p_module_transformer_blocks_10_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_q_bias]
    %p_module_transformer_blocks_10_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_k_weight]
    %p_module_transformer_blocks_10_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_k_bias]
    %p_module_transformer_blocks_10_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_v_weight]
    %p_module_transformer_blocks_10_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_v_bias]
    %p_module_transformer_blocks_10_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_add_k_proj_weight]
    %p_module_transformer_blocks_10_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_add_k_proj_bias]
    %p_module_transformer_blocks_10_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_add_v_proj_weight]
    %p_module_transformer_blocks_10_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_add_v_proj_bias]
    %p_module_transformer_blocks_10_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_add_q_proj_weight]
    %p_module_transformer_blocks_10_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_add_q_proj_bias]
    %p_module_transformer_blocks_10_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_out_0_weight]
    %p_module_transformer_blocks_10_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_out_0_bias]
    %p_module_transformer_blocks_10_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_add_out_weight]
    %p_module_transformer_blocks_10_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_to_add_out_bias]
    %p_module_transformer_blocks_10_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_norm_added_q_weight]
    %p_module_transformer_blocks_10_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_attn_norm_added_k_weight]
    %p_module_transformer_blocks_10_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_net_0_proj_weight]
    %p_module_transformer_blocks_10_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_net_0_proj_bias]
    %p_module_transformer_blocks_10_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_net_2_weight]
    %p_module_transformer_blocks_10_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_net_2_bias]
    %p_module_transformer_blocks_10_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_10_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_10_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_context_net_2_weight]
    %p_module_transformer_blocks_10_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_10_ff_context_net_2_bias]
    %p_module_transformer_blocks_11_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_norm1_linear_weight]
    %p_module_transformer_blocks_11_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_norm1_linear_bias]
    %p_module_transformer_blocks_11_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_norm1_context_linear_weight]
    %p_module_transformer_blocks_11_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_norm1_context_linear_bias]
    %p_module_transformer_blocks_11_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_norm_q_weight]
    %p_module_transformer_blocks_11_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_norm_k_weight]
    %p_module_transformer_blocks_11_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_q_weight]
    %p_module_transformer_blocks_11_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_q_bias]
    %p_module_transformer_blocks_11_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_k_weight]
    %p_module_transformer_blocks_11_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_k_bias]
    %p_module_transformer_blocks_11_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_v_weight]
    %p_module_transformer_blocks_11_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_v_bias]
    %p_module_transformer_blocks_11_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_add_k_proj_weight]
    %p_module_transformer_blocks_11_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_add_k_proj_bias]
    %p_module_transformer_blocks_11_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_add_v_proj_weight]
    %p_module_transformer_blocks_11_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_add_v_proj_bias]
    %p_module_transformer_blocks_11_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_add_q_proj_weight]
    %p_module_transformer_blocks_11_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_add_q_proj_bias]
    %p_module_transformer_blocks_11_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_out_0_weight]
    %p_module_transformer_blocks_11_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_out_0_bias]
    %p_module_transformer_blocks_11_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_add_out_weight]
    %p_module_transformer_blocks_11_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_to_add_out_bias]
    %p_module_transformer_blocks_11_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_norm_added_q_weight]
    %p_module_transformer_blocks_11_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_attn_norm_added_k_weight]
    %p_module_transformer_blocks_11_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_net_0_proj_weight]
    %p_module_transformer_blocks_11_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_net_0_proj_bias]
    %p_module_transformer_blocks_11_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_net_2_weight]
    %p_module_transformer_blocks_11_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_net_2_bias]
    %p_module_transformer_blocks_11_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_11_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_11_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_context_net_2_weight]
    %p_module_transformer_blocks_11_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_11_ff_context_net_2_bias]
    %p_module_transformer_blocks_12_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_norm1_linear_weight]
    %p_module_transformer_blocks_12_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_norm1_linear_bias]
    %p_module_transformer_blocks_12_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_norm1_context_linear_weight]
    %p_module_transformer_blocks_12_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_norm1_context_linear_bias]
    %p_module_transformer_blocks_12_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_norm_q_weight]
    %p_module_transformer_blocks_12_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_norm_k_weight]
    %p_module_transformer_blocks_12_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_q_weight]
    %p_module_transformer_blocks_12_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_q_bias]
    %p_module_transformer_blocks_12_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_k_weight]
    %p_module_transformer_blocks_12_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_k_bias]
    %p_module_transformer_blocks_12_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_v_weight]
    %p_module_transformer_blocks_12_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_v_bias]
    %p_module_transformer_blocks_12_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_add_k_proj_weight]
    %p_module_transformer_blocks_12_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_add_k_proj_bias]
    %p_module_transformer_blocks_12_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_add_v_proj_weight]
    %p_module_transformer_blocks_12_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_add_v_proj_bias]
    %p_module_transformer_blocks_12_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_add_q_proj_weight]
    %p_module_transformer_blocks_12_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_add_q_proj_bias]
    %p_module_transformer_blocks_12_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_out_0_weight]
    %p_module_transformer_blocks_12_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_out_0_bias]
    %p_module_transformer_blocks_12_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_add_out_weight]
    %p_module_transformer_blocks_12_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_to_add_out_bias]
    %p_module_transformer_blocks_12_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_norm_added_q_weight]
    %p_module_transformer_blocks_12_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_attn_norm_added_k_weight]
    %p_module_transformer_blocks_12_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_net_0_proj_weight]
    %p_module_transformer_blocks_12_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_net_0_proj_bias]
    %p_module_transformer_blocks_12_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_net_2_weight]
    %p_module_transformer_blocks_12_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_net_2_bias]
    %p_module_transformer_blocks_12_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_12_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_12_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_context_net_2_weight]
    %p_module_transformer_blocks_12_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_12_ff_context_net_2_bias]
    %p_module_transformer_blocks_13_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_norm1_linear_weight]
    %p_module_transformer_blocks_13_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_norm1_linear_bias]
    %p_module_transformer_blocks_13_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_norm1_context_linear_weight]
    %p_module_transformer_blocks_13_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_norm1_context_linear_bias]
    %p_module_transformer_blocks_13_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_norm_q_weight]
    %p_module_transformer_blocks_13_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_norm_k_weight]
    %p_module_transformer_blocks_13_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_q_weight]
    %p_module_transformer_blocks_13_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_q_bias]
    %p_module_transformer_blocks_13_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_k_weight]
    %p_module_transformer_blocks_13_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_k_bias]
    %p_module_transformer_blocks_13_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_v_weight]
    %p_module_transformer_blocks_13_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_v_bias]
    %p_module_transformer_blocks_13_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_add_k_proj_weight]
    %p_module_transformer_blocks_13_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_add_k_proj_bias]
    %p_module_transformer_blocks_13_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_add_v_proj_weight]
    %p_module_transformer_blocks_13_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_add_v_proj_bias]
    %p_module_transformer_blocks_13_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_add_q_proj_weight]
    %p_module_transformer_blocks_13_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_add_q_proj_bias]
    %p_module_transformer_blocks_13_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_out_0_weight]
    %p_module_transformer_blocks_13_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_out_0_bias]
    %p_module_transformer_blocks_13_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_add_out_weight]
    %p_module_transformer_blocks_13_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_to_add_out_bias]
    %p_module_transformer_blocks_13_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_norm_added_q_weight]
    %p_module_transformer_blocks_13_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_attn_norm_added_k_weight]
    %p_module_transformer_blocks_13_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_net_0_proj_weight]
    %p_module_transformer_blocks_13_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_net_0_proj_bias]
    %p_module_transformer_blocks_13_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_net_2_weight]
    %p_module_transformer_blocks_13_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_net_2_bias]
    %p_module_transformer_blocks_13_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_13_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_13_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_context_net_2_weight]
    %p_module_transformer_blocks_13_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_13_ff_context_net_2_bias]
    %p_module_transformer_blocks_14_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_norm1_linear_weight]
    %p_module_transformer_blocks_14_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_norm1_linear_bias]
    %p_module_transformer_blocks_14_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_norm1_context_linear_weight]
    %p_module_transformer_blocks_14_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_norm1_context_linear_bias]
    %p_module_transformer_blocks_14_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_norm_q_weight]
    %p_module_transformer_blocks_14_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_norm_k_weight]
    %p_module_transformer_blocks_14_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_q_weight]
    %p_module_transformer_blocks_14_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_q_bias]
    %p_module_transformer_blocks_14_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_k_weight]
    %p_module_transformer_blocks_14_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_k_bias]
    %p_module_transformer_blocks_14_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_v_weight]
    %p_module_transformer_blocks_14_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_v_bias]
    %p_module_transformer_blocks_14_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_add_k_proj_weight]
    %p_module_transformer_blocks_14_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_add_k_proj_bias]
    %p_module_transformer_blocks_14_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_add_v_proj_weight]
    %p_module_transformer_blocks_14_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_add_v_proj_bias]
    %p_module_transformer_blocks_14_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_add_q_proj_weight]
    %p_module_transformer_blocks_14_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_add_q_proj_bias]
    %p_module_transformer_blocks_14_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_out_0_weight]
    %p_module_transformer_blocks_14_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_out_0_bias]
    %p_module_transformer_blocks_14_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_add_out_weight]
    %p_module_transformer_blocks_14_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_to_add_out_bias]
    %p_module_transformer_blocks_14_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_norm_added_q_weight]
    %p_module_transformer_blocks_14_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_attn_norm_added_k_weight]
    %p_module_transformer_blocks_14_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_net_0_proj_weight]
    %p_module_transformer_blocks_14_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_net_0_proj_bias]
    %p_module_transformer_blocks_14_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_net_2_weight]
    %p_module_transformer_blocks_14_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_net_2_bias]
    %p_module_transformer_blocks_14_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_14_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_14_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_context_net_2_weight]
    %p_module_transformer_blocks_14_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_14_ff_context_net_2_bias]
    %p_module_transformer_blocks_15_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_norm1_linear_weight]
    %p_module_transformer_blocks_15_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_norm1_linear_bias]
    %p_module_transformer_blocks_15_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_norm1_context_linear_weight]
    %p_module_transformer_blocks_15_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_norm1_context_linear_bias]
    %p_module_transformer_blocks_15_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_norm_q_weight]
    %p_module_transformer_blocks_15_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_norm_k_weight]
    %p_module_transformer_blocks_15_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_q_weight]
    %p_module_transformer_blocks_15_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_q_bias]
    %p_module_transformer_blocks_15_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_k_weight]
    %p_module_transformer_blocks_15_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_k_bias]
    %p_module_transformer_blocks_15_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_v_weight]
    %p_module_transformer_blocks_15_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_v_bias]
    %p_module_transformer_blocks_15_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_add_k_proj_weight]
    %p_module_transformer_blocks_15_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_add_k_proj_bias]
    %p_module_transformer_blocks_15_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_add_v_proj_weight]
    %p_module_transformer_blocks_15_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_add_v_proj_bias]
    %p_module_transformer_blocks_15_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_add_q_proj_weight]
    %p_module_transformer_blocks_15_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_add_q_proj_bias]
    %p_module_transformer_blocks_15_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_out_0_weight]
    %p_module_transformer_blocks_15_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_out_0_bias]
    %p_module_transformer_blocks_15_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_add_out_weight]
    %p_module_transformer_blocks_15_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_to_add_out_bias]
    %p_module_transformer_blocks_15_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_norm_added_q_weight]
    %p_module_transformer_blocks_15_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_attn_norm_added_k_weight]
    %p_module_transformer_blocks_15_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_net_0_proj_weight]
    %p_module_transformer_blocks_15_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_net_0_proj_bias]
    %p_module_transformer_blocks_15_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_net_2_weight]
    %p_module_transformer_blocks_15_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_net_2_bias]
    %p_module_transformer_blocks_15_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_15_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_15_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_context_net_2_weight]
    %p_module_transformer_blocks_15_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_15_ff_context_net_2_bias]
    %p_module_transformer_blocks_16_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_norm1_linear_weight]
    %p_module_transformer_blocks_16_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_norm1_linear_bias]
    %p_module_transformer_blocks_16_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_norm1_context_linear_weight]
    %p_module_transformer_blocks_16_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_norm1_context_linear_bias]
    %p_module_transformer_blocks_16_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_norm_q_weight]
    %p_module_transformer_blocks_16_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_norm_k_weight]
    %p_module_transformer_blocks_16_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_q_weight]
    %p_module_transformer_blocks_16_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_q_bias]
    %p_module_transformer_blocks_16_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_k_weight]
    %p_module_transformer_blocks_16_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_k_bias]
    %p_module_transformer_blocks_16_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_v_weight]
    %p_module_transformer_blocks_16_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_v_bias]
    %p_module_transformer_blocks_16_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_add_k_proj_weight]
    %p_module_transformer_blocks_16_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_add_k_proj_bias]
    %p_module_transformer_blocks_16_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_add_v_proj_weight]
    %p_module_transformer_blocks_16_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_add_v_proj_bias]
    %p_module_transformer_blocks_16_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_add_q_proj_weight]
    %p_module_transformer_blocks_16_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_add_q_proj_bias]
    %p_module_transformer_blocks_16_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_out_0_weight]
    %p_module_transformer_blocks_16_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_out_0_bias]
    %p_module_transformer_blocks_16_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_add_out_weight]
    %p_module_transformer_blocks_16_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_to_add_out_bias]
    %p_module_transformer_blocks_16_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_norm_added_q_weight]
    %p_module_transformer_blocks_16_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_attn_norm_added_k_weight]
    %p_module_transformer_blocks_16_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_net_0_proj_weight]
    %p_module_transformer_blocks_16_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_net_0_proj_bias]
    %p_module_transformer_blocks_16_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_net_2_weight]
    %p_module_transformer_blocks_16_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_net_2_bias]
    %p_module_transformer_blocks_16_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_16_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_16_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_context_net_2_weight]
    %p_module_transformer_blocks_16_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_16_ff_context_net_2_bias]
    %p_module_transformer_blocks_17_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_norm1_linear_weight]
    %p_module_transformer_blocks_17_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_norm1_linear_bias]
    %p_module_transformer_blocks_17_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_norm1_context_linear_weight]
    %p_module_transformer_blocks_17_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_norm1_context_linear_bias]
    %p_module_transformer_blocks_17_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_norm_q_weight]
    %p_module_transformer_blocks_17_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_norm_k_weight]
    %p_module_transformer_blocks_17_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_q_weight]
    %p_module_transformer_blocks_17_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_q_bias]
    %p_module_transformer_blocks_17_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_k_weight]
    %p_module_transformer_blocks_17_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_k_bias]
    %p_module_transformer_blocks_17_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_v_weight]
    %p_module_transformer_blocks_17_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_v_bias]
    %p_module_transformer_blocks_17_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_add_k_proj_weight]
    %p_module_transformer_blocks_17_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_add_k_proj_bias]
    %p_module_transformer_blocks_17_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_add_v_proj_weight]
    %p_module_transformer_blocks_17_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_add_v_proj_bias]
    %p_module_transformer_blocks_17_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_add_q_proj_weight]
    %p_module_transformer_blocks_17_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_add_q_proj_bias]
    %p_module_transformer_blocks_17_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_out_0_weight]
    %p_module_transformer_blocks_17_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_out_0_bias]
    %p_module_transformer_blocks_17_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_add_out_weight]
    %p_module_transformer_blocks_17_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_to_add_out_bias]
    %p_module_transformer_blocks_17_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_norm_added_q_weight]
    %p_module_transformer_blocks_17_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_attn_norm_added_k_weight]
    %p_module_transformer_blocks_17_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_net_0_proj_weight]
    %p_module_transformer_blocks_17_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_net_0_proj_bias]
    %p_module_transformer_blocks_17_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_net_2_weight]
    %p_module_transformer_blocks_17_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_net_2_bias]
    %p_module_transformer_blocks_17_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_17_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_17_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_context_net_2_weight]
    %p_module_transformer_blocks_17_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_17_ff_context_net_2_bias]
    %p_module_transformer_blocks_18_norm1_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_norm1_linear_weight]
    %p_module_transformer_blocks_18_norm1_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_norm1_linear_bias]
    %p_module_transformer_blocks_18_norm1_context_linear_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_norm1_context_linear_weight]
    %p_module_transformer_blocks_18_norm1_context_linear_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_norm1_context_linear_bias]
    %p_module_transformer_blocks_18_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_norm_q_weight]
    %p_module_transformer_blocks_18_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_norm_k_weight]
    %p_module_transformer_blocks_18_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_q_weight]
    %p_module_transformer_blocks_18_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_q_bias]
    %p_module_transformer_blocks_18_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_k_weight]
    %p_module_transformer_blocks_18_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_k_bias]
    %p_module_transformer_blocks_18_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_v_weight]
    %p_module_transformer_blocks_18_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_v_bias]
    %p_module_transformer_blocks_18_attn_add_k_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_add_k_proj_weight]
    %p_module_transformer_blocks_18_attn_add_k_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_add_k_proj_bias]
    %p_module_transformer_blocks_18_attn_add_v_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_add_v_proj_weight]
    %p_module_transformer_blocks_18_attn_add_v_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_add_v_proj_bias]
    %p_module_transformer_blocks_18_attn_add_q_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_add_q_proj_weight]
    %p_module_transformer_blocks_18_attn_add_q_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_add_q_proj_bias]
    %p_module_transformer_blocks_18_attn_to_out_0_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_out_0_weight]
    %p_module_transformer_blocks_18_attn_to_out_0_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_out_0_bias]
    %p_module_transformer_blocks_18_attn_to_add_out_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_add_out_weight]
    %p_module_transformer_blocks_18_attn_to_add_out_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_to_add_out_bias]
    %p_module_transformer_blocks_18_attn_norm_added_q_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_norm_added_q_weight]
    %p_module_transformer_blocks_18_attn_norm_added_k_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_attn_norm_added_k_weight]
    %p_module_transformer_blocks_18_ff_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_net_0_proj_weight]
    %p_module_transformer_blocks_18_ff_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_net_0_proj_bias]
    %p_module_transformer_blocks_18_ff_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_net_2_weight]
    %p_module_transformer_blocks_18_ff_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_net_2_bias]
    %p_module_transformer_blocks_18_ff_context_net_0_proj_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_context_net_0_proj_weight]
    %p_module_transformer_blocks_18_ff_context_net_0_proj_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_context_net_0_proj_bias]
    %p_module_transformer_blocks_18_ff_context_net_2_weight : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_context_net_2_weight]
    %p_module_transformer_blocks_18_ff_context_net_2_bias : [num_users=0] = placeholder[target=p_module_transformer_blocks_18_ff_context_net_2_bias]
    %p_module_single_transformer_blocks_0_norm_linear_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_norm_linear_weight]
    %p_module_single_transformer_blocks_0_norm_linear_bias : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_norm_linear_bias]
    %p_module_single_transformer_blocks_0_proj_mlp_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_proj_mlp_weight]
    %p_module_single_transformer_blocks_0_proj_mlp_bias : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_proj_mlp_bias]
    %p_module_single_transformer_blocks_0_proj_out_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_proj_out_weight]
    %p_module_single_transformer_blocks_0_proj_out_bias : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_proj_out_bias]
    %p_module_single_transformer_blocks_0_attn_norm_q_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_norm_q_weight]
    %p_module_single_transformer_blocks_0_attn_norm_k_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_norm_k_weight]
    %p_module_single_transformer_blocks_0_attn_to_q_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_to_q_weight]
    %p_module_single_transformer_blocks_0_attn_to_q_bias : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_to_q_bias]
    %p_module_single_transformer_blocks_0_attn_to_k_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_to_k_weight]
    %p_module_single_transformer_blocks_0_attn_to_k_bias : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_to_k_bias]
    %p_module_single_transformer_blocks_0_attn_to_v_weight : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_to_v_weight]
    %p_module_single_transformer_blocks_0_attn_to_v_bias : [num_users=1] = placeholder[target=p_module_single_transformer_blocks_0_attn_to_v_bias]
    %p_module_single_transformer_blocks_1_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_norm_linear_weight]
    %p_module_single_transformer_blocks_1_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_norm_linear_bias]
    %p_module_single_transformer_blocks_1_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_proj_mlp_weight]
    %p_module_single_transformer_blocks_1_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_proj_mlp_bias]
    %p_module_single_transformer_blocks_1_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_proj_out_weight]
    %p_module_single_transformer_blocks_1_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_proj_out_bias]
    %p_module_single_transformer_blocks_1_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_norm_q_weight]
    %p_module_single_transformer_blocks_1_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_norm_k_weight]
    %p_module_single_transformer_blocks_1_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_to_q_weight]
    %p_module_single_transformer_blocks_1_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_to_q_bias]
    %p_module_single_transformer_blocks_1_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_to_k_weight]
    %p_module_single_transformer_blocks_1_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_to_k_bias]
    %p_module_single_transformer_blocks_1_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_to_v_weight]
    %p_module_single_transformer_blocks_1_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_1_attn_to_v_bias]
    %p_module_single_transformer_blocks_2_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_norm_linear_weight]
    %p_module_single_transformer_blocks_2_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_norm_linear_bias]
    %p_module_single_transformer_blocks_2_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_proj_mlp_weight]
    %p_module_single_transformer_blocks_2_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_proj_mlp_bias]
    %p_module_single_transformer_blocks_2_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_proj_out_weight]
    %p_module_single_transformer_blocks_2_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_proj_out_bias]
    %p_module_single_transformer_blocks_2_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_norm_q_weight]
    %p_module_single_transformer_blocks_2_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_norm_k_weight]
    %p_module_single_transformer_blocks_2_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_to_q_weight]
    %p_module_single_transformer_blocks_2_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_to_q_bias]
    %p_module_single_transformer_blocks_2_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_to_k_weight]
    %p_module_single_transformer_blocks_2_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_to_k_bias]
    %p_module_single_transformer_blocks_2_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_to_v_weight]
    %p_module_single_transformer_blocks_2_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_2_attn_to_v_bias]
    %p_module_single_transformer_blocks_3_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_norm_linear_weight]
    %p_module_single_transformer_blocks_3_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_norm_linear_bias]
    %p_module_single_transformer_blocks_3_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_proj_mlp_weight]
    %p_module_single_transformer_blocks_3_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_proj_mlp_bias]
    %p_module_single_transformer_blocks_3_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_proj_out_weight]
    %p_module_single_transformer_blocks_3_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_proj_out_bias]
    %p_module_single_transformer_blocks_3_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_norm_q_weight]
    %p_module_single_transformer_blocks_3_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_norm_k_weight]
    %p_module_single_transformer_blocks_3_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_to_q_weight]
    %p_module_single_transformer_blocks_3_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_to_q_bias]
    %p_module_single_transformer_blocks_3_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_to_k_weight]
    %p_module_single_transformer_blocks_3_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_to_k_bias]
    %p_module_single_transformer_blocks_3_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_to_v_weight]
    %p_module_single_transformer_blocks_3_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_3_attn_to_v_bias]
    %p_module_single_transformer_blocks_4_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_norm_linear_weight]
    %p_module_single_transformer_blocks_4_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_norm_linear_bias]
    %p_module_single_transformer_blocks_4_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_proj_mlp_weight]
    %p_module_single_transformer_blocks_4_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_proj_mlp_bias]
    %p_module_single_transformer_blocks_4_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_proj_out_weight]
    %p_module_single_transformer_blocks_4_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_proj_out_bias]
    %p_module_single_transformer_blocks_4_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_norm_q_weight]
    %p_module_single_transformer_blocks_4_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_norm_k_weight]
    %p_module_single_transformer_blocks_4_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_to_q_weight]
    %p_module_single_transformer_blocks_4_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_to_q_bias]
    %p_module_single_transformer_blocks_4_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_to_k_weight]
    %p_module_single_transformer_blocks_4_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_to_k_bias]
    %p_module_single_transformer_blocks_4_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_to_v_weight]
    %p_module_single_transformer_blocks_4_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_4_attn_to_v_bias]
    %p_module_single_transformer_blocks_5_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_norm_linear_weight]
    %p_module_single_transformer_blocks_5_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_norm_linear_bias]
    %p_module_single_transformer_blocks_5_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_proj_mlp_weight]
    %p_module_single_transformer_blocks_5_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_proj_mlp_bias]
    %p_module_single_transformer_blocks_5_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_proj_out_weight]
    %p_module_single_transformer_blocks_5_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_proj_out_bias]
    %p_module_single_transformer_blocks_5_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_norm_q_weight]
    %p_module_single_transformer_blocks_5_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_norm_k_weight]
    %p_module_single_transformer_blocks_5_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_to_q_weight]
    %p_module_single_transformer_blocks_5_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_to_q_bias]
    %p_module_single_transformer_blocks_5_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_to_k_weight]
    %p_module_single_transformer_blocks_5_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_to_k_bias]
    %p_module_single_transformer_blocks_5_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_to_v_weight]
    %p_module_single_transformer_blocks_5_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_5_attn_to_v_bias]
    %p_module_single_transformer_blocks_6_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_norm_linear_weight]
    %p_module_single_transformer_blocks_6_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_norm_linear_bias]
    %p_module_single_transformer_blocks_6_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_proj_mlp_weight]
    %p_module_single_transformer_blocks_6_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_proj_mlp_bias]
    %p_module_single_transformer_blocks_6_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_proj_out_weight]
    %p_module_single_transformer_blocks_6_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_proj_out_bias]
    %p_module_single_transformer_blocks_6_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_norm_q_weight]
    %p_module_single_transformer_blocks_6_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_norm_k_weight]
    %p_module_single_transformer_blocks_6_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_to_q_weight]
    %p_module_single_transformer_blocks_6_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_to_q_bias]
    %p_module_single_transformer_blocks_6_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_to_k_weight]
    %p_module_single_transformer_blocks_6_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_to_k_bias]
    %p_module_single_transformer_blocks_6_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_to_v_weight]
    %p_module_single_transformer_blocks_6_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_6_attn_to_v_bias]
    %p_module_single_transformer_blocks_7_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_norm_linear_weight]
    %p_module_single_transformer_blocks_7_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_norm_linear_bias]
    %p_module_single_transformer_blocks_7_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_proj_mlp_weight]
    %p_module_single_transformer_blocks_7_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_proj_mlp_bias]
    %p_module_single_transformer_blocks_7_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_proj_out_weight]
    %p_module_single_transformer_blocks_7_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_proj_out_bias]
    %p_module_single_transformer_blocks_7_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_norm_q_weight]
    %p_module_single_transformer_blocks_7_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_norm_k_weight]
    %p_module_single_transformer_blocks_7_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_to_q_weight]
    %p_module_single_transformer_blocks_7_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_to_q_bias]
    %p_module_single_transformer_blocks_7_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_to_k_weight]
    %p_module_single_transformer_blocks_7_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_to_k_bias]
    %p_module_single_transformer_blocks_7_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_to_v_weight]
    %p_module_single_transformer_blocks_7_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_7_attn_to_v_bias]
    %p_module_single_transformer_blocks_8_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_norm_linear_weight]
    %p_module_single_transformer_blocks_8_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_norm_linear_bias]
    %p_module_single_transformer_blocks_8_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_proj_mlp_weight]
    %p_module_single_transformer_blocks_8_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_proj_mlp_bias]
    %p_module_single_transformer_blocks_8_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_proj_out_weight]
    %p_module_single_transformer_blocks_8_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_proj_out_bias]
    %p_module_single_transformer_blocks_8_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_norm_q_weight]
    %p_module_single_transformer_blocks_8_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_norm_k_weight]
    %p_module_single_transformer_blocks_8_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_to_q_weight]
    %p_module_single_transformer_blocks_8_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_to_q_bias]
    %p_module_single_transformer_blocks_8_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_to_k_weight]
    %p_module_single_transformer_blocks_8_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_to_k_bias]
    %p_module_single_transformer_blocks_8_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_to_v_weight]
    %p_module_single_transformer_blocks_8_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_8_attn_to_v_bias]
    %p_module_single_transformer_blocks_9_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_norm_linear_weight]
    %p_module_single_transformer_blocks_9_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_norm_linear_bias]
    %p_module_single_transformer_blocks_9_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_proj_mlp_weight]
    %p_module_single_transformer_blocks_9_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_proj_mlp_bias]
    %p_module_single_transformer_blocks_9_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_proj_out_weight]
    %p_module_single_transformer_blocks_9_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_proj_out_bias]
    %p_module_single_transformer_blocks_9_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_norm_q_weight]
    %p_module_single_transformer_blocks_9_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_norm_k_weight]
    %p_module_single_transformer_blocks_9_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_to_q_weight]
    %p_module_single_transformer_blocks_9_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_to_q_bias]
    %p_module_single_transformer_blocks_9_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_to_k_weight]
    %p_module_single_transformer_blocks_9_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_to_k_bias]
    %p_module_single_transformer_blocks_9_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_to_v_weight]
    %p_module_single_transformer_blocks_9_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_9_attn_to_v_bias]
    %p_module_single_transformer_blocks_10_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_norm_linear_weight]
    %p_module_single_transformer_blocks_10_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_norm_linear_bias]
    %p_module_single_transformer_blocks_10_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_proj_mlp_weight]
    %p_module_single_transformer_blocks_10_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_proj_mlp_bias]
    %p_module_single_transformer_blocks_10_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_proj_out_weight]
    %p_module_single_transformer_blocks_10_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_proj_out_bias]
    %p_module_single_transformer_blocks_10_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_norm_q_weight]
    %p_module_single_transformer_blocks_10_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_norm_k_weight]
    %p_module_single_transformer_blocks_10_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_to_q_weight]
    %p_module_single_transformer_blocks_10_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_to_q_bias]
    %p_module_single_transformer_blocks_10_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_to_k_weight]
    %p_module_single_transformer_blocks_10_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_to_k_bias]
    %p_module_single_transformer_blocks_10_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_to_v_weight]
    %p_module_single_transformer_blocks_10_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_10_attn_to_v_bias]
    %p_module_single_transformer_blocks_11_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_norm_linear_weight]
    %p_module_single_transformer_blocks_11_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_norm_linear_bias]
    %p_module_single_transformer_blocks_11_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_proj_mlp_weight]
    %p_module_single_transformer_blocks_11_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_proj_mlp_bias]
    %p_module_single_transformer_blocks_11_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_proj_out_weight]
    %p_module_single_transformer_blocks_11_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_proj_out_bias]
    %p_module_single_transformer_blocks_11_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_norm_q_weight]
    %p_module_single_transformer_blocks_11_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_norm_k_weight]
    %p_module_single_transformer_blocks_11_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_to_q_weight]
    %p_module_single_transformer_blocks_11_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_to_q_bias]
    %p_module_single_transformer_blocks_11_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_to_k_weight]
    %p_module_single_transformer_blocks_11_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_to_k_bias]
    %p_module_single_transformer_blocks_11_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_to_v_weight]
    %p_module_single_transformer_blocks_11_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_11_attn_to_v_bias]
    %p_module_single_transformer_blocks_12_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_norm_linear_weight]
    %p_module_single_transformer_blocks_12_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_norm_linear_bias]
    %p_module_single_transformer_blocks_12_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_proj_mlp_weight]
    %p_module_single_transformer_blocks_12_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_proj_mlp_bias]
    %p_module_single_transformer_blocks_12_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_proj_out_weight]
    %p_module_single_transformer_blocks_12_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_proj_out_bias]
    %p_module_single_transformer_blocks_12_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_norm_q_weight]
    %p_module_single_transformer_blocks_12_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_norm_k_weight]
    %p_module_single_transformer_blocks_12_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_to_q_weight]
    %p_module_single_transformer_blocks_12_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_to_q_bias]
    %p_module_single_transformer_blocks_12_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_to_k_weight]
    %p_module_single_transformer_blocks_12_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_to_k_bias]
    %p_module_single_transformer_blocks_12_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_to_v_weight]
    %p_module_single_transformer_blocks_12_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_12_attn_to_v_bias]
    %p_module_single_transformer_blocks_13_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_norm_linear_weight]
    %p_module_single_transformer_blocks_13_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_norm_linear_bias]
    %p_module_single_transformer_blocks_13_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_proj_mlp_weight]
    %p_module_single_transformer_blocks_13_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_proj_mlp_bias]
    %p_module_single_transformer_blocks_13_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_proj_out_weight]
    %p_module_single_transformer_blocks_13_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_proj_out_bias]
    %p_module_single_transformer_blocks_13_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_norm_q_weight]
    %p_module_single_transformer_blocks_13_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_norm_k_weight]
    %p_module_single_transformer_blocks_13_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_to_q_weight]
    %p_module_single_transformer_blocks_13_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_to_q_bias]
    %p_module_single_transformer_blocks_13_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_to_k_weight]
    %p_module_single_transformer_blocks_13_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_to_k_bias]
    %p_module_single_transformer_blocks_13_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_to_v_weight]
    %p_module_single_transformer_blocks_13_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_13_attn_to_v_bias]
    %p_module_single_transformer_blocks_14_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_norm_linear_weight]
    %p_module_single_transformer_blocks_14_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_norm_linear_bias]
    %p_module_single_transformer_blocks_14_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_proj_mlp_weight]
    %p_module_single_transformer_blocks_14_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_proj_mlp_bias]
    %p_module_single_transformer_blocks_14_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_proj_out_weight]
    %p_module_single_transformer_blocks_14_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_proj_out_bias]
    %p_module_single_transformer_blocks_14_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_norm_q_weight]
    %p_module_single_transformer_blocks_14_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_norm_k_weight]
    %p_module_single_transformer_blocks_14_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_to_q_weight]
    %p_module_single_transformer_blocks_14_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_to_q_bias]
    %p_module_single_transformer_blocks_14_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_to_k_weight]
    %p_module_single_transformer_blocks_14_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_to_k_bias]
    %p_module_single_transformer_blocks_14_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_to_v_weight]
    %p_module_single_transformer_blocks_14_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_14_attn_to_v_bias]
    %p_module_single_transformer_blocks_15_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_norm_linear_weight]
    %p_module_single_transformer_blocks_15_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_norm_linear_bias]
    %p_module_single_transformer_blocks_15_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_proj_mlp_weight]
    %p_module_single_transformer_blocks_15_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_proj_mlp_bias]
    %p_module_single_transformer_blocks_15_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_proj_out_weight]
    %p_module_single_transformer_blocks_15_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_proj_out_bias]
    %p_module_single_transformer_blocks_15_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_norm_q_weight]
    %p_module_single_transformer_blocks_15_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_norm_k_weight]
    %p_module_single_transformer_blocks_15_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_to_q_weight]
    %p_module_single_transformer_blocks_15_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_to_q_bias]
    %p_module_single_transformer_blocks_15_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_to_k_weight]
    %p_module_single_transformer_blocks_15_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_to_k_bias]
    %p_module_single_transformer_blocks_15_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_to_v_weight]
    %p_module_single_transformer_blocks_15_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_15_attn_to_v_bias]
    %p_module_single_transformer_blocks_16_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_norm_linear_weight]
    %p_module_single_transformer_blocks_16_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_norm_linear_bias]
    %p_module_single_transformer_blocks_16_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_proj_mlp_weight]
    %p_module_single_transformer_blocks_16_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_proj_mlp_bias]
    %p_module_single_transformer_blocks_16_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_proj_out_weight]
    %p_module_single_transformer_blocks_16_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_proj_out_bias]
    %p_module_single_transformer_blocks_16_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_norm_q_weight]
    %p_module_single_transformer_blocks_16_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_norm_k_weight]
    %p_module_single_transformer_blocks_16_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_to_q_weight]
    %p_module_single_transformer_blocks_16_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_to_q_bias]
    %p_module_single_transformer_blocks_16_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_to_k_weight]
    %p_module_single_transformer_blocks_16_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_to_k_bias]
    %p_module_single_transformer_blocks_16_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_to_v_weight]
    %p_module_single_transformer_blocks_16_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_16_attn_to_v_bias]
    %p_module_single_transformer_blocks_17_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_norm_linear_weight]
    %p_module_single_transformer_blocks_17_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_norm_linear_bias]
    %p_module_single_transformer_blocks_17_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_proj_mlp_weight]
    %p_module_single_transformer_blocks_17_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_proj_mlp_bias]
    %p_module_single_transformer_blocks_17_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_proj_out_weight]
    %p_module_single_transformer_blocks_17_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_proj_out_bias]
    %p_module_single_transformer_blocks_17_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_norm_q_weight]
    %p_module_single_transformer_blocks_17_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_norm_k_weight]
    %p_module_single_transformer_blocks_17_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_to_q_weight]
    %p_module_single_transformer_blocks_17_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_to_q_bias]
    %p_module_single_transformer_blocks_17_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_to_k_weight]
    %p_module_single_transformer_blocks_17_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_to_k_bias]
    %p_module_single_transformer_blocks_17_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_to_v_weight]
    %p_module_single_transformer_blocks_17_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_17_attn_to_v_bias]
    %p_module_single_transformer_blocks_18_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_norm_linear_weight]
    %p_module_single_transformer_blocks_18_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_norm_linear_bias]
    %p_module_single_transformer_blocks_18_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_proj_mlp_weight]
    %p_module_single_transformer_blocks_18_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_proj_mlp_bias]
    %p_module_single_transformer_blocks_18_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_proj_out_weight]
    %p_module_single_transformer_blocks_18_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_proj_out_bias]
    %p_module_single_transformer_blocks_18_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_norm_q_weight]
    %p_module_single_transformer_blocks_18_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_norm_k_weight]
    %p_module_single_transformer_blocks_18_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_to_q_weight]
    %p_module_single_transformer_blocks_18_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_to_q_bias]
    %p_module_single_transformer_blocks_18_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_to_k_weight]
    %p_module_single_transformer_blocks_18_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_to_k_bias]
    %p_module_single_transformer_blocks_18_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_to_v_weight]
    %p_module_single_transformer_blocks_18_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_18_attn_to_v_bias]
    %p_module_single_transformer_blocks_19_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_norm_linear_weight]
    %p_module_single_transformer_blocks_19_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_norm_linear_bias]
    %p_module_single_transformer_blocks_19_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_proj_mlp_weight]
    %p_module_single_transformer_blocks_19_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_proj_mlp_bias]
    %p_module_single_transformer_blocks_19_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_proj_out_weight]
    %p_module_single_transformer_blocks_19_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_proj_out_bias]
    %p_module_single_transformer_blocks_19_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_norm_q_weight]
    %p_module_single_transformer_blocks_19_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_norm_k_weight]
    %p_module_single_transformer_blocks_19_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_to_q_weight]
    %p_module_single_transformer_blocks_19_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_to_q_bias]
    %p_module_single_transformer_blocks_19_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_to_k_weight]
    %p_module_single_transformer_blocks_19_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_to_k_bias]
    %p_module_single_transformer_blocks_19_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_to_v_weight]
    %p_module_single_transformer_blocks_19_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_19_attn_to_v_bias]
    %p_module_single_transformer_blocks_20_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_norm_linear_weight]
    %p_module_single_transformer_blocks_20_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_norm_linear_bias]
    %p_module_single_transformer_blocks_20_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_proj_mlp_weight]
    %p_module_single_transformer_blocks_20_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_proj_mlp_bias]
    %p_module_single_transformer_blocks_20_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_proj_out_weight]
    %p_module_single_transformer_blocks_20_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_proj_out_bias]
    %p_module_single_transformer_blocks_20_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_norm_q_weight]
    %p_module_single_transformer_blocks_20_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_norm_k_weight]
    %p_module_single_transformer_blocks_20_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_to_q_weight]
    %p_module_single_transformer_blocks_20_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_to_q_bias]
    %p_module_single_transformer_blocks_20_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_to_k_weight]
    %p_module_single_transformer_blocks_20_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_to_k_bias]
    %p_module_single_transformer_blocks_20_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_to_v_weight]
    %p_module_single_transformer_blocks_20_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_20_attn_to_v_bias]
    %p_module_single_transformer_blocks_21_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_norm_linear_weight]
    %p_module_single_transformer_blocks_21_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_norm_linear_bias]
    %p_module_single_transformer_blocks_21_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_proj_mlp_weight]
    %p_module_single_transformer_blocks_21_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_proj_mlp_bias]
    %p_module_single_transformer_blocks_21_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_proj_out_weight]
    %p_module_single_transformer_blocks_21_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_proj_out_bias]
    %p_module_single_transformer_blocks_21_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_norm_q_weight]
    %p_module_single_transformer_blocks_21_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_norm_k_weight]
    %p_module_single_transformer_blocks_21_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_to_q_weight]
    %p_module_single_transformer_blocks_21_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_to_q_bias]
    %p_module_single_transformer_blocks_21_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_to_k_weight]
    %p_module_single_transformer_blocks_21_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_to_k_bias]
    %p_module_single_transformer_blocks_21_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_to_v_weight]
    %p_module_single_transformer_blocks_21_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_21_attn_to_v_bias]
    %p_module_single_transformer_blocks_22_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_norm_linear_weight]
    %p_module_single_transformer_blocks_22_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_norm_linear_bias]
    %p_module_single_transformer_blocks_22_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_proj_mlp_weight]
    %p_module_single_transformer_blocks_22_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_proj_mlp_bias]
    %p_module_single_transformer_blocks_22_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_proj_out_weight]
    %p_module_single_transformer_blocks_22_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_proj_out_bias]
    %p_module_single_transformer_blocks_22_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_norm_q_weight]
    %p_module_single_transformer_blocks_22_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_norm_k_weight]
    %p_module_single_transformer_blocks_22_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_to_q_weight]
    %p_module_single_transformer_blocks_22_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_to_q_bias]
    %p_module_single_transformer_blocks_22_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_to_k_weight]
    %p_module_single_transformer_blocks_22_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_to_k_bias]
    %p_module_single_transformer_blocks_22_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_to_v_weight]
    %p_module_single_transformer_blocks_22_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_22_attn_to_v_bias]
    %p_module_single_transformer_blocks_23_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_norm_linear_weight]
    %p_module_single_transformer_blocks_23_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_norm_linear_bias]
    %p_module_single_transformer_blocks_23_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_proj_mlp_weight]
    %p_module_single_transformer_blocks_23_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_proj_mlp_bias]
    %p_module_single_transformer_blocks_23_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_proj_out_weight]
    %p_module_single_transformer_blocks_23_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_proj_out_bias]
    %p_module_single_transformer_blocks_23_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_norm_q_weight]
    %p_module_single_transformer_blocks_23_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_norm_k_weight]
    %p_module_single_transformer_blocks_23_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_to_q_weight]
    %p_module_single_transformer_blocks_23_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_to_q_bias]
    %p_module_single_transformer_blocks_23_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_to_k_weight]
    %p_module_single_transformer_blocks_23_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_to_k_bias]
    %p_module_single_transformer_blocks_23_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_to_v_weight]
    %p_module_single_transformer_blocks_23_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_23_attn_to_v_bias]
    %p_module_single_transformer_blocks_24_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_norm_linear_weight]
    %p_module_single_transformer_blocks_24_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_norm_linear_bias]
    %p_module_single_transformer_blocks_24_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_proj_mlp_weight]
    %p_module_single_transformer_blocks_24_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_proj_mlp_bias]
    %p_module_single_transformer_blocks_24_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_proj_out_weight]
    %p_module_single_transformer_blocks_24_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_proj_out_bias]
    %p_module_single_transformer_blocks_24_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_norm_q_weight]
    %p_module_single_transformer_blocks_24_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_norm_k_weight]
    %p_module_single_transformer_blocks_24_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_to_q_weight]
    %p_module_single_transformer_blocks_24_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_to_q_bias]
    %p_module_single_transformer_blocks_24_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_to_k_weight]
    %p_module_single_transformer_blocks_24_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_to_k_bias]
    %p_module_single_transformer_blocks_24_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_to_v_weight]
    %p_module_single_transformer_blocks_24_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_24_attn_to_v_bias]
    %p_module_single_transformer_blocks_25_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_norm_linear_weight]
    %p_module_single_transformer_blocks_25_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_norm_linear_bias]
    %p_module_single_transformer_blocks_25_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_proj_mlp_weight]
    %p_module_single_transformer_blocks_25_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_proj_mlp_bias]
    %p_module_single_transformer_blocks_25_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_proj_out_weight]
    %p_module_single_transformer_blocks_25_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_proj_out_bias]
    %p_module_single_transformer_blocks_25_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_norm_q_weight]
    %p_module_single_transformer_blocks_25_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_norm_k_weight]
    %p_module_single_transformer_blocks_25_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_to_q_weight]
    %p_module_single_transformer_blocks_25_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_to_q_bias]
    %p_module_single_transformer_blocks_25_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_to_k_weight]
    %p_module_single_transformer_blocks_25_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_to_k_bias]
    %p_module_single_transformer_blocks_25_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_to_v_weight]
    %p_module_single_transformer_blocks_25_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_25_attn_to_v_bias]
    %p_module_single_transformer_blocks_26_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_norm_linear_weight]
    %p_module_single_transformer_blocks_26_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_norm_linear_bias]
    %p_module_single_transformer_blocks_26_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_proj_mlp_weight]
    %p_module_single_transformer_blocks_26_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_proj_mlp_bias]
    %p_module_single_transformer_blocks_26_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_proj_out_weight]
    %p_module_single_transformer_blocks_26_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_proj_out_bias]
    %p_module_single_transformer_blocks_26_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_norm_q_weight]
    %p_module_single_transformer_blocks_26_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_norm_k_weight]
    %p_module_single_transformer_blocks_26_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_to_q_weight]
    %p_module_single_transformer_blocks_26_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_to_q_bias]
    %p_module_single_transformer_blocks_26_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_to_k_weight]
    %p_module_single_transformer_blocks_26_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_to_k_bias]
    %p_module_single_transformer_blocks_26_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_to_v_weight]
    %p_module_single_transformer_blocks_26_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_26_attn_to_v_bias]
    %p_module_single_transformer_blocks_27_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_norm_linear_weight]
    %p_module_single_transformer_blocks_27_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_norm_linear_bias]
    %p_module_single_transformer_blocks_27_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_proj_mlp_weight]
    %p_module_single_transformer_blocks_27_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_proj_mlp_bias]
    %p_module_single_transformer_blocks_27_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_proj_out_weight]
    %p_module_single_transformer_blocks_27_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_proj_out_bias]
    %p_module_single_transformer_blocks_27_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_norm_q_weight]
    %p_module_single_transformer_blocks_27_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_norm_k_weight]
    %p_module_single_transformer_blocks_27_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_to_q_weight]
    %p_module_single_transformer_blocks_27_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_to_q_bias]
    %p_module_single_transformer_blocks_27_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_to_k_weight]
    %p_module_single_transformer_blocks_27_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_to_k_bias]
    %p_module_single_transformer_blocks_27_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_to_v_weight]
    %p_module_single_transformer_blocks_27_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_27_attn_to_v_bias]
    %p_module_single_transformer_blocks_28_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_norm_linear_weight]
    %p_module_single_transformer_blocks_28_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_norm_linear_bias]
    %p_module_single_transformer_blocks_28_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_proj_mlp_weight]
    %p_module_single_transformer_blocks_28_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_proj_mlp_bias]
    %p_module_single_transformer_blocks_28_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_proj_out_weight]
    %p_module_single_transformer_blocks_28_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_proj_out_bias]
    %p_module_single_transformer_blocks_28_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_norm_q_weight]
    %p_module_single_transformer_blocks_28_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_norm_k_weight]
    %p_module_single_transformer_blocks_28_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_to_q_weight]
    %p_module_single_transformer_blocks_28_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_to_q_bias]
    %p_module_single_transformer_blocks_28_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_to_k_weight]
    %p_module_single_transformer_blocks_28_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_to_k_bias]
    %p_module_single_transformer_blocks_28_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_to_v_weight]
    %p_module_single_transformer_blocks_28_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_28_attn_to_v_bias]
    %p_module_single_transformer_blocks_29_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_norm_linear_weight]
    %p_module_single_transformer_blocks_29_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_norm_linear_bias]
    %p_module_single_transformer_blocks_29_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_proj_mlp_weight]
    %p_module_single_transformer_blocks_29_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_proj_mlp_bias]
    %p_module_single_transformer_blocks_29_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_proj_out_weight]
    %p_module_single_transformer_blocks_29_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_proj_out_bias]
    %p_module_single_transformer_blocks_29_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_norm_q_weight]
    %p_module_single_transformer_blocks_29_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_norm_k_weight]
    %p_module_single_transformer_blocks_29_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_to_q_weight]
    %p_module_single_transformer_blocks_29_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_to_q_bias]
    %p_module_single_transformer_blocks_29_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_to_k_weight]
    %p_module_single_transformer_blocks_29_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_to_k_bias]
    %p_module_single_transformer_blocks_29_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_to_v_weight]
    %p_module_single_transformer_blocks_29_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_29_attn_to_v_bias]
    %p_module_single_transformer_blocks_30_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_norm_linear_weight]
    %p_module_single_transformer_blocks_30_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_norm_linear_bias]
    %p_module_single_transformer_blocks_30_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_proj_mlp_weight]
    %p_module_single_transformer_blocks_30_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_proj_mlp_bias]
    %p_module_single_transformer_blocks_30_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_proj_out_weight]
    %p_module_single_transformer_blocks_30_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_proj_out_bias]
    %p_module_single_transformer_blocks_30_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_norm_q_weight]
    %p_module_single_transformer_blocks_30_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_norm_k_weight]
    %p_module_single_transformer_blocks_30_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_to_q_weight]
    %p_module_single_transformer_blocks_30_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_to_q_bias]
    %p_module_single_transformer_blocks_30_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_to_k_weight]
    %p_module_single_transformer_blocks_30_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_to_k_bias]
    %p_module_single_transformer_blocks_30_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_to_v_weight]
    %p_module_single_transformer_blocks_30_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_30_attn_to_v_bias]
    %p_module_single_transformer_blocks_31_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_norm_linear_weight]
    %p_module_single_transformer_blocks_31_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_norm_linear_bias]
    %p_module_single_transformer_blocks_31_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_proj_mlp_weight]
    %p_module_single_transformer_blocks_31_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_proj_mlp_bias]
    %p_module_single_transformer_blocks_31_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_proj_out_weight]
    %p_module_single_transformer_blocks_31_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_proj_out_bias]
    %p_module_single_transformer_blocks_31_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_norm_q_weight]
    %p_module_single_transformer_blocks_31_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_norm_k_weight]
    %p_module_single_transformer_blocks_31_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_to_q_weight]
    %p_module_single_transformer_blocks_31_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_to_q_bias]
    %p_module_single_transformer_blocks_31_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_to_k_weight]
    %p_module_single_transformer_blocks_31_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_to_k_bias]
    %p_module_single_transformer_blocks_31_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_to_v_weight]
    %p_module_single_transformer_blocks_31_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_31_attn_to_v_bias]
    %p_module_single_transformer_blocks_32_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_norm_linear_weight]
    %p_module_single_transformer_blocks_32_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_norm_linear_bias]
    %p_module_single_transformer_blocks_32_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_proj_mlp_weight]
    %p_module_single_transformer_blocks_32_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_proj_mlp_bias]
    %p_module_single_transformer_blocks_32_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_proj_out_weight]
    %p_module_single_transformer_blocks_32_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_proj_out_bias]
    %p_module_single_transformer_blocks_32_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_norm_q_weight]
    %p_module_single_transformer_blocks_32_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_norm_k_weight]
    %p_module_single_transformer_blocks_32_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_to_q_weight]
    %p_module_single_transformer_blocks_32_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_to_q_bias]
    %p_module_single_transformer_blocks_32_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_to_k_weight]
    %p_module_single_transformer_blocks_32_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_to_k_bias]
    %p_module_single_transformer_blocks_32_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_to_v_weight]
    %p_module_single_transformer_blocks_32_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_32_attn_to_v_bias]
    %p_module_single_transformer_blocks_33_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_norm_linear_weight]
    %p_module_single_transformer_blocks_33_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_norm_linear_bias]
    %p_module_single_transformer_blocks_33_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_proj_mlp_weight]
    %p_module_single_transformer_blocks_33_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_proj_mlp_bias]
    %p_module_single_transformer_blocks_33_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_proj_out_weight]
    %p_module_single_transformer_blocks_33_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_proj_out_bias]
    %p_module_single_transformer_blocks_33_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_norm_q_weight]
    %p_module_single_transformer_blocks_33_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_norm_k_weight]
    %p_module_single_transformer_blocks_33_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_to_q_weight]
    %p_module_single_transformer_blocks_33_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_to_q_bias]
    %p_module_single_transformer_blocks_33_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_to_k_weight]
    %p_module_single_transformer_blocks_33_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_to_k_bias]
    %p_module_single_transformer_blocks_33_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_to_v_weight]
    %p_module_single_transformer_blocks_33_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_33_attn_to_v_bias]
    %p_module_single_transformer_blocks_34_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_norm_linear_weight]
    %p_module_single_transformer_blocks_34_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_norm_linear_bias]
    %p_module_single_transformer_blocks_34_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_proj_mlp_weight]
    %p_module_single_transformer_blocks_34_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_proj_mlp_bias]
    %p_module_single_transformer_blocks_34_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_proj_out_weight]
    %p_module_single_transformer_blocks_34_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_proj_out_bias]
    %p_module_single_transformer_blocks_34_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_norm_q_weight]
    %p_module_single_transformer_blocks_34_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_norm_k_weight]
    %p_module_single_transformer_blocks_34_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_to_q_weight]
    %p_module_single_transformer_blocks_34_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_to_q_bias]
    %p_module_single_transformer_blocks_34_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_to_k_weight]
    %p_module_single_transformer_blocks_34_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_to_k_bias]
    %p_module_single_transformer_blocks_34_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_to_v_weight]
    %p_module_single_transformer_blocks_34_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_34_attn_to_v_bias]
    %p_module_single_transformer_blocks_35_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_norm_linear_weight]
    %p_module_single_transformer_blocks_35_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_norm_linear_bias]
    %p_module_single_transformer_blocks_35_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_proj_mlp_weight]
    %p_module_single_transformer_blocks_35_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_proj_mlp_bias]
    %p_module_single_transformer_blocks_35_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_proj_out_weight]
    %p_module_single_transformer_blocks_35_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_proj_out_bias]
    %p_module_single_transformer_blocks_35_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_norm_q_weight]
    %p_module_single_transformer_blocks_35_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_norm_k_weight]
    %p_module_single_transformer_blocks_35_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_to_q_weight]
    %p_module_single_transformer_blocks_35_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_to_q_bias]
    %p_module_single_transformer_blocks_35_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_to_k_weight]
    %p_module_single_transformer_blocks_35_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_to_k_bias]
    %p_module_single_transformer_blocks_35_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_to_v_weight]
    %p_module_single_transformer_blocks_35_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_35_attn_to_v_bias]
    %p_module_single_transformer_blocks_36_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_norm_linear_weight]
    %p_module_single_transformer_blocks_36_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_norm_linear_bias]
    %p_module_single_transformer_blocks_36_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_proj_mlp_weight]
    %p_module_single_transformer_blocks_36_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_proj_mlp_bias]
    %p_module_single_transformer_blocks_36_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_proj_out_weight]
    %p_module_single_transformer_blocks_36_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_proj_out_bias]
    %p_module_single_transformer_blocks_36_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_norm_q_weight]
    %p_module_single_transformer_blocks_36_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_norm_k_weight]
    %p_module_single_transformer_blocks_36_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_to_q_weight]
    %p_module_single_transformer_blocks_36_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_to_q_bias]
    %p_module_single_transformer_blocks_36_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_to_k_weight]
    %p_module_single_transformer_blocks_36_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_to_k_bias]
    %p_module_single_transformer_blocks_36_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_to_v_weight]
    %p_module_single_transformer_blocks_36_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_36_attn_to_v_bias]
    %p_module_single_transformer_blocks_37_norm_linear_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_norm_linear_weight]
    %p_module_single_transformer_blocks_37_norm_linear_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_norm_linear_bias]
    %p_module_single_transformer_blocks_37_proj_mlp_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_proj_mlp_weight]
    %p_module_single_transformer_blocks_37_proj_mlp_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_proj_mlp_bias]
    %p_module_single_transformer_blocks_37_proj_out_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_proj_out_weight]
    %p_module_single_transformer_blocks_37_proj_out_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_proj_out_bias]
    %p_module_single_transformer_blocks_37_attn_norm_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_norm_q_weight]
    %p_module_single_transformer_blocks_37_attn_norm_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_norm_k_weight]
    %p_module_single_transformer_blocks_37_attn_to_q_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_to_q_weight]
    %p_module_single_transformer_blocks_37_attn_to_q_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_to_q_bias]
    %p_module_single_transformer_blocks_37_attn_to_k_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_to_k_weight]
    %p_module_single_transformer_blocks_37_attn_to_k_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_to_k_bias]
    %p_module_single_transformer_blocks_37_attn_to_v_weight : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_to_v_weight]
    %p_module_single_transformer_blocks_37_attn_to_v_bias : [num_users=0] = placeholder[target=p_module_single_transformer_blocks_37_attn_to_v_bias]
    %p_module_norm_out_linear_weight : [num_users=1] = placeholder[target=p_module_norm_out_linear_weight]
    %p_module_norm_out_linear_bias : [num_users=1] = placeholder[target=p_module_norm_out_linear_bias]
    %p_module_proj_out_weight : [num_users=1] = placeholder[target=p_module_proj_out_weight]
    %p_module_proj_out_bias : [num_users=1] = placeholder[target=p_module_proj_out_bias]
    %hidden_states : [num_users=2] = placeholder[target=hidden_states]
    %encoder_hidden_states : [num_users=3] = placeholder[target=encoder_hidden_states]
    %pooled_projections : [num_users=1] = placeholder[target=pooled_projections]
    %timestep : [num_users=1] = placeholder[target=timestep]
    %img_ids : [num_users=1] = placeholder[target=img_ids]
    %txt_ids : [num_users=2] = placeholder[target=txt_ids]
    %sym_size_int_62 : [num_users=7] = call_function[target=torch.ops.aten.sym_size.int](args = (%hidden_states, 0), kwargs = {})
    %sym_size_int_63 : [num_users=36] = call_function[target=torch.ops.aten.sym_size.int](args = (%encoder_hidden_states, 0), kwargs = {})
    %sym_size_int_64 : [num_users=13] = call_function[target=torch.ops.aten.sym_size.int](args = (%encoder_hidden_states, 1), kwargs = {})
    %sym_size_int_68 : [num_users=10] = call_function[target=torch.ops.aten.sym_size.int](args = (%txt_ids, 0), kwargs = {})
    %add_1548 : [num_users=17] = call_function[target=operator.add](args = (4096, %sym_size_int_64), kwargs = {})
    %mul_1164 : [num_users=5] = call_function[target=operator.mul](args = (%sym_size_int_68, %add_1548), kwargs = {})
    %mul_1165 : [num_users=5] = call_function[target=operator.mul](args = (%sym_size_int_63, %add_1548), kwargs = {})
    %eq_666 : [num_users=1] = call_function[target=operator.eq](args = (%mul_1164, %mul_1165), kwargs = {})
    %_assert_scalar_default : [num_users=0] = call_function[target=torch.ops.aten._assert_scalar.default](args = (%eq_666, Runtime assertion failed for expression Eq(s6*(s2 + 4096), s1*(s2 + 4096)) on node 'eq_666'), kwargs = {})
    %mul_1166 : [num_users=1] = call_function[target=operator.mul](args = (3072, %sym_size_int_68), kwargs = {})
    %mul_1167 : [num_users=1] = call_function[target=operator.mul](args = (%mul_1166, %add_1548), kwargs = {})
    %mul_1168 : [num_users=1] = call_function[target=operator.mul](args = (3072, %sym_size_int_63), kwargs = {})
    %mod_3 : [num_users=1] = call_function[target=operator.mod](args = (%mul_1167, %mul_1168), kwargs = {})
    %eq_667 : [num_users=1] = call_function[target=operator.eq](args = (%mod_3, 0), kwargs = {})
    %_assert_scalar_default_1 : [num_users=0] = call_function[target=torch.ops.aten._assert_scalar.default](args = (%eq_667, Runtime assertion failed for expression Eq(Mod(3072*s6*(s2 + 4096), 3072*s1), 0) on node 'eq_667'), kwargs = {})
    %floordiv_3 : [num_users=3] = call_function[target=operator.floordiv](args = (%mul_1164, %sym_size_int_63), kwargs = {})
    %mul_1169 : [num_users=1] = call_function[target=operator.mul](args = (%sym_size_int_63, %floordiv_3), kwargs = {})
    %eq_668 : [num_users=1] = call_function[target=operator.eq](args = (%mul_1164, %mul_1169), kwargs = {})
    %_assert_scalar_default_2 : [num_users=0] = call_function[target=torch.ops.aten._assert_scalar.default](args = (%eq_668, Runtime assertion failed for expression Eq(s6*(s2 + 4096), s1*((s6*(s2 + 4096)//s1))) on node 'eq_668'), kwargs = {})
    %eq_669 : [num_users=1] = call_function[target=operator.eq](args = (%floordiv_3, %add_1548), kwargs = {})
    %_assert_scalar_default_3 : [num_users=0] = call_function[target=torch.ops.aten._assert_scalar.default](args = (%eq_669, Runtime assertion failed for expression Eq((s6*(s2 + 4096)//s1), s2 + 4096) on node 'eq_669'), kwargs = {})
    %mul_3 : [num_users=6] = call_function[target=operator.mul](args = (%sym_size_int_62, 4096), kwargs = {})
    %view : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%hidden_states, [%mul_3, 64]), kwargs = {})
    %t : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_x_embedder_weight,), kwargs = {})
    %addmm : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_x_embedder_bias, %view, %t), kwargs = {})
    %view_1 : [num_users=2] = call_function[target=torch.ops.aten.view.default](args = (%addmm, [%sym_size_int_62, 4096, 3072]), kwargs = {})
    %_to_copy : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%timestep,), kwargs = {dtype: torch.float16})
    %mul_11 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy, 1000), kwargs = {})
    %arange : [num_users=1] = call_function[target=torch.ops.aten.arange.start](args = (0, 128), kwargs = {dtype: torch.float32, device: cuda:0, pin_memory: False})
    %mul_13 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%arange, -9.210340371976184), kwargs = {})
    %div : [num_users=1] = call_function[target=torch.ops.aten.div.Tensor](args = (%mul_13, 128), kwargs = {})
    %exp : [num_users=1] = call_function[target=torch.ops.aten.exp.default](args = (%div,), kwargs = {})
    %slice_1 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%mul_11, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_1, 1), kwargs = {})
    %_to_copy_1 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%unsqueeze,), kwargs = {dtype: torch.float32})
    %unsqueeze_1 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%exp, 0), kwargs = {})
    %slice_2 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%unsqueeze_1, 1, 0, 9223372036854775807), kwargs = {})
    %mul_17 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_1, %slice_2), kwargs = {})
    %mul_20 : [num_users=2] = call_function[target=torch.ops.aten.mul.Tensor](args = (%mul_17, 1), kwargs = {})
    %sin : [num_users=1] = call_function[target=torch.ops.aten.sin.default](args = (%mul_20,), kwargs = {})
    %cos : [num_users=1] = call_function[target=torch.ops.aten.cos.default](args = (%mul_20,), kwargs = {})
    %cat : [num_users=2] = call_function[target=torch.ops.aten.cat.default](args = ([%sin, %cos], -1), kwargs = {})
    %slice_3 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%cat, 0, 0, 9223372036854775807), kwargs = {})
    %slice_4 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%slice_3, 1, 128, 9223372036854775807), kwargs = {})
    %slice_5 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%cat, 0, 0, 9223372036854775807), kwargs = {})
    %slice_6 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%slice_5, 1, 0, 128), kwargs = {})
    %cat_1 : [num_users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%slice_4, %slice_6], -1), kwargs = {})
    %_to_copy_2 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%cat_1,), kwargs = {dtype: torch.float16})
    %t_1 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_time_text_embed_timestep_embedder_linear_1_weight,), kwargs = {})
    %addmm_1 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_time_text_embed_timestep_embedder_linear_1_bias, %_to_copy_2, %t_1), kwargs = {})
    %silu : [num_users=1] = call_function[target=torch.ops.aten.silu.default](args = (%addmm_1,), kwargs = {})
    %t_2 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_time_text_embed_timestep_embedder_linear_2_weight,), kwargs = {})
    %addmm_2 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_time_text_embed_timestep_embedder_linear_2_bias, %silu, %t_2), kwargs = {})
    %t_3 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_time_text_embed_text_embedder_linear_1_weight,), kwargs = {})
    %addmm_3 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_time_text_embed_text_embedder_linear_1_bias, %pooled_projections, %t_3), kwargs = {})
    %silu_1 : [num_users=1] = call_function[target=torch.ops.aten.silu.default](args = (%addmm_3,), kwargs = {})
    %t_4 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_time_text_embed_text_embedder_linear_2_weight,), kwargs = {})
    %addmm_4 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_time_text_embed_text_embedder_linear_2_bias, %silu_1, %t_4), kwargs = {})
    %add_73 : [num_users=4] = call_function[target=torch.ops.aten.add.Tensor](args = (%addmm_2, %addmm_4), kwargs = {})
    %mul_59 : [num_users=6] = call_function[target=operator.mul](args = (%sym_size_int_63, %sym_size_int_64), kwargs = {})
    %view_2 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%encoder_hidden_states, [%mul_59, 4096]), kwargs = {})
    %t_5 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_context_embedder_weight,), kwargs = {})
    %addmm_5 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_context_embedder_bias, %view_2, %t_5), kwargs = {})
    %view_3 : [num_users=2] = call_function[target=torch.ops.aten.view.default](args = (%addmm_5, [%sym_size_int_63, %sym_size_int_64, 3072]), kwargs = {})
    %cat_2 : [num_users=3] = call_function[target=torch.ops.aten.cat.default](args = ([%txt_ids, %img_ids], 1), kwargs = {})
    %select : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%cat_2, 2, 0), kwargs = {})
    %arange_1 : [num_users=1] = call_function[target=torch.ops.aten.arange.start_step](args = (0, 16, 2), kwargs = {dtype: torch.float64, device: cuda:0, pin_memory: False})
    %div_1 : [num_users=1] = call_function[target=torch.ops.aten.div.Tensor](args = (%arange_1, 16), kwargs = {})
    %pow_1 : [num_users=1] = call_function[target=torch.ops.aten.pow.Scalar](args = (10000, %div_1), kwargs = {})
    %reciprocal : [num_users=1] = call_function[target=torch.ops.aten.reciprocal.default](args = (%pow_1,), kwargs = {})
    %mul_73 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%reciprocal, 1.0), kwargs = {})
    %unsqueeze_2 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%select, 2), kwargs = {})
    %permute : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%unsqueeze_2, [0, 1, 2]), kwargs = {})
    %unsqueeze_3 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%mul_73, 1), kwargs = {})
    %unsqueeze_4 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%unsqueeze_3, 2), kwargs = {})
    %permute_1 : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%unsqueeze_4, [1, 2, 0]), kwargs = {})
    %mul_80 : [num_users=2] = call_function[target=torch.ops.aten.mul.Tensor](args = (%permute, %permute_1), kwargs = {})
    %cos_1 : [num_users=1] = call_function[target=torch.ops.aten.cos.default](args = (%mul_80,), kwargs = {})
    %sin_1 : [num_users=2] = call_function[target=torch.ops.aten.sin.default](args = (%mul_80,), kwargs = {})
    %neg : [num_users=1] = call_function[target=torch.ops.aten.neg.default](args = (%sin_1,), kwargs = {})
    %stack : [num_users=1] = call_function[target=torch.ops.aten.stack.default](args = ([%cos_1, %neg, %sin_1, %cos_1], -1), kwargs = {})
    %view_4 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%stack, [%sym_size_int_68, -1, 8, 2, 2]), kwargs = {})
    %_to_copy_3 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_4,), kwargs = {dtype: torch.float32})
    %select_1 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%cat_2, 2, 1), kwargs = {})
    %arange_2 : [num_users=1] = call_function[target=torch.ops.aten.arange.start_step](args = (0, 56, 2), kwargs = {dtype: torch.float64, device: cuda:0, pin_memory: False})
    %div_2 : [num_users=1] = call_function[target=torch.ops.aten.div.Tensor](args = (%arange_2, 56), kwargs = {})
    %pow_2 : [num_users=1] = call_function[target=torch.ops.aten.pow.Scalar](args = (10000, %div_2), kwargs = {})
    %reciprocal_1 : [num_users=1] = call_function[target=torch.ops.aten.reciprocal.default](args = (%pow_2,), kwargs = {})
    %mul_105 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%reciprocal_1, 1.0), kwargs = {})
    %unsqueeze_5 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%select_1, 2), kwargs = {})
    %permute_2 : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%unsqueeze_5, [0, 1, 2]), kwargs = {})
    %unsqueeze_6 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%mul_105, 1), kwargs = {})
    %unsqueeze_7 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%unsqueeze_6, 2), kwargs = {})
    %permute_3 : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%unsqueeze_7, [1, 2, 0]), kwargs = {})
    %mul_112 : [num_users=2] = call_function[target=torch.ops.aten.mul.Tensor](args = (%permute_2, %permute_3), kwargs = {})
    %cos_2 : [num_users=1] = call_function[target=torch.ops.aten.cos.default](args = (%mul_112,), kwargs = {})
    %sin_2 : [num_users=2] = call_function[target=torch.ops.aten.sin.default](args = (%mul_112,), kwargs = {})
    %neg_1 : [num_users=1] = call_function[target=torch.ops.aten.neg.default](args = (%sin_2,), kwargs = {})
    %stack_1 : [num_users=1] = call_function[target=torch.ops.aten.stack.default](args = ([%cos_2, %neg_1, %sin_2, %cos_2], -1), kwargs = {})
    %view_5 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%stack_1, [%sym_size_int_68, -1, 28, 2, 2]), kwargs = {})
    %_to_copy_4 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_5,), kwargs = {dtype: torch.float32})
    %select_2 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%cat_2, 2, 2), kwargs = {})
    %arange_3 : [num_users=1] = call_function[target=torch.ops.aten.arange.start_step](args = (0, 56, 2), kwargs = {dtype: torch.float64, device: cuda:0, pin_memory: False})
    %div_3 : [num_users=1] = call_function[target=torch.ops.aten.div.Tensor](args = (%arange_3, 56), kwargs = {})
    %pow_3 : [num_users=1] = call_function[target=torch.ops.aten.pow.Scalar](args = (10000, %div_3), kwargs = {})
    %reciprocal_2 : [num_users=1] = call_function[target=torch.ops.aten.reciprocal.default](args = (%pow_3,), kwargs = {})
    %mul_137 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%reciprocal_2, 1.0), kwargs = {})
    %unsqueeze_8 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%select_2, 2), kwargs = {})
    %permute_4 : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%unsqueeze_8, [0, 1, 2]), kwargs = {})
    %unsqueeze_9 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%mul_137, 1), kwargs = {})
    %unsqueeze_10 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%unsqueeze_9, 2), kwargs = {})
    %permute_5 : [num_users=1] = call_function[target=torch.ops.aten.permute.default](args = (%unsqueeze_10, [1, 2, 0]), kwargs = {})
    %mul_144 : [num_users=2] = call_function[target=torch.ops.aten.mul.Tensor](args = (%permute_4, %permute_5), kwargs = {})
    %cos_3 : [num_users=1] = call_function[target=torch.ops.aten.cos.default](args = (%mul_144,), kwargs = {})
    %sin_3 : [num_users=2] = call_function[target=torch.ops.aten.sin.default](args = (%mul_144,), kwargs = {})
    %neg_2 : [num_users=1] = call_function[target=torch.ops.aten.neg.default](args = (%sin_3,), kwargs = {})
    %stack_2 : [num_users=1] = call_function[target=torch.ops.aten.stack.default](args = ([%cos_3, %neg_2, %sin_3, %cos_3], -1), kwargs = {})
    %view_6 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%stack_2, [%sym_size_int_68, -1, 28, 2, 2]), kwargs = {})
    %_to_copy_5 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_6,), kwargs = {dtype: torch.float32})
    %cat_3 : [num_users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%_to_copy_3, %_to_copy_4, %_to_copy_5], -3), kwargs = {})
    %unsqueeze_11 : [num_users=8] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%cat_3, 1), kwargs = {})
    %silu_2 : [num_users=1] = call_function[target=torch.ops.aten.silu.default](args = (%add_73,), kwargs = {})
    %t_6 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_norm1_linear_weight,), kwargs = {})
    %addmm_6 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_norm1_linear_bias, %silu_2, %t_6), kwargs = {})
    %split : [num_users=6] = call_function[target=torch.ops.aten.split.Tensor](args = (%addmm_6, 3072, 1), kwargs = {})
    %getitem : [num_users=1] = call_function[target=operator.getitem](args = (%split, 0), kwargs = {})
    %getitem_1 : [num_users=1] = call_function[target=operator.getitem](args = (%split, 1), kwargs = {})
    %getitem_2 : [num_users=1] = call_function[target=operator.getitem](args = (%split, 2), kwargs = {})
    %getitem_3 : [num_users=1] = call_function[target=operator.getitem](args = (%split, 3), kwargs = {})
    %getitem_4 : [num_users=1] = call_function[target=operator.getitem](args = (%split, 4), kwargs = {})
    %getitem_5 : [num_users=1] = call_function[target=operator.getitem](args = (%split, 5), kwargs = {})
    %native_layer_norm : [num_users=1] = call_function[target=torch.ops.aten.native_layer_norm.default](args = (%view_1, [3072], None, None, 1e-06), kwargs = {})
    %getitem_6 : [num_users=1] = call_function[target=operator.getitem](args = (%native_layer_norm, 0), kwargs = {})
    %slice_7 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_1, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_12 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_7, 1), kwargs = {})
    %add_279 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%unsqueeze_12, 1), kwargs = {})
    %mul_201 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%getitem_6, %add_279), kwargs = {})
    %slice_8 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_13 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_8, 1), kwargs = {})
    %add_295 : [num_users=3] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_201, %unsqueeze_13), kwargs = {})
    %silu_3 : [num_users=1] = call_function[target=torch.ops.aten.silu.default](args = (%add_73,), kwargs = {})
    %t_7 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_norm1_context_linear_weight,), kwargs = {})
    %addmm_7 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_norm1_context_linear_bias, %silu_3, %t_7), kwargs = {})
    %split_1 : [num_users=6] = call_function[target=torch.ops.aten.split.Tensor](args = (%addmm_7, 3072, 1), kwargs = {})
    %getitem_9 : [num_users=1] = call_function[target=operator.getitem](args = (%split_1, 0), kwargs = {})
    %getitem_10 : [num_users=1] = call_function[target=operator.getitem](args = (%split_1, 1), kwargs = {})
    %getitem_11 : [num_users=1] = call_function[target=operator.getitem](args = (%split_1, 2), kwargs = {})
    %getitem_12 : [num_users=1] = call_function[target=operator.getitem](args = (%split_1, 3), kwargs = {})
    %getitem_13 : [num_users=1] = call_function[target=operator.getitem](args = (%split_1, 4), kwargs = {})
    %getitem_14 : [num_users=1] = call_function[target=operator.getitem](args = (%split_1, 5), kwargs = {})
    %native_layer_norm_1 : [num_users=1] = call_function[target=torch.ops.aten.native_layer_norm.default](args = (%view_3, [3072], None, None, 1e-06), kwargs = {})
    %getitem_15 : [num_users=1] = call_function[target=operator.getitem](args = (%native_layer_norm_1, 0), kwargs = {})
    %slice_9 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_10, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_14 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_9, 1), kwargs = {})
    %add_343 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%unsqueeze_14, 1), kwargs = {})
    %mul_239 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%getitem_15, %add_343), kwargs = {})
    %slice_10 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_9, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_15 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_10, 1), kwargs = {})
    %add_359 : [num_users=3] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_239, %unsqueeze_15), kwargs = {})
    %view_7 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_295, [%mul_3, 3072]), kwargs = {})
    %t_8 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_to_q_weight,), kwargs = {})
    %addmm_8 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_attn_to_q_bias, %view_7, %t_8), kwargs = {})
    %view_8 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_8, [%sym_size_int_62, 4096, 3072]), kwargs = {})
    %view_9 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_295, [%mul_3, 3072]), kwargs = {})
    %t_9 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_to_k_weight,), kwargs = {})
    %addmm_9 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_attn_to_k_bias, %view_9, %t_9), kwargs = {})
    %view_10 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_9, [%sym_size_int_62, 4096, 3072]), kwargs = {})
    %view_11 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_295, [%mul_3, 3072]), kwargs = {})
    %t_10 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_to_v_weight,), kwargs = {})
    %addmm_10 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_attn_to_v_bias, %view_11, %t_10), kwargs = {})
    %view_12 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_10, [%sym_size_int_62, 4096, 3072]), kwargs = {})
    %view_13 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_8, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose : [num_users=2] = call_function[target=torch.ops.aten.transpose.int](args = (%view_13, 1, 2), kwargs = {})
    %view_14 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_10, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_1 : [num_users=2] = call_function[target=torch.ops.aten.transpose.int](args = (%view_14, 1, 2), kwargs = {})
    %view_15 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_12, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_2 : [num_users=1] = call_function[target=torch.ops.aten.transpose.int](args = (%view_15, 1, 2), kwargs = {})
    %_to_copy_6 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%transpose,), kwargs = {dtype: torch.float32})
    %pow_4 : [num_users=1] = call_function[target=torch.ops.aten.pow.Tensor_Scalar](args = (%_to_copy_6, 2), kwargs = {})
    %mean : [num_users=1] = call_function[target=torch.ops.aten.mean.dim](args = (%pow_4, [-1], True), kwargs = {})
    %add_439 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mean, 1e-06), kwargs = {})
    %rsqrt : [num_users=1] = call_function[target=torch.ops.aten.rsqrt.default](args = (%add_439,), kwargs = {})
    %mul_296 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%transpose, %rsqrt), kwargs = {})
    %_to_copy_7 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_296,), kwargs = {dtype: torch.float16})
    %mul_301 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_7, %p_module_transformer_blocks_0_attn_norm_q_weight), kwargs = {})
    %_to_copy_8 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%transpose_1,), kwargs = {dtype: torch.float32})
    %pow_5 : [num_users=1] = call_function[target=torch.ops.aten.pow.Tensor_Scalar](args = (%_to_copy_8, 2), kwargs = {})
    %mean_1 : [num_users=1] = call_function[target=torch.ops.aten.mean.dim](args = (%pow_5, [-1], True), kwargs = {})
    %add_480 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mean_1, 1e-06), kwargs = {})
    %rsqrt_1 : [num_users=1] = call_function[target=torch.ops.aten.rsqrt.default](args = (%add_480,), kwargs = {})
    %mul_314 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%transpose_1, %rsqrt_1), kwargs = {})
    %_to_copy_9 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_314,), kwargs = {dtype: torch.float16})
    %mul_319 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_9, %p_module_transformer_blocks_0_attn_norm_k_weight), kwargs = {})
    %view_16 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_359, [%mul_59, 3072]), kwargs = {})
    %t_11 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_add_q_proj_weight,), kwargs = {})
    %addmm_11 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_attn_add_q_proj_bias, %view_16, %t_11), kwargs = {})
    %view_17 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_11, [%sym_size_int_63, %sym_size_int_64, 3072]), kwargs = {})
    %view_18 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_359, [%mul_59, 3072]), kwargs = {})
    %t_12 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_add_k_proj_weight,), kwargs = {})
    %addmm_12 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_attn_add_k_proj_bias, %view_18, %t_12), kwargs = {})
    %view_19 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_12, [%sym_size_int_63, %sym_size_int_64, 3072]), kwargs = {})
    %view_20 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_359, [%mul_59, 3072]), kwargs = {})
    %t_13 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_add_v_proj_weight,), kwargs = {})
    %addmm_13 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_attn_add_v_proj_bias, %view_20, %t_13), kwargs = {})
    %view_21 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_13, [%sym_size_int_63, %sym_size_int_64, 3072]), kwargs = {})
    %view_22 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_17, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_3 : [num_users=2] = call_function[target=torch.ops.aten.transpose.int](args = (%view_22, 1, 2), kwargs = {})
    %view_23 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_19, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_4 : [num_users=2] = call_function[target=torch.ops.aten.transpose.int](args = (%view_23, 1, 2), kwargs = {})
    %view_24 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_21, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_5 : [num_users=1] = call_function[target=torch.ops.aten.transpose.int](args = (%view_24, 1, 2), kwargs = {})
    %_to_copy_10 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%transpose_3,), kwargs = {dtype: torch.float32})
    %pow_6 : [num_users=1] = call_function[target=torch.ops.aten.pow.Tensor_Scalar](args = (%_to_copy_10, 2), kwargs = {})
    %mean_2 : [num_users=1] = call_function[target=torch.ops.aten.mean.dim](args = (%pow_6, [-1], True), kwargs = {})
    %add_581 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mean_2, 1e-06), kwargs = {})
    %rsqrt_2 : [num_users=1] = call_function[target=torch.ops.aten.rsqrt.default](args = (%add_581,), kwargs = {})
    %mul_383 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%transpose_3, %rsqrt_2), kwargs = {})
    %_to_copy_11 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_383,), kwargs = {dtype: torch.float16})
    %mul_390 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_11, %p_module_transformer_blocks_0_attn_norm_added_q_weight), kwargs = {})
    %_to_copy_12 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%transpose_4,), kwargs = {dtype: torch.float32})
    %pow_7 : [num_users=1] = call_function[target=torch.ops.aten.pow.Tensor_Scalar](args = (%_to_copy_12, 2), kwargs = {})
    %mean_3 : [num_users=1] = call_function[target=torch.ops.aten.mean.dim](args = (%pow_7, [-1], True), kwargs = {})
    %add_622 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mean_3, 1e-06), kwargs = {})
    %rsqrt_3 : [num_users=1] = call_function[target=torch.ops.aten.rsqrt.default](args = (%add_622,), kwargs = {})
    %mul_409 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%transpose_4, %rsqrt_3), kwargs = {})
    %_to_copy_13 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_409,), kwargs = {dtype: torch.float16})
    %mul_416 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_13, %p_module_transformer_blocks_0_attn_norm_added_k_weight), kwargs = {})
    %cat_4 : [num_users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%mul_390, %mul_301], 2), kwargs = {})
    %cat_5 : [num_users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%mul_416, %mul_319], 2), kwargs = {})
    %cat_6 : [num_users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%transpose_5, %transpose_2], 2), kwargs = {})
    %_to_copy_14 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%cat_4,), kwargs = {dtype: torch.float32})
    %view_25 : [num_users=2] = call_function[target=torch.ops.aten.view.default](args = (%_to_copy_14, [%sym_size_int_63, 24, %add_1548, -1, 1, 2]), kwargs = {})
    %_to_copy_15 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%cat_5,), kwargs = {dtype: torch.float32})
    %view_26 : [num_users=2] = call_function[target=torch.ops.aten.view.default](args = (%_to_copy_15, [%sym_size_int_63, 24, %add_1548, -1, 1, 2]), kwargs = {})
    %select_3 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 0), kwargs = {})
    %select_4 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_25, 5, 0), kwargs = {})
    %mul_468 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_3, %select_4), kwargs = {})
    %select_5 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 1), kwargs = {})
    %select_6 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_25, 5, 1), kwargs = {})
    %mul_481 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_5, %select_6), kwargs = {})
    %add_723 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_468, %mul_481), kwargs = {})
    %select_7 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 0), kwargs = {})
    %select_8 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_26, 5, 0), kwargs = {})
    %mul_498 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_7, %select_8), kwargs = {})
    %select_9 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 1), kwargs = {})
    %select_10 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_26, 5, 1), kwargs = {})
    %mul_511 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_9, %select_10), kwargs = {})
    %add_766 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_498, %mul_511), kwargs = {})
    %view_27 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_723, [%sym_size_int_63, 24, %add_1548, 128]), kwargs = {})
    %_to_copy_16 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_27,), kwargs = {dtype: torch.float16, layout: torch.strided, device: cuda:0})
    %view_28 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_766, [%sym_size_int_63, 24, %add_1548, 128]), kwargs = {})
    %_to_copy_17 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_28,), kwargs = {dtype: torch.float16, layout: torch.strided, device: cuda:0})
    %_scaled_dot_product_flash_attention : [num_users=1] = call_function[target=torch.ops.aten._scaled_dot_product_flash_attention.default](args = (%_to_copy_16, %_to_copy_17, %cat_6), kwargs = {scale: 0.08838834764831843})
    %getitem_18 : [num_users=1] = call_function[target=operator.getitem](args = (%_scaled_dot_product_flash_attention, 0), kwargs = {})
    %transpose_6 : [num_users=1] = call_function[target=torch.ops.aten.transpose.int](args = (%getitem_18, 1, 2), kwargs = {})
    %mul_562 : [num_users=0] = call_function[target=operator.mul](args = (%mul_1164, 24), kwargs = {})
    %clone : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%transpose_6,), kwargs = {memory_format: torch.contiguous_format})
    %_unsafe_view : [num_users=1] = call_function[target=torch.ops.aten._unsafe_view.default](args = (%clone, [%sym_size_int_63, %floordiv_3, 3072]), kwargs = {})
    %_to_copy_18 : [num_users=2] = call_function[target=torch.ops.aten._to_copy.default](args = (%_unsafe_view,), kwargs = {dtype: torch.float16})
    %slice_11 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%_to_copy_18, 0, 0, 9223372036854775807), kwargs = {})
    %slice_12 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%slice_11, 1, 0, %sym_size_int_64), kwargs = {})
    %slice_13 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%_to_copy_18, 0, 0, 9223372036854775807), kwargs = {})
    %slice_14 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%slice_13, 1, %sym_size_int_64, 9223372036854775807), kwargs = {})
    %t_14 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_to_out_0_weight,), kwargs = {})
    %mul_602 : [num_users=1] = call_function[target=operator.mul](args = (%sym_size_int_68, 4096), kwargs = {})
    %clone_1 : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%slice_14,), kwargs = {memory_format: torch.contiguous_format})
    %_unsafe_view_1 : [num_users=1] = call_function[target=torch.ops.aten._unsafe_view.default](args = (%clone_1, [%mul_602, 3072]), kwargs = {})
    %mm : [num_users=1] = call_function[target=torch.ops.aten.mm.default](args = (%_unsafe_view_1, %t_14), kwargs = {})
    %_unsafe_view_2 : [num_users=1] = call_function[target=torch.ops.aten._unsafe_view.default](args = (%mm, [%sym_size_int_68, 4096, 3072]), kwargs = {})
    %add_850 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%_unsafe_view_2, %p_module_transformer_blocks_0_attn_to_out_0_bias), kwargs = {})
    %clone_2 : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%add_850,), kwargs = {})
    %t_15 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_attn_to_add_out_weight,), kwargs = {})
    %mul_635 : [num_users=1] = call_function[target=operator.mul](args = (%sym_size_int_68, %sym_size_int_64), kwargs = {})
    %clone_3 : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%slice_12,), kwargs = {memory_format: torch.contiguous_format})
    %_unsafe_view_3 : [num_users=1] = call_function[target=torch.ops.aten._unsafe_view.default](args = (%clone_3, [%mul_635, 3072]), kwargs = {})
    %mm_1 : [num_users=1] = call_function[target=torch.ops.aten.mm.default](args = (%_unsafe_view_3, %t_15), kwargs = {})
    %_unsafe_view_4 : [num_users=1] = call_function[target=torch.ops.aten._unsafe_view.default](args = (%mm_1, [%sym_size_int_68, %sym_size_int_64, 3072]), kwargs = {})
    %add_873 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%_unsafe_view_4, %p_module_transformer_blocks_0_attn_to_add_out_bias), kwargs = {})
    %unsqueeze_16 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%getitem_2, 1), kwargs = {})
    %mul_671 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%unsqueeze_16, %clone_2), kwargs = {})
    %add_886 : [num_users=2] = call_function[target=torch.ops.aten.add.Tensor](args = (%view_1, %mul_671), kwargs = {})
    %native_layer_norm_2 : [num_users=1] = call_function[target=torch.ops.aten.native_layer_norm.default](args = (%add_886, [3072], None, None, 1e-06), kwargs = {})
    %getitem_27 : [num_users=1] = call_function[target=operator.getitem](args = (%native_layer_norm_2, 0), kwargs = {})
    %slice_15 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_4, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_17 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_15, 1), kwargs = {})
    %add_910 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%unsqueeze_17, 1), kwargs = {})
    %mul_688 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%getitem_27, %add_910), kwargs = {})
    %slice_16 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_3, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_18 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_16, 1), kwargs = {})
    %add_926 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_688, %unsqueeze_18), kwargs = {})
    %view_29 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_926, [%mul_3, 3072]), kwargs = {})
    %t_16 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_ff_net_0_proj_weight,), kwargs = {})
    %addmm_14 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_ff_net_0_proj_bias, %view_29, %t_16), kwargs = {})
    %view_30 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_14, [%sym_size_int_62, 4096, 12288]), kwargs = {})
    %gelu : [num_users=1] = call_function[target=torch.ops.aten.gelu.default](args = (%view_30,), kwargs = {approximate: tanh})
    %clone_4 : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%gelu,), kwargs = {})
    %view_31 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%clone_4, [%mul_3, 12288]), kwargs = {})
    %t_17 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_ff_net_2_weight,), kwargs = {})
    %addmm_15 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_ff_net_2_bias, %view_31, %t_17), kwargs = {})
    %view_32 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_15, [%sym_size_int_62, 4096, 3072]), kwargs = {})
    %unsqueeze_19 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%getitem_5, 1), kwargs = {})
    %mul_723 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%unsqueeze_19, %view_32), kwargs = {})
    %add_967 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%add_886, %mul_723), kwargs = {})
    %unsqueeze_20 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%getitem_11, 1), kwargs = {})
    %mul_730 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%unsqueeze_20, %add_873), kwargs = {})
    %add_980 : [num_users=2] = call_function[target=torch.ops.aten.add.Tensor](args = (%view_3, %mul_730), kwargs = {})
    %native_layer_norm_3 : [num_users=1] = call_function[target=torch.ops.aten.native_layer_norm.default](args = (%add_980, [3072], None, None, 1e-06), kwargs = {})
    %getitem_30 : [num_users=1] = call_function[target=operator.getitem](args = (%native_layer_norm_3, 0), kwargs = {})
    %slice_17 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_13, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_21 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_17, 1), kwargs = {})
    %add_1004 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%unsqueeze_21, 1), kwargs = {})
    %mul_750 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%getitem_30, %add_1004), kwargs = {})
    %slice_18 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_12, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_22 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_18, 1), kwargs = {})
    %add_1020 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_750, %unsqueeze_22), kwargs = {})
    %view_33 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1020, [%mul_59, 3072]), kwargs = {})
    %t_18 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_ff_context_net_0_proj_weight,), kwargs = {})
    %addmm_16 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_ff_context_net_0_proj_bias, %view_33, %t_18), kwargs = {})
    %view_34 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_16, [%sym_size_int_63, %sym_size_int_64, 12288]), kwargs = {})
    %gelu_1 : [num_users=1] = call_function[target=torch.ops.aten.gelu.default](args = (%view_34,), kwargs = {approximate: tanh})
    %clone_5 : [num_users=1] = call_function[target=torch.ops.aten.clone.default](args = (%gelu_1,), kwargs = {})
    %view_35 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%clone_5, [%mul_59, 12288]), kwargs = {})
    %t_19 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_transformer_blocks_0_ff_context_net_2_weight,), kwargs = {})
    %addmm_17 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_transformer_blocks_0_ff_context_net_2_bias, %view_35, %t_19), kwargs = {})
    %view_36 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_17, [%sym_size_int_63, %sym_size_int_64, 3072]), kwargs = {})
    %unsqueeze_23 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%getitem_14, 1), kwargs = {})
    %mul_793 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%unsqueeze_23, %view_36), kwargs = {})
    %add_1061 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%add_980, %mul_793), kwargs = {})
    %clamp : [num_users=1] = call_function[target=torch.ops.aten.clamp.default](args = (%add_1061, -65504, 65504), kwargs = {})
    %cat_7 : [num_users=2] = call_function[target=torch.ops.aten.cat.default](args = ([%clamp, %add_967], 1), kwargs = {})
    %silu_4 : [num_users=1] = call_function[target=torch.ops.aten.silu.default](args = (%add_73,), kwargs = {})
    %t_20 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_single_transformer_blocks_0_norm_linear_weight,), kwargs = {})
    %addmm_18 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_single_transformer_blocks_0_norm_linear_bias, %silu_4, %t_20), kwargs = {})
    %split_2 : [num_users=3] = call_function[target=torch.ops.aten.split.Tensor](args = (%addmm_18, 3072, 1), kwargs = {})
    %getitem_33 : [num_users=1] = call_function[target=operator.getitem](args = (%split_2, 0), kwargs = {})
    %getitem_34 : [num_users=1] = call_function[target=operator.getitem](args = (%split_2, 1), kwargs = {})
    %getitem_35 : [num_users=1] = call_function[target=operator.getitem](args = (%split_2, 2), kwargs = {})
    %native_layer_norm_4 : [num_users=1] = call_function[target=torch.ops.aten.native_layer_norm.default](args = (%cat_7, [3072], None, None, 1e-06), kwargs = {})
    %getitem_36 : [num_users=1] = call_function[target=operator.getitem](args = (%native_layer_norm_4, 0), kwargs = {})
    %slice_19 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_34, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_24 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_19, 1), kwargs = {})
    %add_1108 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%unsqueeze_24, 1), kwargs = {})
    %mul_829 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%getitem_36, %add_1108), kwargs = {})
    %slice_20 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_33, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_25 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_20, 1), kwargs = {})
    %add_1124 : [num_users=4] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_829, %unsqueeze_25), kwargs = {})
    %view_37 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1124, [%mul_1165, 3072]), kwargs = {})
    %t_21 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_single_transformer_blocks_0_proj_mlp_weight,), kwargs = {})
    %addmm_19 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_single_transformer_blocks_0_proj_mlp_bias, %view_37, %t_21), kwargs = {})
    %view_38 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_19, [%sym_size_int_63, %add_1548, 12288]), kwargs = {})
    %gelu_2 : [num_users=1] = call_function[target=torch.ops.aten.gelu.default](args = (%view_38,), kwargs = {approximate: tanh})
    %view_39 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1124, [%mul_1165, 3072]), kwargs = {})
    %t_22 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_single_transformer_blocks_0_attn_to_q_weight,), kwargs = {})
    %addmm_20 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_single_transformer_blocks_0_attn_to_q_bias, %view_39, %t_22), kwargs = {})
    %view_40 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_20, [%sym_size_int_63, %add_1548, 3072]), kwargs = {})
    %view_41 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1124, [%mul_1165, 3072]), kwargs = {})
    %t_23 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_single_transformer_blocks_0_attn_to_k_weight,), kwargs = {})
    %addmm_21 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_single_transformer_blocks_0_attn_to_k_bias, %view_41, %t_23), kwargs = {})
    %view_42 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_21, [%sym_size_int_63, %add_1548, 3072]), kwargs = {})
    %view_43 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1124, [%mul_1165, 3072]), kwargs = {})
    %t_24 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_single_transformer_blocks_0_attn_to_v_weight,), kwargs = {})
    %addmm_22 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_single_transformer_blocks_0_attn_to_v_bias, %view_43, %t_24), kwargs = {})
    %view_44 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_22, [%sym_size_int_63, %add_1548, 3072]), kwargs = {})
    %view_45 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_40, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_7 : [num_users=2] = call_function[target=torch.ops.aten.transpose.int](args = (%view_45, 1, 2), kwargs = {})
    %view_46 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_42, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_8 : [num_users=2] = call_function[target=torch.ops.aten.transpose.int](args = (%view_46, 1, 2), kwargs = {})
    %view_47 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%view_44, [%sym_size_int_63, -1, 24, 128]), kwargs = {})
    %transpose_9 : [num_users=1] = call_function[target=torch.ops.aten.transpose.int](args = (%view_47, 1, 2), kwargs = {})
    %_to_copy_19 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%transpose_7,), kwargs = {dtype: torch.float32})
    %pow_8 : [num_users=1] = call_function[target=torch.ops.aten.pow.Tensor_Scalar](args = (%_to_copy_19, 2), kwargs = {})
    %mean_4 : [num_users=1] = call_function[target=torch.ops.aten.mean.dim](args = (%pow_8, [-1], True), kwargs = {})
    %add_1218 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mean_4, 1e-06), kwargs = {})
    %rsqrt_4 : [num_users=1] = call_function[target=torch.ops.aten.rsqrt.default](args = (%add_1218,), kwargs = {})
    %mul_912 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%transpose_7, %rsqrt_4), kwargs = {})
    %_to_copy_20 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_912,), kwargs = {dtype: torch.float16})
    %mul_919 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_20, %p_module_single_transformer_blocks_0_attn_norm_q_weight), kwargs = {})
    %_to_copy_21 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%transpose_8,), kwargs = {dtype: torch.float32})
    %pow_9 : [num_users=1] = call_function[target=torch.ops.aten.pow.Tensor_Scalar](args = (%_to_copy_21, 2), kwargs = {})
    %mean_5 : [num_users=1] = call_function[target=torch.ops.aten.mean.dim](args = (%pow_9, [-1], True), kwargs = {})
    %add_1259 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mean_5, 1e-06), kwargs = {})
    %rsqrt_5 : [num_users=1] = call_function[target=torch.ops.aten.rsqrt.default](args = (%add_1259,), kwargs = {})
    %mul_938 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%transpose_8, %rsqrt_5), kwargs = {})
    %_to_copy_22 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_938,), kwargs = {dtype: torch.float16})
    %mul_945 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%_to_copy_22, %p_module_single_transformer_blocks_0_attn_norm_k_weight), kwargs = {})
    %_to_copy_23 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_919,), kwargs = {dtype: torch.float32})
    %view_48 : [num_users=2] = call_function[target=torch.ops.aten.view.default](args = (%_to_copy_23, [%sym_size_int_63, 24, %add_1548, 64, 1, 2]), kwargs = {})
    %_to_copy_24 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%mul_945,), kwargs = {dtype: torch.float32})
    %view_49 : [num_users=2] = call_function[target=torch.ops.aten.view.default](args = (%_to_copy_24, [%sym_size_int_63, 24, %add_1548, 64, 1, 2]), kwargs = {})
    %select_11 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 0), kwargs = {})
    %select_12 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_48, 5, 0), kwargs = {})
    %mul_988 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_11, %select_12), kwargs = {})
    %select_13 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 1), kwargs = {})
    %select_14 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_48, 5, 1), kwargs = {})
    %mul_999 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_13, %select_14), kwargs = {})
    %add_1345 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_988, %mul_999), kwargs = {})
    %select_15 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 0), kwargs = {})
    %select_16 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_49, 5, 0), kwargs = {})
    %mul_1013 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_15, %select_16), kwargs = {})
    %select_17 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%unsqueeze_11, 5, 1), kwargs = {})
    %select_18 : [num_users=1] = call_function[target=torch.ops.aten.select.int](args = (%view_49, 5, 1), kwargs = {})
    %mul_1024 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%select_17, %select_18), kwargs = {})
    %add_1388 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_1013, %mul_1024), kwargs = {})
    %view_50 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1345, [%sym_size_int_63, 24, %add_1548, 128]), kwargs = {})
    %_to_copy_25 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_50,), kwargs = {dtype: torch.float16, layout: torch.strided, device: cuda:0})
    %view_51 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1388, [%sym_size_int_63, 24, %add_1548, 128]), kwargs = {})
    %_to_copy_26 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_51,), kwargs = {dtype: torch.float16, layout: torch.strided, device: cuda:0})
    %_scaled_dot_product_flash_attention_1 : [num_users=1] = call_function[target=torch.ops.aten._scaled_dot_product_flash_attention.default](args = (%_to_copy_25, %_to_copy_26, %transpose_9), kwargs = {scale: 0.08838834764831843})
    %getitem_39 : [num_users=1] = call_function[target=operator.getitem](args = (%_scaled_dot_product_flash_attention_1, 0), kwargs = {})
    %transpose_10 : [num_users=1] = call_function[target=torch.ops.aten.transpose.int](args = (%getitem_39, 1, 2), kwargs = {})
    %view_52 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%transpose_10, [%sym_size_int_63, -1, 3072]), kwargs = {})
    %_to_copy_27 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%view_52,), kwargs = {dtype: torch.float16})
    %cat_8 : [num_users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%_to_copy_27, %gelu_2], 2), kwargs = {})
    %unsqueeze_26 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%getitem_35, 1), kwargs = {})
    %view_53 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%cat_8, [%mul_1164, 15360]), kwargs = {})
    %t_25 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_single_transformer_blocks_0_proj_out_weight,), kwargs = {})
    %addmm_23 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_single_transformer_blocks_0_proj_out_bias, %view_53, %t_25), kwargs = {})
    %view_54 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_23, [%sym_size_int_68, %add_1548, 3072]), kwargs = {})
    %mul_1104 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%unsqueeze_26, %view_54), kwargs = {})
    %add_1459 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%cat_7, %mul_1104), kwargs = {})
    %clamp_1 : [num_users=1] = call_function[target=torch.ops.aten.clamp.default](args = (%add_1459, -65504, 65504), kwargs = {})
    %slice_21 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%clamp_1, 0, 0, 9223372036854775807), kwargs = {})
    %slice_22 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%slice_21, 1, %sym_size_int_64, 9223372036854775807), kwargs = {})
    %silu_5 : [num_users=1] = call_function[target=torch.ops.aten.silu.default](args = (%add_73,), kwargs = {})
    %_to_copy_28 : [num_users=1] = call_function[target=torch.ops.aten._to_copy.default](args = (%silu_5,), kwargs = {dtype: torch.float16})
    %t_26 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_norm_out_linear_weight,), kwargs = {})
    %addmm_24 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_norm_out_linear_bias, %_to_copy_28, %t_26), kwargs = {})
    %split_3 : [num_users=2] = call_function[target=torch.ops.aten.split.Tensor](args = (%addmm_24, 3072, 1), kwargs = {})
    %getitem_48 : [num_users=1] = call_function[target=operator.getitem](args = (%split_3, 0), kwargs = {})
    %getitem_49 : [num_users=1] = call_function[target=operator.getitem](args = (%split_3, 1), kwargs = {})
    %native_layer_norm_5 : [num_users=1] = call_function[target=torch.ops.aten.native_layer_norm.default](args = (%slice_22, [3072], None, None, 1e-06), kwargs = {})
    %getitem_50 : [num_users=1] = call_function[target=operator.getitem](args = (%native_layer_norm_5, 0), kwargs = {})
    %add_1503 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%getitem_48, 1), kwargs = {})
    %slice_23 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%add_1503, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_27 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_23, 1), kwargs = {})
    %slice_24 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%unsqueeze_27, 2, 0, 9223372036854775807), kwargs = {})
    %mul_1143 : [num_users=1] = call_function[target=torch.ops.aten.mul.Tensor](args = (%getitem_50, %slice_24), kwargs = {})
    %slice_25 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%getitem_49, 0, 0, 9223372036854775807), kwargs = {})
    %unsqueeze_28 : [num_users=1] = call_function[target=torch.ops.aten.unsqueeze.default](args = (%slice_25, 1), kwargs = {})
    %slice_26 : [num_users=1] = call_function[target=torch.ops.aten.slice.Tensor](args = (%unsqueeze_28, 2, 0, 9223372036854775807), kwargs = {})
    %add_1533 : [num_users=1] = call_function[target=torch.ops.aten.add.Tensor](args = (%mul_1143, %slice_26), kwargs = {})
    %mul_1157 : [num_users=1] = call_function[target=operator.mul](args = (%sym_size_int_63, 4096), kwargs = {})
    %view_55 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%add_1533, [%mul_1157, 3072]), kwargs = {})
    %t_27 : [num_users=1] = call_function[target=torch.ops.aten.t.default](args = (%p_module_proj_out_weight,), kwargs = {})
    %addmm_25 : [num_users=1] = call_function[target=torch.ops.aten.addmm.default](args = (%p_module_proj_out_bias, %view_55, %t_27), kwargs = {})
    %view_56 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%addmm_25, [%sym_size_int_63, 4096, 64]), kwargs = {})
    return (view_56,)
1 Free mem: 11680415744, Total mem: 47736619008
2 Free mem: 11680415744, Total mem: 47736619008
===========HERE 1=======, query shape:  torch.Size([s6, 24, s2 + 4096, 128])
======== HERE 2 ========, key shape:  torch.Size([s6, 24, s2 + 4096, 128])
===========HERE 1=======, query shape:  torch.Size([s6, 24, s2 + 4096, 128])
======== HERE 2 ========, key shape:  torch.Size([s6, 24, s2 + 4096, 128])
Traceback (most recent call last):
  File "/work/TensorRT/examples/dynamo/run_2.py", line 144, in <module>
    trt_model = torch_tensorrt.dynamo.compile(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch_tensorrt/dynamo/_compiler.py", line 666, in compile
    exported_program = exported_program.run_decompositions(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/export/exported_program.py", line 128, in wrapper
    return fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/export/exported_program.py", line 1268, in run_decompositions
    return _decompose_exported_program(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/export/exported_program.py", line 742, in _decompose_exported_program
    gm, new_graph_signature = _decompose_and_get_gm_with_new_signature_constants(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/export/exported_program.py", line 454, in _decompose_and_get_gm_with_new_signature_constants
    aten_export_artifact = _export_to_aten_ir(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/export/_trace.py", line 741, in _export_to_aten_ir
    gm, graph_signature = transform(aot_export_module)(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1332, in aot_export_module
    fx_g, metadata, in_spec, out_spec = _aot_export_function(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1571, in _aot_export_function
    fx_g, meta = create_aot_dispatcher_function(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 580, in create_aot_dispatcher_function
    return _create_aot_dispatcher_function(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 830, in _create_aot_dispatcher_function
    compiled_fn, fw_metadata = compiler_fn(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/jit_compile_runtime_wrappers.py", line 118, in aot_dispatch_export
    graph, _, _ = aot_dispatch_base_graph(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/dispatch_and_compile_graph.py", line 153, in aot_dispatch_base_graph
    fw_module = _create_graph(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/dispatch_and_compile_graph.py", line 55, in _create_graph
    fx_g = make_fx(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 2196, in wrapped
    return make_fx_tracer.trace(f, *args)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 2134, in trace
    return self._trace_inner(f, *args)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 2105, in _trace_inner
    t = dispatch_trace(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_compile.py", line 32, in inner
    return disable_fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 751, in _fn
    return fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1138, in dispatch_trace
    graph = tracer.trace(root, concrete_args)  # type: ignore[arg-type]
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1694, in trace
    res = super().trace(root, concrete_args)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 843, in trace
    (self.create_arg(fn(*args)),),
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1193, in wrapped
    out = f(*tensors)  # type:ignore[call-arg]
  File "<string>", line 1, in <lambda>
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/traced_function_transforms.py", line 693, in inner_fn
    outs = fn(*args)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/traced_function_transforms.py", line 413, in _functionalized_f_helper
    f_outs = fn(*f_args)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/traced_function_transforms.py", line 78, in inner_fn
    outs = fn(*args)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/utils.py", line 184, in flat_fn
    tree_out = fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/traced_function_transforms.py", line 875, in functional_call
    out = PropagateUnbackedSymInts(mod).run(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/interpreter.py", line 167, in run
    self.env[node] = self.run_node(node)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 6795, in run_node
    result = super().run_node(n)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/interpreter.py", line 230, in run_node
    return getattr(self, n.op)(n.target, args, kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/interpreter.py", line 310, in call_function
    return target(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_ops.py", line 722, in __call__
    return self._op(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1241, in __torch_function__
    return func(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_ops.py", line 722, in __call__
    return self._op(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py", line 527, in __torch_dispatch__
    outs_unwrapped = func._op_dk(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1343, in __torch_dispatch__
    return proxy_call(self, func, self.pre_dispatch, args, kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 912, in proxy_call
    out = func(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_ops.py", line 722, in __call__
    return self._op(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1276, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1817, in dispatch
    return self._cached_dispatch_impl(func, types, args, kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1387, in _cached_dispatch_impl
    output = self._dispatch_impl(func, types, args, kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 2291, in _dispatch_impl
    decomposition_table[func](*args, **kwargs)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_refs/__init__.py", line 4674, in view
    return _reshape_view_helper(a, *shape, allow_copy=False)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/torch/_refs/__init__.py", line 3807, in _reshape_view_helper
    raise ValueError(msg)
ValueError: Cannot view a tensor with shape torch.Size([s6, s2 + 4096, 24, 128]) and strides (3072*s2 + 12582912, 128, 128*s2 + 524288, 1) as a tensor with shape (s1, (s6*(s2 + 4096)//s1), 3072)!

While executing %view_52 : [num_users=1] = call_function[target=torch.ops.aten.view.default](args = (%transpose_10, [%sym_size_int_63, -1, 3072]), kwargs = {})
Original traceback:
File "/work/TensorRT/examples/dynamo/run_2.py", line 48, in forward
    return self.module.forward(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 438, in forward
    hidden_states = block(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/diffusers/models/transformers/transformer_flux.py", line 119, in forward
    attn_output = self.attn(
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 490, in forward
    return self.processor(