Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] AttributeError: can't set attribute when training chatglm2-6b #2706

Closed
wangshuai09 opened this issue Nov 21, 2023 · 0 comments · Fixed by #2710
Closed

[BUG] AttributeError: can't set attribute when training chatglm2-6b #2706

wangshuai09 opened this issue Nov 21, 2023 · 0 comments · Fixed by #2710

Comments

@wangshuai09
Copy link
Contributor

Describe the bug
Training chatglm2-6b reported a AttributeError.
Traceback (most recent call last): File "fastchat/train/train.py", line 304, in <module> train() File "fastchat/train/train.py", line 283, in train tokenizer.pad_token = tokenizer.unk_token AttributeError: can't set attribute

To Reproduce
Model download form huggingface-chatglm2-6b
Script is
torchrun --nproc_per_node=4 --master_port=20001 fastchat/train/train.py \ --model_name_or_path /home/xxx/models/chatglm2-6b \ --data_path /home/xxx/datasets/evol-instruct-chinese/evol-instruct-chinese-1024-subset.json \ --fp16 True \ --output_dir output_chatglm \ --num_train_epochs 5 \ --per_device_train_batch_size 8 \ --per_device_eval_batch_size 1 \ --gradient_accumulation_steps 1 \ --evaluation_strategy "no" \ --save_strategy "epoch" \ --learning_rate 5e-5 \ --weight_decay 0. \ --lr_scheduler_type "cosine" \ --logging_steps 1 \ --fsdp "full_shard auto_wrap" \ --model_max_length 512 \ --gradient_checkpointing True \ --lazy_preprocess True

Reason
In chatglm2-6b code, pad_token is readonly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant