Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: The shape of the 2D attn_mask is torch.Size([77, 77]), but should be (1, 1). #31

Open
aulaywang opened this issue Jul 16, 2024 · 2 comments

Comments

@aulaywang
Copy link

When I use the command sh configs/inference/run.sh, this error occurs:

Traceback (most recent call last):                                                                                                                                                                                     
  File "/data/code/MotionCtrl/main/evaluation/motionctrl_inference.py", line 354, in <module>                                                                              
    run_inference(args, gpu_num, rank)                                                                                                                                                                                 
  File "/data/code/MotionCtrl/main/evaluation/motionctrl_inference.py", line 261, in run_inference                                                                         
    batch_samples = motionctrl_sample(
  File "/data/code/MotionCtrl/main/evaluation/motionctrl_inference.py", line 146, in motionctrl_sample
    cond = model.get_learned_conditioning(prompts)
  File "/data/code/MotionCtrl/main/evaluation/../../lvdm/models/ddpm3d.py", line 552, in get_learned_conditioning
    c = self.cond_stage_model.encode(c)
  File "/data/code/MotionCtrl/main/evaluation/../../lvdm/modules/encoders/condition2.py", line 238, in encode
    return self(text)
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/code/MotionCtrl/main/evaluation/../../lvdm/modules/encoders/condition2.py", line 215, in forward
    z = self.encode_with_transformer(tokens.to(self.device))
  File "/data/code/MotionCtrl/main/evaluation/../../lvdm/modules/encoders/condition2.py", line 222, in encode_with_transformer
    x = self.text_transformer_forward(x, attn_mask=self.model.attn_mask)
  File "/data/code/MotionCtrl/main/evaluation/../../lvdm/modules/encoders/condition2.py", line 234, in text_transformer_forward
    x = r(x, attn_mask=attn_mask)
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/open_clip/transformer.py", line 263, in forward
    x = q_x   self.ls_1(self.attention(q_x=self.ln_1(q_x), k_x=k_x, v_x=v_x, attn_mask=attn_mask))
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/open_clip/transformer.py", line 250, in attention
    return self.attn(
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/torch/nn/modules/activation.py", line 1167, in forward
    attn_output, attn_output_weights = F.multi_head_attention_forward(
  File "/root/miniconda3/envs/motionctrl/lib/python3.10/site-packages/torch/nn/functional.py", line 5069, in multi_head_attention_forward
    raise RuntimeError(f"The shape of the 2D attn_mask is {attn_mask.shape}, but should be {correct_2d_size}.")
RuntimeError: The shape of the 2D attn_mask is torch.Size([77, 77]), but should be (1, 1).
@aulaywang
Copy link
Author

I have solved this problem by link
pip install open-clip-torch==2.24.0

@ShiMinghao0208
Copy link

I have solved this problem by link pip install open-clip-torch==2.24.0

When running the SDXL source code, I was mad by this bug, thanks!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants