Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ValueError when Loading Qwen2-VL Model with Liger Kernel #249

Open
rahatarinasir opened this issue Sep 16, 2024 · 1 comment
Open

ValueError when Loading Qwen2-VL Model with Liger Kernel #249

rahatarinasir opened this issue Sep 16, 2024 · 1 comment

Comments

@rahatarinasir
Copy link

rahatarinasir commented Sep 16, 2024

🐛 Describe the bug

I'm encountering a ValueError when trying to load the Qwen2-VL model using the AutoLigerKernelForCausalLM class from the Liger Kernel. The error message indicates an unrecognized configuration class for this model.

Reproduce

Clone the repository:
bash

git clone https://github.com/linkedin/Liger-Kernel.git
cd Liger-Kernel
pip install -e .

Install necessary packages:
python

from huggingface_hub import login
hf_token = input("Enter your Hugging Face token: ")
login(token=hf_token)
pip install torch
pip install triton # >= 2.3.0
pip install liger-kernel
pip install liger-kernel-nightly
pip install git https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830

Attempt to load the model:
python

from transformers import Qwen2VLConfig, Qwen2VLForConditionalGeneration
from liger_kernel.transformers import AutoLigerKernelForCausalLM

config = Qwen2VLConfig.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")
model = AutoLigerKernelForCausalLM.from_pretrained("Qwen/Qwen2-VL-2B-Instruct", config=config)

Error Message
angelscript

ValueError: Unrecognized configuration class <class 'transformers.models.qwen2_vl.configuration_qwen2_vl.Qwen2VLConfig'> for this kind of AutoModel: AutoLigerKernelForCausalLM.
Model type should be one of [list of model types].

It seems that the AutoLigerKernelForCausalLM class does not recognize the Qwen2VLConfig. Is there a workaround or an update planned to support this model? Any guidance would be greatly appreciated!

Versions

Environment Report:

Operating System: Linux-6.1.85 -x86_64-with-glibc2.35
Python version: 3.10.12
PyTorch version: 2.4.0 cu121
CUDA version: 12.1
Triton version: 3.0.0
Transformers version: 4.45.0.dev0

@tyler-romero
Copy link
Collaborator

tyler-romero commented Sep 16, 2024

Qwen2-VL isnt based off of AutoModelForCausalLM, so it cant be loaded with AutoLigerKernelForCausalLM. Instead its a "ForConditionalGeneration" model, which doesn't have an AutoModel parent class.

As a workaround, I'd recommend using the monkeypatch function apply_liger_kernel_to_qwen2_vl after from transformers import Qwen2VLForConditionalGeneration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants