Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to generate ONNX graph for GPT-2. #16

Closed
JongminKim-KU opened this issue Dec 9, 2024 · 1 comment
Closed

Unable to generate ONNX graph for GPT-2. #16

JongminKim-KU opened this issue Dec 9, 2024 · 1 comment
Assignees
Labels
bug Something isn't working documentation Improvements or additions to documentation help wanted Extra attention is needed

Comments

@JongminKim-KU
Copy link

Hello, I'm Jongmin Kim, a student studying DNN accelerator, and I have been interested in your simulator.

According to the README, the following command should generate an ONNX graph for GPT-2:
python3 ./scripts/generate_transformer_onnx.py --model gpt2

However, it results in the following error message:

File "/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py", line 2007, in _run_symbolic_function
raise errors.UnsupportedOperatorError(
torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 11 is not supported. Support for this operator was added in version 14, try exporting with this version.
Traceback (most recent call last):

I have verified that the version is 22, and I don’t believe this is the issue, as the latest versions of onnx and onnxruntime are installed, as specified in the Dockerfile.

I am currently attempting to export ONNX graphs for GPT-2, Llama 3, and OPT using the ONNX optimizer (AutoModelForXXX, ORTModelForXXX, etc.).
However, I’m wondering if there are any simpler or more efficient methods to resolve this issue.

Thank you for your time, and I look forward to your guidance.

@YWHyuk YWHyuk self-assigned this Dec 10, 2024
@YWHyuk YWHyuk added bug Something isn't working documentation Improvements or additions to documentation help wanted Extra attention is needed labels Dec 10, 2024
@YWHyuk YWHyuk closed this as completed in 3cfd369 Dec 10, 2024
@YWHyuk
Copy link
Collaborator

YWHyuk commented Dec 10, 2024

Hi @JongminKim-KU ,

This issue was caused by a version mismatch and has been fixed and uploaded. Additionally, a guide for the LLM mode has been added to the README.

If you encounter any further issues, please don't hesitate to let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants