-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Llama 3.1] Currently only named tools are supported error when using it for llama 3.1 when not setting toolChoice or using toolChoice: 'auto' #2503
Comments
Afaik OpenAI does not offer Llama 3.1. The error you see comes from a specific provider. Which provider are you using? |
I am using the steps provided on vercel sdk page
This should work according to documentation but it doesnt because of the toolChoice parameter |
@vishalsaugat can you try removing the tool function call, see #2513 |
@lgrammel Yes I have tried that too, dint work. |
@vishalsaugat I believe this might be caused by some changes on the groq side. I've reached out to them. Can you try the 8b model instead? |
Hey @vishalsaugat - would you be able to upload your repository so I can take a look? I've just run all of the code snippets from the guide again (including the |
Ok, so I guess its because groq is handling this on its own side and not sending |
This is my code
This code works if I also pass |
@lgrammel It will work with Llama 8b. Is it possible to add the support for above mentioned ? |
UPDATE: same error with llama8b also |
Description
I am using llama3.1 model using
@ai-sdk/openai
package.I get this error
when using 'toolchoice': 'auto' in streamUI or not set it at all.
On reading further I found out that llama doesnt support "tool_choice": "auto" parameter in its API. If its set to null it automatically gets converted to auto.
https://github.com/vercel/ai/blob/f7a94535f1d8b8a6f4179d7f5cd762389ef6de4b/packages/core/core/prompt/prepare-tools-and-tool-choice.ts#L37C21-L37C25
Solution: - support null as parameter which doesnt send
tool-choice
to Llama or some other models APICode example
The text was updated successfully, but these errors were encountered: