-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
'NaN' token counts when using streamText with Azure OpenAI models #2543
Comments
It seems that token counts for streaming are currently not supported by Azure OpenAI: https://learn.microsoft.com/en-us/answers/questions/1805363/azure-openai-streaming-token-usage |
@lgrammel Not only related to Azure, but it's also not working for OpenAI/Google/Mistral, fireworks works when using openai with fireworks basepath... When I use
But when I use
|
@trulymittal how are you setting up the providers in the registry? |
Double-checked Mistral and it provides the token usage information, try e.g. https://github.com/vercel/ai/blob/main/examples/ai-core/src/stream-text/mistral.ts |
@lgrammel apologies, it was my mistake not to be using the strict compatibility mode. Thanks for this. |
Description
I am using streamText with the Azure OpenAI provider for the AI SDK and there models. I use createAzure to create the provider instance. When I try to get the token counts, I get NaN values.(both in API endpoint
onFinish()
anduseChat({onFinish:() => {}})
)log
Code example
Additional context
The text was updated successfully, but these errors were encountered: