AI_TypeValidationError error with streamText().toAIStream() #2656
Comments
|
@marclelamy can you post the details of the error that you are seeing (from the console/terminal)? |
|
Oh my bad I forgot to paste it. |
|
it seems that the an error response chunk is incorrectly parsed. this is provider specific. which provider are you using? can you add your provider initialization code (in particular if you are using the openai provider with a non-openai service)? |
|
@lgrammel For non OpenAI/Perplexity models I'm using OpenRouter. Here's most of the code. The strange thing is that the error doesn't get caught in the try/catch. What better way should I implement to fix/catch that? 'use server';
import { streamText } from 'ai';
import { createOpenAI } from '@ai-sdk/openai';
import { FormattedRequestObject, User } from "@/lib/types"
// instantiating clients
const openaiClient = createOpenAI({
apiKey: process.env.OPENAI_API_KEY || "",
})
const openrouterClient = createOpenAI({
baseURL: "https://openrouter.ai/api/v1",
apiKey: process.env.OPENROUTER_API_KEY || "",
headers: {
"HTTP-Referer": "tryblend.ai",
"X-Title": "blendai",
},
})
const perplexityClient = createOpenAI({
apiKey: process.env.PERPLEXITY_API_KEY || "",
baseURL: "https://api.perplexity.ai",
})
// specify type returned from openai
export async function generate(data: FormattedRequestObject): Promise<any> {
// does some pre request stuff
// Setting stream value
const stream = createStreamableValue('');
const processStream = async () => {
// setting client based on model provider
let client
if (selectedModel.provider === "openai") {
client = openaiClient
} else if (selectedModel.provider === "openrouter") {
client = openrouterClient
} else if (selectedModel.provider === "perplexity") {
client = perplexityClient
} else {
return { error: "Invalid model provider" }
}
// Run model
let error: any = false
const result = await streamText({
model: client(selectedModel.providerModelId as string),
messages: messages as any[],
onFinish: async (final) => {
// does some other stuff
}
});
// process stream
let firstToken: any = undefined
result.toAIStream({
onStart: () => {
console.log('on start')
},
onToken: (token) => {
if (firstToken === undefined) {
firstToken = token
console.log('on token')
}
},
});
// Send each chunk one by one to the client
for await (const delta of result.textStream) {
stream.update(delta);
}
// close stream
stream.done();
};
// process stream
try {
processStream();
} catch (error: any) {
return { error: {message: error.message, status: error.status} }
}
return { output: stream.value }
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
When running models like llama or mistral, I'm getting the type validation error but it's not always happening, sometimes the completion runs just fine.
In the code below, I see the onStart() log and the error happens right after and the first token doesn't get looged
Code example
Additional context
No response
The text was updated successfully, but these errors were encountered: