Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BedrockRuntimeClient ConverseStreamCommand can't receive any response in react native #6394

Open
3 tasks done
zhu-xiaowei opened this issue Aug 19, 2024 · 3 comments
Open
3 tasks done
Assignees
Labels
bug This issue is a bug. p2 This is a standard priority issue response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days.

Comments

@zhu-xiaowei
Copy link

zhu-xiaowei commented Aug 19, 2024

Checkboxes for prior research

Describe the bug

When using BedrockRuntimeClient in React Native environment, InvokeModelCommand works well, but ConverseStreamCommand and InvokeModelWithResponseStreamCommand will not return any response.

And I've read the getting-started for react native and added following imports

import "react-native-get-random-values";
import "react-native-url-polyfill/auto";
import "web-streams-polyfill/dist/polyfill";

then added blow code in metro.config.js

const config = {
  transformer: {
    getTransformOptions: async () => ({
      transform: {
        experimentalImportSupport: true, // false is still not work. 
        inlineRequires: false,
      },
    }),
  },
};

module.exports = mergeConfig(getDefaultConfig(__dirname), config);

the bedrock code as:

const command = new ConverseStreamCommand({
  messages: messages,
  modelId,
});
const apiResponse = await client.send(command);
console.log(JSON.stringify(apiResponse.stream));

for await (const item of apiResponse.stream) {
  console.log(item);
  if (item.contentBlockDelta) {
    const text = item.contentBlockDelta.delta?.text;
    console.log(text);
  }
}

client.send(command) works well, and the log for apiResponse.stream is:

{"options":{"messageStream":{"options":{"inputStream":{},"decoder":{"headerMarshaller":{},"messageBuffer":[],"isEndOfStream":false}}}}}

The for await block never goes inside, then I try to remove the transformer in metro.config.js and the apiResponse.stream will return the following error:

 WARN  Invalid responseType: blob
 ERROR [TypeError: Cannot read property 'blobId' of undefined]
 WARN  Invalid responseType: blob
 WARN  Invalid responseType: blob
 WARN  Invalid responseType: blob

I don't know what configuration I'm missing or is there any sample code for calling methods of ConverseStreamCommandor InvokeModelWithResponseStreamCommand in a React Native environment?

SDK version number

"@aws-sdk/client-bedrock-runtime": "^3.614.0"

Which JavaScript Runtime is this issue in?

React Native

Details of the browser/Node.js/ReactNative version

"react-native": "0.74.5"

Reproduction Steps

see the above describe.

Observed Behavior

ConverseStreamCommand command send successfully but apiResponse.stream receive without any streaming response.

Expected Behavior

can receive the streaming response for ConverseStreamCommand

Possible Solution

No response

Additional Information/Context

The credentials are configured correctly and the same code runs perfectly in the node environment.

@zhu-xiaowei zhu-xiaowei added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Aug 19, 2024
@aBurmeseDev aBurmeseDev self-assigned this Aug 20, 2024
@aBurmeseDev
Copy link
Member

aBurmeseDev commented Aug 20, 2024

HI @zhu-xiaowei - thanks for reaching out.

It sounds like SDK call was made successfully as you receive the response from client but not from apiResponse.stream. I quickly attempted to reproduce it in Node env with simple code below and wasn't able to reproduce.

import {
  BedrockRuntimeClient,
  ConverseStreamCommand,
} from "@aws-sdk/client-bedrock-runtime"; // ES Modules import
const client = new BedrockRuntimeClient(config);
const input = {
  modelId: "STRING_VALUE",
  messages: messages,
};
const command = new ConverseStreamCommand(input);
const response = await client.send(command);

Before i attempt to reproduce with React Native, can you clarify a few things:

  • Which Bedrock model are you using?
  • Have you had a chance to try it in different env like Node or browser?
  • Are you using Expo when setting up React Native? Can you share steps you took to set up your React Native env?

@aBurmeseDev aBurmeseDev added response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Aug 20, 2024
@zhu-xiaowei
Copy link
Author

Hi @aBurmeseDev, thanks for working on this issue, here are more details:

  1. I am using the anthropic.claude-3-sonnet-20240229-v1:0 model, it looks like this issue is encountered in all straming modes, not limited to a certain model in React Native env.
  2. Node and browser work fine.
  3. I am using a bare React Native environment created with react-native-cli, see the official documentation Getting Started Without a Framework, we are not using Expo.

@zhu-xiaowei
Copy link
Author

zhu-xiaowei commented Aug 21, 2024

To correct, the error above is because I also added another polyfill: import 'react-native-polyfill-globals/auto'; After removing this polyfill, error still appears, and the error content is:

[TypeError: Object is not async iterable]

Then I tried to add polyfill import '@azure/core-asynciterator-polyfill'; After adding this polyfill, the error becomes:

[TypeError: source[Symbol.asyncIterator] is not a function (it is undefined)]

In short, I can't find a way to successfully read the results returned by the straming model in the React Native environment:

const apiResponse = await client.send(command);

for await (const item of apiResponse.stream) {
  console.log(item); // can't get the streaming response.
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p2 This is a standard priority issue response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days.
Projects
None yet
Development

No branches or pull requests

2 participants