Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Override maxToolRoundtrips per tool call #2693

Open
blechatellier opened this issue Aug 16, 2024 · 2 comments · May be fixed by #2757
Open

Override maxToolRoundtrips per tool call #2693

blechatellier opened this issue Aug 16, 2024 · 2 comments · May be fixed by #2757
Labels
ai/ui enhancement New feature or request

Comments

@blechatellier
Copy link
Contributor

blechatellier commented Aug 16, 2024 •
edited

Feature Description

Following this Twitter discussion: https://x.com/blechatellier/status/1823688077006790917 with @lgrammel

It would be great to allow setting a tool call result (on either the server or client) while controlling whether to automatically trigger another call to the LLM.

This use case involves rendering a custom UI component on the client using the tool call result, without needing a roundtrip to the LLM. However, it should still allow multiple roundtrips for other tools.

One possible solution could be overriding maxToolRoundtrips per tool call, or passing a flag to addToolResult on the client side to prevent triggering another LLM call. A similar mechanism would be needed on the server if the tool call is running via the execute method.

Or maybe exposing triggerRequest so I can recall the LLM for some specific tools only.
@blechatellier blechatellier changed the title Override max tool call roundtrip per tool Override maxToolRoundtrips per tool call Aug 16, 2024
@lgrammel lgrammel added enhancement New feature or request ai/ui labels Aug 16, 2024
@blechatellier
Copy link
Contributor Author

blechatellier commented Aug 20, 2024 •
edited

Solved it for now by setting maxToolRoundtrips: 0 and re-submitting for some tool calls only.

interface UseToolRoundtripsParams {
  handleSubmit: (
    event?: {
      preventDefault?: () => void;
    },
    chatRequestOptions?: ChatRequestOptions,
  ) => void;
  messages: Message[];
  toolNames: string[];
}

export const useToolRoundtrips = ({
  handleSubmit,
  messages,
  toolNames,
}: UseToolRoundtripsParams) => {
  useEffect(() => {
    const lastMessage = messages[messages.length - 1];

    for (const toolInvocation of lastMessage?.toolInvocations || []) {
      if (toolInvocation.state !== 'result') {
        continue;
      }
      if (toolNames.includes(toolInvocation.toolName)) {
        handleSubmit(undefined, { allowEmptySubmit: true });
      }
    }
  }, [messages]);
};

@blechatellier blechatellier linked a pull request Aug 21, 2024 that will close this issue
Open
@blechatellier
Copy link
Contributor Author

Another convo to do that on the server #1943

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ai/ui enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: exclude tools from automatic roundtrips blechatellier/ai
2 participants
@lgrammel @blechatellier