Calling AI Functions #10

nicolascepeda · 2023-09-18T08:23:43Z

nicolascepeda
Sep 18, 2023

Hi,

thanks for the great work with Kani. I decided to use it on my AI projects as it is a very modulare, low on boilerplate & very well structured project. Great Kudos for the work so far.

I have a question, I am trying to use AI functions to retunr structured JSON. This is the code I am using calling directly OpenAI api:

import openai
from pydantic import BaseModel, Field
import json

class Sentiment(BaseModel):
    sentiment: str = Field(description="Sentiment of the review")
    sentiment_score: int = Field(description="Score between 1-100 of the sentiment")

def analyse_reviews(user_input):
    prompt = f"""
    {user_input}
    Analyse the sentiment of the review above.
    """
    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user", "content": prompt},
        ],
        functions=[{"name": "dummy_fn_set_sentiment", "parameters": Sentiment.model_json_schema()}],
    )
    try:
        generated_text = completion.choices[0].message.function_call.arguments
        return json.loads(generated_text)
    except Exception as e:
        print(f"An error occurred: {e}")
        return None

sentiment = analyse_reviews("Possibly the worst book I've ever read.It's a huge collection of biases for all the possible countries and cultures. The whole book is structured with examples like: if you are working with Chinese people, you should take this approach, instead if your team is composed by German people you should do this etc....")
sentiment

I don't manage to have kani calling the same APIs. I want to be able to define the schema of the function as a Pydantic object. Is this somethign that is not supported at all at the moment? I have tryied everything but the function doesn't get called.

Running out of ideas, any inputs would be highly appreciated. Thanks!

class MyKani(Kani):
    @ai_function(json_schema = Sentiment.model_json_schema())
    def dummy_fn_set_sentiment(self, sentiment : Sentiment):
        return

    async def get_model_completion(self, *args, **kwargs) -> BaseCompletion:
        completion = await super().get_model_completion(*args, **kwargs)
        print("completion: ", completion)
        return completion

engine = OpenAIEngine(os.environ["OPENAI_API_KEY"], model="gpt-3.5-turbo", temperature=0)
ai = MyKani(engine, system_prompt="You are a helpful review analysis tool.")

query = f"""
    Possibly the worst book I've ever read.It's a huge collection of biases for all the possible countries and cultures. The whole book is structured with examples like: if you are working with Chinese people, you should take this approach, instead if your team is composed by German people you should do this etc....

    Analyse the sentiment of the reviews above"""
message = await ai.chat_round_str(query)

nicolascepeda · 2023-09-18T08:29:27Z

nicolascepeda
Sep 18, 2023
Author

What bugs me the most is that even the examples in the documentation doesn't seem to work. The print in the function never gets called, and this is the return of the message:

role=<ChatRole.ASSISTANT: 'assistant'> content="I'm sorry, I am an AI language model and I do not have real-time data. To find out the current weather in Buenos Aires, you can check a reliable weather website or use a weather app for the most up-to-date information." name=None function_call=None

import enum
from typing import Annotated

from kani import AIParam, Kani, ai_function, chat_in_terminal
from kani.engines.openai import OpenAIEngine
import os

engine = OpenAIEngine(os.environ["OPENAI_API_KEY"], model="gpt-3.5-turbo")

class Unit(enum.Enum):
    FAHRENHEIT = "fahrenheit"
    CELSIUS = "celsius"

class MyKani(Kani):
    @ai_function()
    def get_weather(
        self,
        location: Annotated[str, AIParam(desc="The city and state, e.g. San Francisco, CA")],
        unit: Unit,
    ):
        """Get the current weather in a given location."""
        # call some weather API, or just mock it for this example
        print("AI Function call")
        degrees = 72 if unit == Unit.FAHRENHEIT else 22
        return f"Weather in {location}: Sunny, {degrees} degrees {unit.value}."

ai = MyKani(engine)
message = await ai.chat_round_str("what is the weather in buenos aires?")
print(message)

0 replies

nicolascepeda · 2023-09-18T08:45:14Z

nicolascepeda
Sep 18, 2023
Author

Update:

Calling chat_round_str does not call AI Functions.

Calling the example from the documentation like below does indeed call AI functions.

async for item in ai.full_round("what is the weather in buenos aires?"):
    print(item)

I guess the only question is, is there a WAY to use pydantic objects to define the payload of the funcitons?

0 replies

nicolascepeda · 2023-09-18T08:52:59Z

nicolascepeda
Sep 18, 2023
Author

Update:

Code below uses the example inclusive Pydantic objects.

class Unit(enum.Enum):
    FAHRENHEIT = "fahrenheit"
    CELSIUS = "celsius"

class Weather(BaseModel):
    location: str = Field(description="The city and state, e.g. San Francisco, CA")
    unit : Unit


class MyKani(Kani):
    @ai_function(after= ChatRole.USER)
    def get_weather(
        self,
        weather : Weather
    ):
        """Get the current weather in a given location."""
        # call some weather API, or just mock it for this example
        print("AI Function call ", weather)
        #degrees = 72 if unit == Unit.FAHRENHEIT else 22
        return ""

ai = MyKani(engine)
async for item in ai.full_round("what is the weather in buenos aires?"):
    print("aha: " ,item)

0 replies

liamdugan · 2023-09-18T15:24:20Z

liamdugan
Sep 18, 2023
Collaborator

Hey @nicolascepeda - Thanks for the kind words! So the code you had originally was fine, only issue was using chat_round_str (we'll probably add some warning log message to chat_round_str in the future to help minimize confusion there).

A full code example (using json_schema= to initialize the ai function args) would look something like this:

from pydantic import BaseModel, Field
from kani import Kani, ai_function, full_round
from kani.engines.openai import OpenAIEngine

class Sentiment(BaseModel):
    sentiment: str = Field(description="Sentiment of the review")
    sentiment_score: int = Field(description="Score between 1-100 of the sentiment")

class MyKani(Kani):
    @ai_function(json_schema = Sentiment.model_json_schema(), after= ChatRole.USER)
    def analyze_sentiment(self, **kwargs):
        """ Output sentiment of review as a pydantic object """
        return Sentiment.model_validate(kwargs)

engine = OpenAIEngine(os.environ["OPENAI_API_KEY"], model="gpt-3.5-turbo", temperature=0)
ai = MyKani(engine, system_prompt="You are a helpful review analysis tool.")

review = "..."
query = f"{review}\nAnalyze the sentiment of the review above"
messages = [item async for item in ai.full_round(query)]

Then parse the messages to get the return of the function call

As a quick note, your example for get_weather requires the model to construct the Pydantic object to pass into the function. This may result in less accurate function calling (especially if certain models aren't properly fine-tuned to do this). Thus we recommend initializing the arguments of your function with your pydantic schema instead and re-constructing the object within the function using model_validate()

1 reply

zhudotexe Sep 18, 2023
Maintainer

We added a warning to Kani.chat_round if you have functions defined. Thanks for the feedback!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Calling AI Functions #10

{{title}}

Replies: 4 comments 1 reply

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Calling AI Functions #10

nicolascepeda Sep 18, 2023

Replies: 4 comments · 1 reply

nicolascepeda Sep 18, 2023 Author

nicolascepeda Sep 18, 2023 Author

nicolascepeda Sep 18, 2023 Author

liamdugan Sep 18, 2023 Collaborator

zhudotexe Sep 18, 2023 Maintainer

nicolascepeda
Sep 18, 2023

Replies: 4 comments 1 reply

nicolascepeda
Sep 18, 2023
Author

nicolascepeda
Sep 18, 2023
Author

nicolascepeda
Sep 18, 2023
Author

liamdugan
Sep 18, 2023
Collaborator

zhudotexe Sep 18, 2023
Maintainer