gsm8k cleanup #6
Conversation
| @@ -15,7 15,7 @@ | |||
| "url": os.getenv("AZURE_OPENAI_EMBEDDINGS_URL"), | |||
| }, | |||
| "azure": { | |||
| "headers": {"Authorization": f"Bearer {os.getenv('AZURE_OPENAI_CHAT_API_KEY')}"}, | |||
| "headers": {"api-key": f"{os.getenv('AZURE_OPENAI_CHAT_API_KEY')}"}, | |||
There was a problem hiding this comment.
Do we document anywhere exactly which values should go into these environment variables? I'm guessing that this is the key which users can get from the AOAI playground if they select 'View Code' ?
There was a problem hiding this comment.
Yes, that's the right key. It's also visible from a keys panel in the portal. We will document necessary environment variables in the readme.
| @@ -8,16 8,6 @@ | |||
|
|
|||
| my_path = pathlib.Path(__file__).parent.resolve() | |||
|
|
|||
| ds = load_dataset("gsm8k", "main")["test"] | |||
There was a problem hiding this comment.
I approve of not doing this on import :-)
| @@ -67,17 57,26 @@ def solve(idx): | |||
| break | |||
|
|
|||
| if answer: | |||
| with open(my_path.parent / "datasets" / "gsm8k.jsonl", "a") as f: | |||
| with open(my_path.parent / "generations" / "gsm8k.jsonl", "a") as f: | |||
There was a problem hiding this comment.
Should the 'generations' directory be added to gitignore?
There was a problem hiding this comment.
I think it should. I'll add that. I do want to keep the readme in there to document the purpose of that directory though.
Fixes needed to make the gsm8k benchmark run against Azure OpenAI GPT-4 chat model.
Also contains some additional small changes:
generations/directory which will be used to store the intermediate outputs moving forward