• I have searched the existing issues

Current behavior

When starting a conversation that requires front-end code in the form of HTML and CSS, a weird behavior starts to occur at the token level and the UI Level. For example, some tokens are missing in the UI and in the respective response found in the ~/jan/threads/jan_*/threads.jsonl files.

For simplicity, I am going to first highlight the details of the request, and then proceed to explain what I've tried to identify this behavior.

• system prompt: You are a helpful web development instructor that knows the ins and outs of JavaScript, has created and deployed thousands of websites and web apps, and you are here to help me learn how to become a front-end developer. Please keep in mind that my favorite colors are black and the same green as the Green Lantern and must of the things we create together should have these colors.
• user prompt: I have never done any web development but would love to start by building a straightforward website with HTML and CSS and deploying it to GitHub pages or Vercel. Please walk me through the process of accomplishing this in a step-by-step way and everything should be in 1 file to start with. I want this first website to be about teaching people how to meditate. It should start by explaining to people what meditation is and how it works, then move on to suggestions of the first steps to take to meditate daily. The website should have placeholders for images. I will find these later.
• parameters
  • temperature: 1
  • max_tokes: 4096
  • top p: 0.95
  • stream: true
  • frequency and presence penalty: 0
• model: GPT-4o

Jan Behavior

• code response is missing tokens
  
• UI elements break down
  

I validated this behavior with 3 different tools

OpenAI Playground

• All tokens are shown and output is as expected
  

LibreChat

• All tokens are shown and output is as expected
  

HeyGPT.chat

• Breaks in the exact same way as Jan
  

OpenAI SDK

• The following is meant to be the same call Jan is doing but the response is not the same from the UI vs the SDK

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
  model="gpt-4o",
  messages=[
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "You are a helpful web development instructor that knows the ins and outs of JavaScript, has created and deployed thousands of websites and web apps, and you are here to help me learn how to become a front-end developer. Please keep in mind that my favorite colors are black and the same green as the Green Lantern and must of the things we create together should have these colors."
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "I have never done any web development but would love to start by building a straightforward website with HTML and CSS and deploying it to GitHub pages or Vercel. Please walk me through the process of accomplishing this in a step-by-step way and everything should be in 1 file to start with. I want this first website to be about teaching people how to meditate. It should start by explaining to people what meditation is and how it works, then move on to suggestions of the first steps to take to meditate daily. The website should have placeholders for images. I will find these later."
        }
      ]
    }
  ],
  temperature=1,
  max_tokens=4095,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0
)

Minimum reproduction step

1. Copy the prompts above
2. Open Jan
3. Add you OpenAI API keys
4. Start a new chat
5. Adjust the parameters to reflect the ones above
6. Send the request
7. To test it to the point of the UI breaking down even further, send these subsequent prompts

• Please explain every section in the HTML code you wrote and every line, tag, and format in it. Remember, I have not done a any web development before and need to understand all of the pieces of the puzzle.
• Since this is for meditation purposes, please recreate the html file with more pastel-like colors, center everything in the middle of the page and please pay attention to the HTML tags and the CSS code you write as there were typos in the previous one.
• The HTML code you gave me is not formatted correctly. It has lots of typos, misspelled ids and tags. Please fix it and no need to explain what everything is doing again.

That last prompt will make Jan's UI change its appearance as in the image above.

Expected behavior

To provide accurate tokens and keep Jan's UI constant.

Screenshots / Logs

Please see the first section.

Jan version

v0.5.2

In which operating systems have you tested?

• macOS
• Windows
• Linux

Environment details

No response