Problem

• There are many users looking to enter multiple stop words
• The "stop" section for configuring the model does not make it clear whether it should be a comma-separated list, a newline-separated list, or something else.
• It would be nifty (and I think aligned with how other apps do it) if as you insert a token, it creates a standalone entry in the box. - Something like: <endofstring>, <new_sentence>, <this_is_the_end>.
• And then you could individually click-to-delete any of the "stop" tags you created. Hopefully that is comprehensible.

Solution

From technical aspect, there are 2 cases:

• Models from cortexso source: the stop token is included in <model_id>.yaml file, so Jan app just read from it to place it as default.
• Models from other source: With cortex.cpp, when read model from gguf file, it also contains the stop words list, cortex.cpp will read and write to <model_id>.yaml, Jan app can read from <model_id>.yaml to set as default

Format

Most of model introduce stop tokens with this format <{content}> . The content is different for each model arch. So I think we can predefine a list option of map model's arch : list stop words like this for user to choses:

• llama3: ["<|eot_id|>", "<|end_of_text|>"]
• mistral: ["< /s > "]
• ...

Maybe 5 or 6 popular models arch is enough and another option to let users input whatever they want (this feature may be only for power user or dev because normal user might only use default configuration)

Design

Figma: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8281-97234&t=qb7yU8r2PAayVdNW-4

• Use a tag-like interface where each stop word is in its own removable "pill"
• Users can add new tags and remove existing ones

Default Stop Words:

• These come from the model's YAML file
• Should be visually distinct and are recommended not to be removed (they're usually carefully chosen by the model creators for optimal performance, removing these could potentially cause issues with the model's behavior or output quality).
• Users should understand these are recommended for the model

Predefined Options:

Offer a dropdown preset or quick-select for common stop words based on model architecture

Custom/User-Added Stop Words:

• Added by the user & should be removable

Task

• Figure out what stop tokens format should be in @nguyenhoangthuan99
• Design @imtuyethan
• UI implementation @urmauur