Share your AI ideas! #263

end-4 · 2024-02-16T14:00:37Z

end-4
Feb 16, 2024
Maintainer

My dotfiles are not only for visual pleasure. The experience I created surely looks good to me and to an average person, which is wonderful. But what really excites me is how it's different: a centered bar layout that's still mouse-friendly, an overview widget, and most recently, an AI widget that does not use a webview.

The real question

Since Google Gemini is free, I'm wondering if anyone has any idea about how else we can utilize AI in the desktop, other than general chatting.

Some ideas and problems with them

Colorscheme generator using `gemini-pro-vision`

While it can't be worse than pywal, the results so far look uninteresting either due to [1 hue only], [colors look too similar], or [look too much like material colors -> meaningless]
It doesn't work at all times.

Smart window rearrangement

Only the window title and class don't seem to be enough for the model to give a good arrangement.

Chat based on window content

Privacy concerns in case a user accidentally activates it
We can only use a screenshot but not select text -> how well will that work?...

Keep in mind

The application should be somewhat complex enough. So far assistants in major OSes are impractical because they only work for basic tasks
We can tell the model to give a formalized response like a JSON, so don't just think of it as a text generator.

Let me know your ideas!

clsty · 2024-02-16T15:57:43Z

clsty
Feb 16, 2024
Collaborator

Talking about json, AI is very useful to generate things in specified format.

e.g. You can give it some markdown formatted text to translate and still formatted as markdown,
while the non-AI translators will often mess up with the part that is formatting-related and should not be translated.

P.S. Google gemini sure is free, but still it's not available in some regions where ChatGPT is.

0 replies

midn8hustlr · 2024-02-23T06:56:11Z

midn8hustlr
Feb 23, 2024

Regarding colorscheme generator, I feel that the 8 terminal colors are not much different to provide much meaningful information. IMO, it works great for ags top bar, side bar or overview as it utilizes material-like layout, but for terminal it's kinda useless. I realized this when I was writing a material-like tmux bar config, and picking colors from terminal itself (as its easier to update them dynamically with escape sequences), but the 8-colors look very similar.

Btw, how did you decide how to map the material colors to 8 terminal colors, did you follow any guide or template, or u are the first one to do it? Maybe I will try to look into this to make this more engaging.

1 reply

end-4 Feb 23, 2024
Maintainer Author

Btw, how did you decide how to map the material colors to 8 terminal colors, did you follow any guide or template, or u are the first one to do it? Maybe I will try to look into this to make this more engaging.

No basis. All random.

midn8hustlr · 2024-02-23T07:04:46Z

midn8hustlr
Feb 23, 2024

Can you elaborate more on smart window rearrangement. Like, what kind of rearrangement are you trying to achieve?

1 reply

end-4 Feb 23, 2024
Maintainer Author

Just grouping related stuff together... Here's the prompt I tried:

Init prompt

You are given a JSON array of windows of a desktop environment. Each window is an object containing the following keys: address (which serves as an unique identifier for each window), class (which is basically the window's app name), title (which is basically a summary of the window's current task), workspace (the number of the workspace in which the window is, can only be from 1 to 10. if a window has a negative workspace number, ignore it)- It is your job to reorganize windows across workspaces. That is, sorting related apps together into workspaces for each category.- Steps to perform this task: 1. Evaluating: Take a look at window titles and their classes to see how different windows relate. Put them in categories. Also decide if each category is a foreground or background one (for example, updates or setting apps are background tasks). 2. Ordering: Put foreground categories (that is, every window in a group) in workspaces at start (1, 2, 3, etc.). Put background categories in workspaces at the end (10, 9, 8, etc.) 3. Returning final result: Output your suggestion as a JSON object, with key/value pairs in this format: ADDRESS: WORKSPACE

I also gave it some examples

2 cases

User

[{ "address": "0x56459a8dd620", "class": "foot", "title": "sudo pacman -Syu","workspace": 1  },{ "address": "0x56459a8e89b0", "class": "org.gnome.Nautilus", "title": "ags","workspace": 5  },{ "address": "0x56459a8dd670", "class": "Code", "title": "~/.config/ags/config.js - Visual Studio Code","workspace": 4},{ "address": "0x56459a8adc10", "class": "thorium-browser","title": "Untitled prompt | Google AI Studio - Thorium","workspace": 9},{ "address": "0x56459a9460ba", "class": "foot","title": "~","workspace": 10},{ "address": "0x56459a9360d1", "class": "foot","title": "~","workspace": 1},{ "address": "0x56459a9460b6", "class": "foot","title": "~","workspace": 2},{ "address": "0x56459a9420c4", "class": "foot","title": "~","workspace": 9},{ "address": "0x56459a9460b2", "class": "foot","title": "~","workspace": 8},{ "address": "0x56459a9460b1", "class": "foot","title": "~","workspace": 8}]

Model

{"0x56459a8dd620": 10,"0x56459a8e89b0": 1,"0x56459a8dd670": 1,"0x56459a8adc10": 2,"0x56459a9460ba": 9,"0x56459a9360d1": 9,"0x56459a9460b6": 9,"0x56459a9420c4": 9,"0x56459a9460b2": 9,"0x56459a9460b1": 9}

User

[{ "address": "0x56459a98607b", "class": "gnome-tweaks","title": "GNOME Tweaks","workspace": 8,},{ "address": "0x56419a94607f", "class": "org.gnome.Settings","title": "Settings","workspace": 1},{ "address": "0x56459a9460d6", "class": "chromium","title": "GitHub - Chromium","workspace": 7},{ "address": "0x56459a9460a7", "class": "GitHub Desktop","title": "GitHub Desktop","workspace": 6},{ "address": "0x56459aa56769", "class": "foot","title": "~","workspace": 2},{ "address": "0x56459a9193ec", "class": "foot","title": "~","workspace": 3}]

Model

{"0x56459a98607b": 10,"0x56419a94607f": 10,"0x56459a9460d6": 1,"0x56459a9460a7": 1,"0x56459aa56769": 9,"0x56459a9193ec": 9}

H0mire · 2024-05-01T14:51:12Z

H0mire
May 1, 2024

A "Chat with Hyprland" Chatbot. A LLM with added knowledge of Hyprland, dots, ags and more related projects (Wikis, Forums) . Advanced: capable of fixing hyprland. For example add common fixes: ("reinstall_project", "reload ags", etc...) or even suggesting solutions after reading log files (privacy concerns? Use local LLM?).

2 replies

end-4 May 1, 2024
Maintainer Author

I guess we can feed the models the hyprland/other relevant docs?
(I just asked Gemini how 2 disable damage tracking and it told me to use gsettings lol)

H0mire May 1, 2024

Yes this should be possible. As I have no access to Gemini in my region I can only speak about openai. With the OpenAI library for python you can add files as knowledge. Should be the same for other Languages.

Boforg · 2024-05-03T13:43:05Z

Boforg
May 3, 2024

The AI assistant seems really cool! Is there a way for me to just install that widget as I already the rest of my Hyprland set-up. I'm not a programmer(I'm trying to learn) and making my own widget with waybar/eww/whatever else is above my abilities but I'd like to see how yours works. I was considering trying to implement dolphin-2.7-mixtral-8x7b and run it locally through ollama. It's supposed to be able to code pretty well so I was hoping to use it to help troubleshoot as I learn.

2 replies

end-4 May 3, 2024
Maintainer Author

you can go into .config/ags/config.js and comment out everything other than stuff related to "Sideleft" in the Windows array. then only the left sidebar will be loaded when you run ags. to toggle it, run ags -t sideleft
idk about dependencies

Boforg May 3, 2024

Thanks! I'll give it a go later.

bpawnzZ · 2024-05-30T02:24:11Z

bpawnzZ
May 30, 2024

Please add the ability to use custom endpoints! For instance I use https://openrouter.ai/ for interacting with AI. Would you please consider the ability to set Custom API endpoints and model. This would be amazing!. Not a fan of Gemini and OpenAI is overpriced. Openrouter is an amazing service and it would be amazing if you would implement it. Just a suggestion that would make this setup even better. In general props on what you have accomplished.

4 replies

end-4 Jun 1, 2024
Maintainer Author

Actually for now you can edit the PROVIDERS object in ~/.config/ags/services/gpt.js
It's just not very update-friendly in case you actually do. Though I'd be happy to add that api if you provide the needed fields for each provider in that list (name, logo, description, base url, url to get link, model name (if necessary))

bpawnzZ Jun 6, 2024

Actually for now you can edit the PROVIDERS object in ~/.config/ags/services/gpt.js It's just not very update-friendly in case you actually do. Though I'd be happy to add that api if you provide the needed fields for each provider in that list (name, logo, description, base url, url to get link, model name (if necessary))

That was actually extremely easy to implement once I knew where the file location was. Thanks for that. I attached the svg for icon i used and I also created a file called openrouter_key in its appropriate location. Works as you would expect. Good shit man!

I added the following to the top of PROVIDERS object in ~/.config/ags/services/gpt.js

'openrouter': {
        'name': 'OpenRouter',
        'logo_name': 'openrouter-symbolic',
        'description': 'A unified interface for LLMs',
        'base_url': 'https://openrouter.ai/api/v1/chat/completions',
        'key_get_url': 'https://openrouter.ai/keys',
        'key_file': 'openrouter_key.txt',
        'model': 'meta-llama/llama-3-70b-instruct',
    },

bpawnzZ Jun 6, 2024

end-4 Jun 6, 2024
Maintainer Author

cool
added that in 2a11489

x-leehe · 2024-06-18T02:20:53Z

x-leehe
Jun 18, 2024

The Gemini API needs an upgrade, for the gemini-pro API was deprecated a few days ago. See this:

I've already made a modification on my computer myself.

The process:

Replaced "const CHAT_MODELS = ["gemini-pro"]" to "const CHAT_MODELS = ["gemini-1.5-flash"]" in ~/.config/ags/services/gemini.js Line 47
Replaced "label: 'Assistant (Gemini Pro)'," to "Assistant (Gemini 1.5 Flash)'," in ~/.config/ags/modules/sideleft/apis/gemini.js Line 37 (So does the tipText in Line 52)

Anyway though, this function is stunning!

2 replies

end-4 Jun 18, 2024
Maintainer Author

i noticed that, but it's for 1.0 pro vision
but it doesn't hurt upgrading, and 1.5-flash seems to be faster, cool 👀

x-leehe Jun 18, 2024

Yeah, but the thing is that maybe the gemini-pro stopped working too. Last time (about 4 days ago) it gave me an empty response when I was trying to use it, and then Google emailed me. Thx😉

bpawnzZ · 2024-07-11T07:30:38Z

bpawnzZ
Jul 11, 2024

I have a suggestion. You should allow the connection to a searxng instance to give the ability for websearch. Searx is implemented in crewai, chat-ui, langchain connects to it. its the 'no api' local way(runs in docker) way to do web search, atleast that i know of. That would really turn up the badass of AI in this project.

I believe this could be very easily implemented with Langchain. Just wondering what u thought about it.

I been running end-4 for a couple months now and like it alot. Nice job by the way

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Share your AI ideas! #263

{{title}}

Replies: 8 comments 12 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Share your AI ideas! #263

end-4 Feb 16, 2024 Maintainer

The real question

Some ideas and problems with them

Colorscheme generator using gemini-pro-vision

Smart window rearrangement

Chat based on window content

Keep in mind

Replies: 8 comments · 12 replies

clsty Feb 16, 2024 Collaborator

end-4 Feb 23, 2024 Maintainer Author

end-4 Feb 23, 2024 Maintainer Author

end-4 May 1, 2024 Maintainer Author

end-4 May 3, 2024 Maintainer Author

end-4 Jun 1, 2024 Maintainer Author

end-4 Jun 6, 2024 Maintainer Author

end-4 Jun 18, 2024 Maintainer Author

end-4
Feb 16, 2024
Maintainer

Colorscheme generator using `gemini-pro-vision`

Replies: 8 comments 12 replies

clsty
Feb 16, 2024
Collaborator

end-4 Feb 23, 2024
Maintainer Author

end-4 Feb 23, 2024
Maintainer Author

end-4 May 1, 2024
Maintainer Author

end-4 May 3, 2024
Maintainer Author

end-4 Jun 1, 2024
Maintainer Author

end-4 Jun 6, 2024
Maintainer Author

end-4 Jun 18, 2024
Maintainer Author