Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement: Improve RAG scalability #2044

Closed
bucovaina opened this issue May 7, 2024 · 3 comments
Closed

enhancement: Improve RAG scalability #2044

bucovaina opened this issue May 7, 2024 · 3 comments

Comments

@bucovaina
Copy link

bucovaina commented May 7, 2024

Is your feature request related to a problem? Please describe.
I want to export hundreds/thousands of documents to be embedded. Then I want to use them to provide help if I want to retrieve something about those documents. As in: "I want to upgrade our ESXi servers, what do I need to check before I do that". Or another prompt example: "I want to add a user to a project mailinglist. We have a script for that, but I forgot its name, can you tell me and create a short example how to do it?" (Last example implies the script itself is also embedded). Then the RAG implementation should make sure to first look at the local documents, there could be procedures that are company specific, not to be found in any LLM.

If I want to do that now, it's possible with tagging a document. But that requires a user to know in which document the information is located. So the closest I get is tagging a collection of documents (Like #usermanagement or #vsphere, ... ) but if the collection grows too large - which it quicly does - the context window of the LLM is usually too small to remember what it just read. Mostly the LLM replies with a generic answer, or something like: "the provided context does not seem to be relevant to your question"

Describe the solution you'd like
When I've embedded a bunch of text files (hundreds/thousands), I'd like to be able to let RAG do the work and if my prompt alone is specific enough, it'll find the relevant information. This without the need to tag a specific document. Tagging could perhaps still be possible if you want to limit the search, but should be optional, not required.

Describe alternatives you've considered
None

Additional context
I'm a SysAdmin, we have documentation in Confluence, I'd like to export it all to txt files, then import with Open WebUI and have a documents folder to embed all the txt files.

EDIT: forgot to mention: Thanks for your great work already, I love Open WebUI, fantastic software!!

@tjbck tjbck changed the title Improve RAG scalability enhancement: Improve RAG scalability May 7, 2024
@defaultsecurity
Copy link

We would also love this feature for our thousands of text files.

Could Open WebUI use LlamaIndex embeddings?
https://docs.llamaindex.ai/en/stable/examples/embeddings/ollama_embedding/

Could there be a LlamaIndex embeddings selector next to the model selector?

We would be able to combine any embedding with any model. Could that work? That would be awesome.

I am so happy that this project exists. You are true pioneers.

@tjbck
Copy link
Contributor

tjbck commented May 26, 2024

@tjbck
Copy link
Contributor

tjbck commented Jun 29, 2024

Closing in favour of #3527, Let's continue our discussion there!

@tjbck tjbck closed this as completed Jun 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants