SylphAI

Only public for 3 days, the LightRAG library has already seen an impressive project from our user. Brandon Phillips created an automatic offer generation pipeline for job postings on Upwork! It’s brilliant. Here is the task pipeline: - Input: Job description [OfferInput class] - Step 1️⃣ : Given a job description, list all of the perceived and real problems and obstacles that the job poster could or is currently facing. Input: OfferInput Output: Problems[List[Problem]] - Step 2️⃣ : Break down this problem/obstacle into very intricate steps that the job poster would have to take in order to be successful. Input: Problem Output: SubProblems[List[SubProblem]] - Step 3️⃣ : For each problem and its subproblems, generate objections on why the customer can't solve the problem. Input: Problem, SubProblems Output: Objections[List[Objection]] - Step 4️⃣ : Given this problem, sub-problems, and objections, generate single sentences on how a single service provider would deliver one-on-one solutions. Here are the reasons why this project is possible with LightRAG and what stands out: 1️⃣ Powerful base class DataClass to handle advanced data interaction with LLM. 2️⃣ Built-in structure visualization following PyTorch’s visualization of the model pipeline. I can quickly understand all pieces of the pipeline by simply print(task_pipeline). 3️⃣ Chaining steps via Sequential, another PyTorch design pattern for AI modeling. As some steps involve loops, using async calls to run sequentially reduced the time from 2 minutes to 17 seconds. This is a truly interesting and practical project. On top of everything, he leveraged a customized child class to easily integrate with other observation tools like LangFuse. This project also comes with FastAPI for the web app. We have learned a lot about how PyTorch design patterns for composing layers for models can be helpful for LLM pipelines and have seen how users use it. We caught some bugs and improved our library. This project has given me confidence that working with the community is the right way to build a great library. 👉 Links in comments! 😁 Feel free to discuss your projects in our Discord channel. Would love to follow closely. #lightrag #artificialintelligence #llm ________________________________ LightRAG, The PyTorch Library for LLM Applications. It is light, modular, and robust. 🌟 Star the Github repo to stay updated. Official release next week.

We don't believe that simple API integration is the purpose of an LLM library. We believe in the best-performing, most elegant way of building LLM applications. The process should be less frustrating, and developers should steadily push and iterate their applications to the states they want. LightRAG just came out in alpha. I was worried that we wouldn't get too many GitHub stars. But what I truly wanted is for the world to see its beauty in the design and practicability. I wrote every tutorial not just showing how to use APIs, but how to understand, how to reason, and how it can help in different ways. "If I can't explain clearly, I have failed as someone creating the library." I am doing my best, and it will take a bit more time to have it mature. I truly want developers to learn and have that freedom and boost of creativity. It is like giving all the brushes and paints to a painter. Elon Musk would say he is not a businessman but a technologist. So am I. I am going after perfection; not having too many stars might give me more freedom to perfect it. On the other hand, I am also curious about what a community can do together on an open-source project. It is quite a dilemma. I would only want it to move in the direction of something great. We are LightRAG, the PyTorch library for LLM applications. https://lightrag.sylph.ai/ #lightrag #artificialintelligence #llm

What is the equivalent of Tensor (PyTorch) in LLM applications? A Tensor wraps a multi-dimensional matrix to better support its operations and computations in modules and optimizers. In LLM applications, data constantly needs to interact with LLMs in the form of strings via prompts and be parsed back to structured data from LLMs’ text predictions. This requires us to: - Describe the format of the structured data. This also includes the format of using a tool. - Show a few examples of the structured data (few-shot learning). - Parse the string output to structured data (dict or class). LightRAG has designed a base class, DataClass, to ease the data interaction with LLMs. Check out the why and how-to behind this class and let me know your thoughts about it! Links in comments! #artificialintelligence #llm #lightrag ________________________________ LightRAG, The PyTorch Library for LLM Applications, is now publicly available in alpha release. Follow hit 🔔 to stay updated. Official release next week.

Writing documentation for the LightRAG library feels like writing a hands-on, in-depth book on building the best LLM applications. How much content I need to crank out in the next few days is insane. 😂 #artificialintelligence #llm

I posted my frustration with the Langchain source code. Who would have thought—it got me nearly 1000 followers along with a bunch of haters? Why do I read source code in the first place? I initially built our LLM use cases with most of the code of our own and some parts from LlamaIndex. There was no doubt that no library could deliver 100% of the flexibility and customization I needed in our case, which is a conversational search engine and web info extractor. True, if I use numpy or pandas, I most likely don’t need to look at the source code at all. Because they do exactly what they say they will do, compute things the way you want, and give you exactly the result you expect. But LLM applications are so close to our product, to our users, and they are like water, all up to you how to shape it. You want 100% clarity on what's going on. That’s why a lot of people find it easy to build a demo with these libraries but hard to adapt them in any way. #artificialintelligence #llm #lightrag

The past six weeks have been intense! Starting with our internal codebase, which solves the most challenging LLM applications, built on 160 research papers and over 10 years of AI and coding experience, our library LightRAG is coming. We are polishing the code and writing developer notes and tutorials. LightRAG is the "PyTorch" library for LLM applications, where "Light" signifies clear and fast code, and RAG stands for Retriever Agent Generator! We can't wait to release this as soon as possible! #artificialintelligence #llm #lightrag

You cannot improve what you cannot measure. Evaluating LLMs and their applications is crucial in both research and production. In production, how you measure impacts the product user experience you want to deliver. Researchers care more about benchmarks. Production can focus more on your tasks, your data, and pick appropriate metrics that can help you the most. You might consider experimenting on a public dataset/benchmark similar to your task for development and comparison. Here is how we will decide on metrics: - Is this LLM for classical tasks such as classification? I would go for torchmetrics and sklearn.metrics. - Is this for generative tasks? If it's RAG, we have to measure both the retriever and the generator. RAGAS is a good library for that. If it's a code generation task, we will consider the execution accuracy and its efficiency. - Do I have enough ground truth? If not, I will create and align an LLM evaluator on a smaller labeled dataset and use it to evaluate more data. Here is a great blog that provides a comprehensive listing of most metrics for each task category: https://lnkd.in/grmUpg5D #artificialintelligence #llm #lightrag ____________ This is part of the work of LightRAG, the "PyTorch" library for LLM applications. Follow hit 🔔 to stay updated. We are going to public beta next week!

Manual prompting is a painful process; auto-prompt optimization is the future. Building the task pipeline accounts for only 10% of the work; the other 90% lies in optimizing it. LLM prompting is highly sensitive: the accuracy gap between top-performing and lower-performing prompts can be as high as 40%. It is also a brittle process that breaks the moment your model changes. Manual prompting is not the answer, but it is a good starting point for auto-prompt optimization. Here are two papers on auto-prompting you can read: - "Large Language Models as Optimizers" [DeepMind]: https://lnkd.in/gJSkbfn6 - "Automatic prompt optimization with gradient descent and beam search" [Microsoft Research] https://lnkd.in/gsV3vFkK #artificialintelligence #lightrag #llms ____________ This is part of the work of LightRAG, the "PyTorch" library for LLM applications. Follow hit 🔔 to stay updated. We are going to public beta next week!

There is no best model, only the best model for your task and your data. We benchmarked a classification task across major LLMs with few-shot ICL: GPT-4o, Gemini-Pro-1.5, LLama3, Claude3, and Mixtral, using the same prompt and the same temperature. Here are three major discoveries: 1. You never know how models will perform on your task unless you try: - GPT-4o is claimed to be better than GPT-4-turbo, but it falls short by 5% compared with even GPT-3.5. And GPT-3.5 can be almost as competitive as GPT-4s given the right task. - Gemini-Pro-1.5 is surprisingly the best even at zero-shot. - Llama3-70B and Claude3-opus are on par for this task. 2. Few-shot ICL is not always necessary. For some models, it boosts performance, and for others, it can cause a regression. 3. The same prompt, tuned on one model, can be applied directly to almost all models and still perform best across all. What are your experiences? 🤔 #ml #lightrag #llms #artificialintelligence ____________ This is part of the work of LightRAG, the "PyTorch" library for LLM applications. Follow hit 🔔 to stay updated.

There are two types of AI experts thanks to LLMs: - The Grandmaster: 'Yeah, I used the OpenAI API and Langchain. Now I can call myself an AI expert. Let me teach you how to do it.’ - The Imposter: 'This is so frustrating. Iterating on the prompt doesn't improve it much. Fine-tuning doesn't help either. What's going on with my data? I call myself an AI expert, but what do I really know?' Which one are you? 😉 #ai #ml #llms