Dmitry Zhelezov

Thrilled by Dr. Adam Bujak's Splendid Presentation at ABSL Conference in Cracow! Adam Bujak's recent presentation at the ABSL conference in Cracow was nothing short of inspiring! His insights on navigating the treacherous waters of the AI swamp left the audience spellbound. As he eloquently shared strategies to avoid being drawn into the murky depths of AI complexity, I could feel the collective nod of understanding from the crowd. Adam's clarity and passion were contagious, igniting a fire within each of us to tackle the upcoming challenges head-on. And here's the cherry on top: KYP.ai, the remarkable tool he introduced, promises to be our trusty guide through this AI wilderness. With KYP.ai, we're equipped to address the very challenges Adam highlighted. It's like having a compass that always points toward success! So, let's embrace the future with open arms, fueled by Adam's wisdom and powered by KYP.ai. Together, we'll conquer the AI swamp and emerge stronger than ever! #ABSLConference #AIExcellence #KYPaiMagic

63

3 Comments

It seems likely that OpenAI’s recent release of the #o1 model, despite its apparent lack of polish, is tied to the company’s current funding situation. At this point, however, the release of new AI models has become less of a significant event, given the abundance of both small and large language models, each with its own unique features and techniques. After trying o1, I noticed some improvements, but there were enough drawbacks that I question whether it was ready for release. If this were Apple, they likely would have held off. I can't help but feel that OpenAI is acting a bit hastily, especially when compared to Apple’s more measured approach just days apart. This contrast makes OpenAI’s urgency stand out even more. I wouldn’t be surprised to see similar models emerging from others within the next month or so. Personally, I’m more interested in the evolution of products rather than models themselves—how they will integrate into real-world applications, reduce costs, and prove their value. The endless race to release the next model is starting to feel exhausting.

43

3 Comments

Excited to be open-Sourcing long context (128K) Llama-3 70B - Llama-3-Giraffe-70B and Giraffe-Instruct From Abacus.AI The biggest issue with the Llama-3 family is that they are fairly small context models. So, even though they are as performant as GPT-4 on critical benchmarks, they can only be used in the real world with a bigger context. We just open-sourced two 128K long-context fine-tunes for Llama-3-70B, which effectively puts it on par with GPT-4 regarding context length support and real-world usage. At Abacus AI, we already use Llama-3 on a few production workloads as a great base model for our customer fine-tunes. Links to both models - https://lnkd.in/gPX_XpD6 https://lnkd.in/gaeEq94t

178

4 Comments

Level up your Large Language Models (LLMs) with a teamwork approach! Imagine your LLM as a brilliant student. Great at learning, but sometimes needs help tackling complex problems. AutoGen is like a tutoring system for LLMs. It facilitates conversations between different "agents": *LLMs: Your powerful student, bringing vast knowledge and generation skills. * Humans: The expert teacher, offering guidance and domain expertise. * Tools: Calculators, code interpreters, or other programs to assist with specific tasks. This teamwork unlocks next-gen LLM applications: * Complex problem-solving: Break down challenges into manageable steps with LLM, human, and tool collaboration. * Human-in-the-loop control: Ensure tasks align with your intent and leverage human judgment when needed. * Simplified workflows: Build complex LLM applications through easy-to-understand conversations between agents. Think of AutoGen as the conductor, harmonizing the strengths of LLMs, humans, and tools for powerful results! #AutoGen #LLM #AI #NextGenApplications https://lnkd.in/dfRbeK2W

36

3 Comments

💡I heard someone saying that he clearly understood the principles of input and output in neural networks, but he did not understand what happens in the "black box" in between. This made me think about people's and my own mistrust towards AI achievements. We see leaders of IT giants considering AI as the most impactful technology and general optimism prevails, I can assume that many people engage with AI primarily to avoid being left behind by this technology. Or even by technological breakthrough, we are all facing, not only AI. Stephen Hawking warned that AI might pose a threat to the human race, particularly highlighting the problem of singularity where robots could control humans. I don’t think we are already frightened by robots. I feel like people are mostly afraid of losing their jobs or social status and connections and consequently the need to acquire new skills. As we became lazy – to get new skills is quite challenging for all of us. Additionally, there is a fear of not being able to profit from the technology to support our health and improve our living standards. So, it is not the competition between humans and machines but among humans themselves. One is logically connected to the other. There are Ad, E-com and Finance where AI is strong, largely and successfully used even if sometimes hyped. There are healthcare and climate change issues where AI technology could be extremely promising but it will take more time. In Bridgen we are willing to explore AI solutions across various sectors. But also, we want to talk out AI to better understand how the technology can be the most helpful in people's self-development. What do you think about it? #ai #innovations #technology #personaldevelopment #business

5

One of the best indicators of a data science team's knowledge and maturity around LLM usage is the level of attention they are currently giving to upfront data preparation. 📣 Tomorrow, we're excited to be teaming up with Jerry Liu and the team from LlamaIndex to host a webinar on best practice data preparation for RAG workflows. We'll be showing how their LlamaParse solution for intelligent parsing and document transformation can be combined with Deasie's metadata creation workflow to build accurate and scalable knowledge retrieval over complex documents. When: Thursday 18th July, 9am PT / 12 ET Where: https://lu.ma/ufx77ve8 Who: Relevant for both AI engineers as well as company leaders looking to build their data foundation for GenAI

46

One of the best blogs I've come across on quantization, with outstanding visualizations. Highly recommend a read for anyone looking to understand LLM quantization in more depth! 📐 - https://lnkd.in/e_WTgq6x At Felafax AI (YC S24), we are building Infra to make full precision multi-GPU fine-tuning easier. #AI #Infra #LLM

16

3 Comments

We recently started driving a lot of optimizations on the Vajro platform. The focus is either on increasing app speed or reducing cost for us. We started with a couple of small optimizations: 1. Migration from NAT (Network Address Translation) Services: In Q4 of last year, we encountered intermittent throttling issues from Shopify while accessing their storefront, leading to failures in our downstream APIs. To mitigate this, we temporarily expanded our pool of IPs by implementing NAT gateways. However, after several rounds of API optimizations, we conducted experiments without NAT gateways and found that the system operated normally without experiencing throttling from Shopify. As a result, we have discontinued the use of NAT gateways entirely. 2. Optimizing Container Image Pulls: In our effort to improve Vajro analytics sync, we transitioned from AWS Lambda to Fargate to accommodate larger workloads. However, this change led to frequent retrieval of container images, resulting in increased costs. To address this, we have optimized the process by reconfiguring the virtual networks to fetch container images from the AWS internal network rather than from the Internet. This has effectively reduced egress expenses. We are currently working on reducing costs relating to Firebase and Cloudinary, which could potentially be substantial for us. Can't wait to share those when we get there! What optimizations have you recently worked on? #buildinpublic #aws #firebase #cloudinary #optimization

16

2 Comments

Thrilled to enable 💪 Bedrock - llama 3.2 cross region inferencing - https://lnkd.in/dad-phUS s/o Steffen Ridderbusch ( 4 more updates 👇) 💸 Azure AI rerank endpoint cost tracking s/o Emerson Gomes 🛠️ Logging - response redaction on `turn_off_message_logging` s/o Ilari Scheinin 🔨 Cohere - support cost override for cohere rerank models s/o Jerome Roussin ⚡️ Proxy - fix returned headers 

8

Llama 3.1 was just released and from the benchmarks it's performance is on par or better than other SOTA closed models (Whether current benchmarks are a good way to judge model performance is a whole can of worms I want to explore in another article…). The most interesting excerpt in the paper states that despite the architecture staying relatively similar to previous iterations of Llama, "the performance gains are primarily driven by improvements in data quality and diversity." The trend we observe throughout the paper highlights the importance of high-quality and diverse data in enhancing the training process. This is evident from several key stages. For instance, annealing data, despite its relatively small sample size, yields significant performance gains. Additionally, synthetic data, generated by model-based domain experts which were grounded using high quality data, were utilised to tune the model. The paper goes into the issues surrounding complex model architectures and attributes the large gains in performance to their larger, more diverse and higher quality training data. This is a trend we are seeing in the industry where AI companies are starting to license more proprietary data in order to give their models an edge. At Valyu, we understand the trends in the market and as a data licensing and distribution platform are excited to see the industry’s growing demand for high quality data. We remain committed to our mission to advance AI with responsible high quality data and look out for our full article breaking down the Llama 3.1 paper that we will be releasing soon. In the meantime read the Llama 3.1 paper here: https://lnkd.in/edBBN2yS

20

Have folks experienced #GPT4o’s instruction following / structured output to be more error prone than #GPT4-turbo and #GPT4? It seems to me GPT4o not just slowed down a lot in the last couple of weeks, but it also failing with instruction following where GPT4 did much better... Still, for my usecase the latency of GPT4o is superior but i sense it came with tradeoffs in terms of instruction following.

3

3 Comments

When you quantize down an LLM model, can you maintain its accuracy? I used to think no, but the data says otherwise. For example, Llama 3.1 405B FP16 has an average Accuracy score of 86.63. If you quantize down this model from FP16 to INT4, its Accuracy score falls to 86.47. That’s a recovery ratio of 99.81%. 🔥 🔥 In return, you get 4x more VRAM and higher performance. As people struggle to fit their models onto 8x H100s, a tiny loss in accuracy for 4x more VRAM sounds like a great bargain. Secret sauce seems to be the GPTQ algorithm, combined with neural networks being robust to quantization. Source: Neural Magic's Llama-3.1 70B and 405B INT4 models on Hugging Face. The link's in the comments. #opensource #llama3

35

5 Comments

In my view, advancements in and around LLMs are the first real shot the Real Estate industry has, at making use of unstructured municipal information like zoning regulation, at scale. This is why we started Plotzy. The time is now. The amazing work done by Arpit, Alex, and Dan, linked below, shows the kind of value applying LLMs to this problem space can yield. As to their findings, you're better off reading his post or the full paper, but here was a tidbit I found interesting: 7% of Municipalities have a smallest lot size 𝐚𝐛𝐨𝐯𝐞 𝐚 𝐡𝐚𝐥𝐟-𝐚𝐜𝐫𝐞 🤯 I'm inspired to do a similar research project for the Front Range area - what statistics and findings would you like to see? Drop a comment & I'll try and work it into the list. #CRE #PropTech #AI #LLMs #Zoning #RAG #RETech #AgenticAI

8

Update from Iteraite - we've added the R.I.C.E Prioritization template to our "Product Manager Toolkit" - as well as made our "Entrepreneur Toolkit" completely FREE 🎉 With the "PM Toolkit", you can go through the entire Discovery of a new product/feature using #AI as both an assistant and feedback generator. It costs less than a cup of coffee a month, and we're always adding more features. ☕️📅 If you just want to test it for free - just use Access Code: freePRD2024 to get access to the PRD (Product Requirements Document) generator, and more. We're not a "chatbot" - so you need to actively work through your Product, with the assistance of AI 🤓 The RICE Prioritization is a good example of that - AI is a great tool to help you think through your prioritization, but in the end, you (the PM) were hired to make the final call. Also - we've made the "Entrepreneur's Toolkit" completely #free! Why? Because we want people to be making better products and startups, and there is no excuse anymore to not knowing how to make an Elevator Pitch or Pitch Deck (with our AI-assisted tools). It's on our dime, so "you're welcome" 😃 Remember - we won't use/sell/see anything you write on our site - it's completely safe... because that's how we roll. 🎳 Just go to our site at iteraite.com to try them out! #freetools #aiforgood #productmanagement #productmanager #strategy #startup #entrepreneur #pitchdeck

15

4 Comments

Big fan of OpenAI's batch mode API. I can outsource a load of infra to them and also get a 50% discount. Now analysing 750k podcast episode transcripts each month for useful insights like guests, sponsors, topics, brand safety. Then I feed those outputs back in to more batches to derive podcast-level insights like predicted demographics, political leaning, what types of guests usually go on that show, etc. Next step is to start to surface all of this new data in the app.

5

Thanks Alex Nedyalkov for sharing this insightful research paper! In the near future, AI will drive most of the information we consume. This research shows why we need a range of LLM systems (proprietary and open-source) to ensure cultural diversity, democracy, and inclusivity are well represented. #AI #OpenSource #LLM

2

The documentation is an outline, and blog posts are coloring. They complement each other. When one gives direction, the other explains why. We believe that both are equally important. That is why we improve them daily on volkovlabs.io. The analytical dashboard is our web heart rate monitor that helps us to maneuver in the sea of questions and requests. We explained how we created this dashboard in January, 2023 and it becomes one of the community favorites: https://lnkd.in/eVi_BT42 #Grafana #Analytics

18

4 Comments

How do you format your Few-Shot Prompts? We almost exclusively use: Q: <Input> A: <Output> But would love to hear what others use. With Nathan Labenz on https://lnkd.in/egdXwCpy

13

During my attendance in two ruby conferences in Europe, I interviewed top Ruby developers at the conferences about the current job market. Given that Ruby on Rails is a mature framework, I wanted to know: what's the current state of the Ruby job market? These are the  valuable insights I got from industry experts on job prospects, challenges for juniors, and tips for success in the Rails ecosystem. Watch my full video on my YouTube channel (@tillcarlos) for more in-depth insights and discussions.

10

1 Comment