Kence Anderson

Thanks for sharing these insights. What I am seeing is that orgs test an idea using a proprietary #LLM like #OpenAI, then put it into production with an open source LLM using a development platform like #Llamaindex. That way, private data remains private.

1

Batch processing for LLM inference activity: "One sign of that is the rise of batch processing. Today, popular consumer AI products like ChatGPT and Perplexity provide users with near instantaneous responses, otherwise known as real-time inference. However, businesses don't always need immediate responses and are often willing to wait hours, days, or even weeks for responses, as long as they don't have to shell out as much money. Founders of cloud and inference providers have told me there's growing pressure from business customers for this kind of flexibility, otherwise known as batch processing." -- Nathaniel Whittemore on "The AI Daily Brief" (with much deserved shout-out to his inspo from Stephanie Palazzolo at The Information) I fall into this bucket as we have a lot of backfilling to do as we're building out the underlying data in our service, and some of that could just as well be done in the wee hours of the night. The new stuff flowing in I want processed ASAFP, but these backfills—I'd mos def take more cost-efficient delayed batch processing. There's also use cases like needing (near) real-time inference for the first inference pass for timeliness / contemporaneousness value, but then a second pass slow-time inference for wider classification exercises over an ever-growing data corpus. I could see us mixing Sonnet & Haiku models in both cases. Just a heads up to Anthropic Mike Krieger as you contemplate this, along with Amazon Web Services (AWS) Jeff Barr Mark Roy Swami Sivasubramanian on the Bedrock side of things...and I don't mean the "batch" mode that doesn't seem to have any real value other than helping AWS manage its own rate limit challenges. (DeepCast is using Bedrock.)

11

2 Comments

Next level AI application from the NewNow team. Check out the article.

11

We’ve already seen that software eats the world. Ali Ghodsi predicts that, in the next phase, AI eats software. During a conversation on the B2BaCEO podcast, Ali, the Co-founder & CEO of Databricks, said AI will not just be a feature; it will be the foundation of most platforms, as we move toward a world where AI is seamlessly integrated into nearly every tool. Imagine every piece of software evolving from static code into an autonomous decision-maker, as intelligence and automation are built into all data architecture underlying enterprise software. Our full conversation on the B2BaCEO podcast here: https://lnkd.in/gXvSSf_C

56

4 Comments

As we look ahead to 2025, it's becoming clear that the future of enterprise AI lies in compound systems, not monolithic models. Compound AI systems combine multiple models that work together iteratively, coordinating and leveraging external tools to evaluate, refine, and improve their own results. 🔄 The benefits are clear: ☑ Better performance than individual models ☑ Faster and less expensive iteration  ☑ Granular access controls on sensitive data  ☑ More reliably ensure that models behave as intended ☑ Flexible performance-cost optimization Early examples like AlphaCode 2, LLM Debate, and CoT-SC are already showing impressive results. 📈 As we move towards 2025, compound AI systems will become increasingly critical for enterprises looking to deploy AI at scale reliably, securely, and cost-effectively. 🎯 Check out my latest B2BaCEO newsletter for a detailed look at where AI is headed and what it means for founders: https://lnkd.in/gRrzX9nq h/t to Berkeley Artificial Intelligence Research and my colleagues Jaya Gupta Andrew Han Courtney Fiske for their research help

65

6 Comments

How do you pick the perfect AI model for our business needs ? "Every leader faces this challenge. Rapid value, mounting pressures. How do we navigate it?" "Choosing the right AI model isn't just technical—it's strategic. 1. Model flexibility is key. 2. Budget decisions drive scope. 3. Criteria for evaluation are critical. Leaders from AWS and Arthur AI share innovative strategies to guide your choice, ensuring alignment with business needs and technical specifications." "Mastering AI selection transforms outcomes." I did enjoy watching the webinar and so will you i guess , Here →

1

The Benchmarking Challenge ‼ As generative models like LLMs demonstrate remarkable capabilities in language understanding and complex reasoning, the need for evolving our evaluation methods becomes crucial. How do we effectively measure the progress of these increasingly sophisticated systems? Comprehensive benchmarking of LLMs at scale presents obstacles that must be overcome: 1️⃣ Cost Efficiency: Large benchmarks demand substantial energy and compute. How can we make benchmarking faster and more affordable without compromising quality? 2️⃣ Evaluating Complex Tasks: LLMs excel at multi-task, open-ended problems beyond traditional narrow benchmarks. How do we assess these broader capabilities? 3️⃣ Specialized Domain Performance: As LLMs advance in areas like code generation, how can AI community ensure benchmarks stay relevant and accurately assess evolving performance in specialized tasks? While benchmarks play a crucial role in evaluating AI systems, it's important to remember that they aren't the be-all and end-all of GenAI progress. They're an important factor in assessing AI capabilities, but not the only one. ❗ They're like academic grades in the real world - important, but not everything. We say "grades don't define you," yet we can't help comparing them. Similarly, the AI community both critiques and relies on benchmarks. 🔑 Real-world application, ethical considerations, and adaptability to diverse scenarios are equally crucial in determining #GenAI system's true value and impact. What are your perspectives on this rapidly evolving landscape? Keen to hear from researchers, developers and leaders tackling these issues.

5

1 Comment

In a recent Office Hours, Baris Gultekin, Head of AI at Snowflake & a friend from Google, shared his insights into Snowflake’s approach to AI. It was a great conversation to understand how one of the leading data companies is pushing AI forward. The video for the session is here : https://lnkd.in/gfBesq35 & the audio is here : https://lnkd.in/gUK8hR66. There are four major pushes for Snowflake within AI. 1. Bringing AI to Data with Cortex. Cortex is a suite of AI building blocks that enable customers to leverage large language models (LLMs) & build applications. Both structured data & unstructured data (extracting information from presentations & pdfs) can be built using Cortex. 2. Developing open source initiatives including a data catalog, Polaris, & an open LLM Model Arctic which focuses on SQL performance. Open source is a new motion for Snowflake. 3. Securing data sharing to bring data into customers’ accounts for AI. Data sharing across customers enables a marketing organization to use data from a credit card company to build a model for customer marketing. 4. Developing a platform atop Snowflake with native apps to enable a retailer to provide a search experience to their wholesalers, for example. We also chatted about the importance of data access security for AI applications. For many data leaders, this is one of the top priorities to understand before deploying AI. Thanks, Baris, for sharing your perspectives!

31

4 Comments