OmniAI

New privacy features coming to OmniAI this week! 🔐 Here's a peek at the secret sauce. If you're in a regulated industry (AI applications or not) PII redaction is going be best practice. But simple redaction isn't going to cut it for LLM applications. The models need context to function correctly. i.e. replacing a user's name with "**** *****" is going to produce some unpredictable results. Instead we create a lookup table that substitutes the PII for a semantically similar value. That means `Dan Smith` is substituted for a randomly generated `John Baker` at query time. And the real user information is stored in a secure fashion. And the same substitution is applied as the data comes back from the model. Because we know `John` is a substitution for `Dan`, we're able to swap out response values so the user gets the expected output.

We're hiring!

We are thrilled to announce our $3.2M seed fundraising round led by FundersClub, and supported by investors including Transpose Platform, Imagination Capital, Kulveer Taggar, Richard Aberman, JJ Fliegelman, and Y Combinator! At Omni, we're building an entirely new way to work with unstructured data. The most valuable data is often the least accessible. Only 20% of corporate data fits in a SQL table today, with the rest scattered across unstructured formats (customer reviews, chat logs, transcripts, PDFs, etc.) With Omni, you can query and analyze that data as if it were SQL. We're enabling large scale unstructured data analytics that would have been impossible a few years ago. We are growing very fast and there's a lot to build. If you're an engineer passionate about building the future of AI data infrastructure, join us!

Turns out self hosting can be more expensive than GPT I hear people all the time say: "yea GPT is expensive, but we'll just self host and save a bunch of money". And I thought the same thing! But after some experiments in hosting different models on AWS, it can be really hard to get down to the same cost / token that you get from OpenAI. This comparison is between 3.5 Turbo and Llama 3 8b. To match the same TPS of 3.5 Turbo, I was running vLLM on an AWS g4dn.12xlarge instance. For the on demand tier ($3.912 / gpu hour) we got to $18 / M tokens. Roughly 9x the cost of GPT 3.5 Turbo! Now there are lots of optimizations that can get your token cost down with self hosted models. But in general, OpenAI's token pricing is pretty close to bare metal.

🌟 Happy Feature Friday! 🌟 We decided to do something fun this week and recorded a last minute video of us! Two major updates on OmniAI’s side this week - we shipped a unified API for LLMs and added PII redaction. 🔒 PII redaction: If you don’t trust OpenAI with your data, this feature is good for you. When users input PII like bank accounts or names, Omni automatically maps this data to a placeholder before sending it to the LLM. When the results come back, the data is mapped back so users still see the information they entered. 🔗 Unified API for LLMs: As we have seen even with yesterday’s news of Claude 3.5 Sonnet, new models are coming out every day. And we want to make sure that people using Omni are able to switch between whatever models are available. Stay tuned for next week!

Happy Feature Friday! 🛝 We are launching our OmniAI playground! Now you can test different default tools with different LLMs like GPT, Claude, Llama, and more. Compare the outputs from each model to determine which one gets the job done best. Stay tuned for chat completions, coming soon to the playground!

Excited to announce that OmniAI (YC W24) is officially live on AWS Marketplace! 🎉 OmniAI is the go-to platform to connect and run LLMs across your enterprise data. OmniAI provides a unified API to industry-leading LLMs and pre-built tools to categorize, extract, summarize, translate, and more. If you have AWS credits and want to use OmniAI, it's now simpler than ever. AWS Marketplace simplifies and consolidates procurement and invoicing, so you can use your existing AWS credits to purchase OmniAI directly. Learn more below! 👇

We had an amazing time at our first Snowflake Summit ❄️ In celebration, we're excited to announce OmniAI's new in-warehouse functionality for Snowflake. Now, you can do LLM data transformations right inside your existing Snowflake database. If you are looking to explore, extract, search from your data using LLMs, now is the time. Thanks to everyone who made the Snowflake Summit unforgettable. Off to Databricks next week!

Feature Friday! The RBAC feature is now live on OmniAI (YC W24)! 🔓 Admins can now add new users and assign them roles with just a few clicks. Admins have full access, while viewers are limited to read-only permissions.

Between Slack messages, shared drives, long-lost files, and the increasing number of applications used to receive and transfer information, the volume of internal company data is growing by as much as 63% a month from an average of 400 different sources. 💻 But thanks to advancements in AI, more particularly in NLPs and LLMs, we see a massive opportunity in organizing, centralizing, and locating unstructured data. In our latest publication, Jonathan and Nina dive into the sticking points of unstructured data, and how today's winners can capture this market. As always, if you're building in this space, please reach out! Read the full publication with the 🔗 in the comments. Companies included: OmniAI (YC W24) Amazon Web Services (AWS) Graphlit, by Unstruk Data Google Cloud, Microsoft Azure, unstructured.io #Data #AI #unstructureddata