Straive’s Post

Transcript

One of the things that every. Technology head, Every generative AI head, every CIO is looking for is. How can I get the team, the organization to use LM's for innovation effectively, but at the same time making sure that it doesn't? Have a huge cost and making sure that it has a reasonably good quality. What is the cost? Quality trade off? We can plot the model against a cost versus quality score. The X axis is the cost. And this is a fairly objective, uh metric. So for example, GPT 4 O costs $5 per million tokens as of now, and the quality is based on an ELO score. Now what this means is that there are some models like let's say Gemini 1.5 Flash API today, no model is better that is on top as well as cheaper that is on the left, which makes it a frontier model. There are some models which are the exact opposite as well, O LAMA 27B for instance. Every model is better or cheaper, at least one or the other. So the ones in red there's no reason to use. Actually the ones in Gray which are in the middle also there's no reason to use. The only models you ever want to use are the frontier models, which at the moment and this list keeps changing, is LAMA 38B which is the lowest cost, followed by Clot 3 high QJ mini 1.5 flash. Given a 1.5 pro and GPT 4 and then beyond that there's just no reason to use anything because there's nothing else that is of better quality, just more expensive but worse models out there. Let's take the difference between plot 3, high Q, GPT 4 turbo and GPT 4. For one million tokens, uh, a token is roughly 4 characters, so that's roughly 4 megabytes. Costs just $0.25 with Clot 3 IQ. The same thing costs $10, that's 40 times as much with GPT 4 Turbo and with the older GPT 4 model, it costs $30, which is 120 times and to give you a feel for that, and this is important for you to absorb what this means, a $10,000 budget with clot 3 high Q can blow up to $1.2 million budget with the old GBT 4.