Google Cloud’s Post

Gemini 1.5 Flash 3 key differentiations: - 1 million token context window, which is approximately 60x bigger than the context window provided by GPT-3.5 Turbo - On average, 40% faster than GPT-3.5 Turbo when given input of 10,000 characters - Up to 4X lower input price than GPT-3.5 Turbo

GPT-4o Benchmark Results Showing that it is Truly a Next-Generation Model We tested the performance of GPT-4 Omni (model name: gpt-4o) on our finRAG dataset, the results show that this is truly a next generation model that does not seem to have some common issues that previous generation models had, making it possibly the first model suitable for reliable enterprise use. https://lnkd.in/ehNvXSzF

GPT-4o Benchmark Results Showing that it is Truly a Next-Generation Model We tested the performance of GPT-4 Omni (model name: gpt-4o) on our finRAG dataset, the results show that this is truly a next generation model that does not seem to have some common issues that previous generation models had, making it possibly the first model suitable for reliable enterprise use. https://lnkd.in/e7WRQGM9

👀 Just saw phi-3 drop. It's outperforming other models across the board. This is big news on a few different levels: * Phi-3 mini: Edging out on benchmarks versus Llama 3 8B while crushing Mistral and Gemma at 3.8B parameters. * Phi-3 small: Solid across tasks Nearly matches GPT-3.5 at 7B parameters. * Phi-3 medium: Crushing the Mixtral at 14B parameters versus the MoE 56B total parameters with about 14B active ones for a given response! 🔄P.S. Planning to give phi-3 a try? GPT-5 coming soon? What do you think?

Stumbled upon this tweet this morning where Greg Kamradt had completed a comprehensive study to evaluate the recall performance of GPT-4 with 128K tokens. Here's a summary of what he found: 📉 Recall Performance Degradation: Above 73K tokens, GPT-4's ability to recall information started to diminish significantly. This works out at about ~110 pages (if you do about 500 words a page) - so again we are still somewhat limited to what can be ingested in one go. 📍 Importance of Document Positioning: Facts placed at the beginning of documents were recalled more effectively. However, facts in the middle portion (7%-50% of the document) showed lower recall rates. 🔑 Key Insights: No Guarantees on Recall: Essential to recognise that there's no certainty in GPT-4's ability to retrieve specific facts. Less is More for Context: Reducing the context sent to GPT-4 can improve accuracy, reaffirming the principle that less context can lead to better performance. Fact Placement Matters: The study underscores the impact of fact positioning within a document on recall efficiency. #legaltech #openai #contextwindow https://lnkd.in/eYNrF5X2

Finally an open-source model has caught up to GPT-4-0314. We have new open-source models in town. Thanks to the team at WizardLM Here are some benchmarks for their three models respectively: 8x22B 70B 7B

I combined GPT and Gemini on one screen. An easy way to compare output from different models. One message - two answers. Check it out: https://bit.ly/gpt_arena

OpenAI’s newest GPT-4 model has a 128K context length, meaning it can read (or write!) a book of about 300 pages. But how well does it use that full context? Some new tests suggest that (1) it’s more accurate than prior OpenAI models (see https://lnkd.in/g7f2jpgK) and (2) it doesn’t suffer recall degradation until about 73K tokens (see https://lnkd.in/gaNA_59d). #artificialintelligence #largelanguagemodels #gpt4