Timothee Lacroix

LLMs can take gigabytes of memory to store, which limits what can be run on consumer hardware. But quantization can dramatically compress models, making a wider selection of models available to developers. You can often reduce model size by 4x or more while maintaining reasonable performance. In our new short course Quantization Fundamentals taught by Hugging Face's Younes Belkada and Marc Sun, you'll: - Learn how to quantize nearly any open source model - Use int8 and bfloat16 (Brain float 16) data types to load and run LLMs using PyTorch and the Hugging Face Transformers library - Dive into the technical details of linear quantization to map 32-bit floats to 8-bit integers As models get bigger and bigger, quantization becomes more important for making models practical and accessible. Please check out the course here: https://lnkd.in/g66yNW8W

10 603

144 commentaires

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval , MBPP , and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

175

2 commentaires

Stole a few points from Prof Christopher Manning, who talked about LLMs and language modeling in general in the latest TWIML AI Podcast: 📌 Humans acquire language skills in a way very different from LLMs. We need millions of words, compared to billions or even trillions of tokens for LLMs. LLM researchers may want to investigate and learn from how humans acquire language skills. 📌 LLMs cannot reason. However, there are other deep learning models, such as AlphaGo, that can. LLM researchers may want to look into how to integrate that type of reasoning/searching/planning capability into LLMs. 📌 LLMs' world models should enable search and discovery. Although Prof. Manning didn’t call this out explicitly, my understanding is more similar to a knowledge graph type of structure. 📌 Next-gen LLM idea: a soft form of locality and hierarchy. Transformers attend every token to every other token, which is very inefficient, while human language can be modeled by n-grams most of the time. #artificialintelligence #machinelearning #deeplearning https://lnkd.in/euzwMQ6p

84

18 commentaires

Instruction back-and-forth translation: technique to generate high-quality synthetic data for better LLM Alignment In this paper, authors propose a new method, instruction back-and-forth translation, to construct high-quality synthetic data grounded in world knowledge for aligning large language models (LLMs). In this approach, given documents from a web corpus, authors generate and curate synthetic instructions using the backtranslation approach and rewrite the responses to improve their quality further based on the initial documents. 𝗠𝗲𝘁𝗵𝗼𝗱 𝗼𝘃𝗲𝗿𝘃𝗶𝗲𝘄 Proposed pipeline consist of following steps: i) Backtranslation - first fine-tune a base model, e.g. Llama-2, on some seed examples from Open Assistant, for the task of instruction generation.  - then extract initial candidate responses from a web corpus, e.g. Dolma, and use the fine-tuned model to obtain synthetic instructions that would go with the corresponding responses ii) Filtering - separately obtain an instruction-following model by fine-tuning the same base model on the seed examples, and use it to score the quality of the (synthetic instruction, web-scraped response) pairs iii) Rewriting -  With the highest scoring pairs, we ask an existing aligned model (e.g. Llama-2-chat) to improve the responses further, conditioned on the generated instructions and the initial web texts.  - Steps (i) and (ii) follow Backtranslation approach proposed in (Self-alignment with instruction backtranslation. by Li et al.) with some modifications (i.e. using preprocessed documents from Dolma instead of parsing raw HTMLs from ClueWeb). 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 - Fine-tuning with the resulting (backtranslated instruction, rewritten response) pairs yields higher win rates on AlpacaEval than using other common instruction datasets such as Humpback, ShareGPT, Open Orca, Alpaca-GPT4 and Self-instruct - demonstrate that rewriting the responses with an LLM outperforms direct distillation, and the two generated text distributions exhibit significant distinction in embedding space - results shows that backtranslated instructions are of higher quality than other sources of synthetic instructions, while our responses are more diverse and complex than those obtained from distillation - find that instruction back-and-forth translation combines the best of both worlds—making use of the information diversity and quantity found on the web, while ensuring the quality of the responses which is necessary for effective alignment. 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀 - proposed pipeline revolves around general-purpose English instructions, with limited coding or science-related tasks 𝗕𝗹𝗼𝗴: https://lnkd.in/ei7pUGP9 𝗣𝗮𝗽𝗲𝗿: https://lnkd.in/e26geGgC

30

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16