Next-Token Prediction is All You Need

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 and reasoning techniques.

DSIR large-scale data selection framework for language model training

Library for fast text representation and classification.

DataComp for Language Models

OLMoE: Open Mixture-of-Experts Language Models

[NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other models

[ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning

Measuring Massive Multitask Language Understanding | ICLR 2021

GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Implementation for ICLR 2024 paper “Multimodal Molecular Pretraining via Modality Blending"

Code for Discovering Preference Optimization Algorithms with and for Large Language Models

Official implement of paper "AutoCrawler: A Progressive Understanding Web Agent for Web Crawler Generation"

official code for "Large Language Models as Optimizers"

Resources for our paper: "EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms"

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference,…

System 2 Reasoning Link Collection

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

Let's Build A Simple Interpreter

Implementation for ICLR2024 Oral paper "Unified Generative Modeling of 3D Molecules with Bayesian Flow Networks"

[NeurIPS 2023] Reflexion: Language Agents with Verbal Reinforcement Learning

Automated Design of Agentic Systems

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Ongoing research training transformer models at scale

Scalable toolkit for efficient model alignment

fabric is an open-source framework for augmenting humans using AI. It provides a modular framework for solving specific problems using a crowdsourced set of AI prompts that can be used anywhere.

A one-stop, open-source, high-quality data extraction tool, supports PDF/webpage/e-book extraction.一站式开源高质量数据提取工具，支持PDF/网页/多格式电子书提取。