Using FlexAttention to compute attention with different masking patterns

Efficient Triton Kernels for LLM Training

Megatron's multi-modal data loader

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 and reasoning techniques.

NumPy tutorials & educational content in notebook format

『ゼロから作る Deep Learning ❸』(O'Reilly Japan, 2020)

LLM related research papers curated by LLMs themselves

Neural Networks: Zero to Hero

LLM101n: Let's build a Storyteller

Pytorch implementation of "Genie: Generative Interactive Environments", Bruce et al. (2024).

Official implementation of AnimateDiff.

Lumina-T2X is a unified framework for Text to Any Modality Generation

📺 An End-to-End Solution for High-Resolution and Long Video Generation Based on Transformer Diffusion

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

Here we will keep track of the latest AI Game Development Tools, including LLM, Agent, Code, Writer, Image, Texture, Shader, 3D Model, Animation, Video, Audio, Music, Singing Voice and Analytics. 🔥

[SIGGRAPH Asia 2024, Journal Track] ToonCrafter: Generative Cartoon Interpolation

[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (PRG)

Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (2024)

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

A curated list of recent diffusion models for video generation, editing, restoration, understanding, etc.

DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained in a diffusion world model.

[ICML 2024 Best Paper] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)

[CVPR24 Highlights] Polos: Multimodal Metric Learning from Human Feedback for Image Captioning

Simple AI agents / assistants

A list of AI autonomous agents

When do we not need larger vision models?

Official implementation of "HowToCaption: Prompting LLMs to Transform Video Annotations at Scale." ECCV 2024

Website for hosting the Open Foundation Models Cheat Sheet.

Open-Sora: Democratizing Efficient Video Production for All

Open-source and strong foundation image recognition models.