FlashInfer: Kernel Library for LLM Serving

Compiler for Dynamic Neural Networks

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

ppl.cv is a high-performance image processing library of openPPL supporting various platforms.

AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI 23)

Text-to-video generation: CogVideoX (2024) and CogVideo (ICLR 2023)

A list of awesome compiler projects and papers for tensor computation and deep learning.

SGLang is yet another fast serving framework for large language models and vision language models.

OneDiff: An out-of-the-box acceleration library for diffusion models.

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

Open-Sora: Democratizing Efficient Video Production for All

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

A framework for few-shot evaluation of language models.

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Measuring Massive Multitask Language Understanding | ICLR 2021

Natural Questions (NQ) contains real user questions issued to Google search, and answers found from Wikipedia by annotators. NQ is designed for the training and evaluation of automatic question ans…

A collection of benchmarks and datasets for evaluating LLM.

A collection of AWESOME things about mixture-of-experts

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

🎉CUDA/C 笔记 / 大模型手撕CUDA / 技术博客，更新随缘: flash_attn、sgemm、sgemv、warp reduce、block reduce、dot product、elementwise、softmax、layernorm、rmsnorm、hist etc.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

ATC23 AE

AIGC-interview/CV-interview/LLMs-interview面试问题与答案集合仓，同时包含工作和科研过程中的新想法、新问题、新资源与新项目

A LaTeX resume template designed for optimal information density and aesthetic appeal.