Get your documents ready for gen AI

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

PDF to Markdown with vision models

No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents

Interactive Tools for Machine Learning, Deep Learning and Math

Graph-structured Indices for Scalable, Fast, Fresh and Filtered Approximate Nearest Neighbor Search

Dispatch and distribute your ML training to "serverless" clusters in Python, like PyTorch for ML infra. Iterable, debuggable, multi-cloud/on-prem, identical across research and production.

A native PyTorch Library for large model training

dstack is a lightweight, open-source alternative to Kubernetes & Slurm, simplifying AI container orchestration with multi-cloud & on-prem support. It natively supports NVIDIA, AMD, & TPU.

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Reverse Engineering the Abstraction and Reasoning Corpus

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12 clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.

Annotated version of the Mamba paper

Machine Learning Engineering Open Book

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

DocLLM: A layout-aware generative language model for multimodal document understanding

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

A Unified Library for Parameter-Efficient and Modular Transfer Learning

Algorithms, papers, datasets, performance comparisons for Document AI. Continuously updating.

Serving multiple LoRA finetuned LLM as one

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

Sampling profiler for Python programs

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

An unofficial Pytorch implementation of ERNIE-Layout which is originally released through PaddleNLP.

docTR (Document Text Recognition) - a seamless, high-performing & accessible library for OCR-related tasks powered by Deep Learning.

Example of a python monorepo using pip, the poetry backend, and Pants

Domain Specific Language for the Abstraction and Reasoning Corpus

batched loras

We identify the desiderata for a comprehensive benchmark and propose Visually Rich Document Understanding (VRDU). VRDU contains two datasets that represent several challenges: rich schema including…