int4

Here are 5 public repositories matching this topic...

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

row-major matmul optimization

vulkan cuda armv7 arm64 ptx gemm-optimization cuda-kernel int4

An innovative library for efficient LLM inference via low-bit quantization

Advanced Quantization Algorithm for LLMs/VLMs.

rounding quantization awq int4 gptq neural-compressor

rust library to write integer types of any bit length into a buffer - from `i1` to `i64`.

rust bytebuffer bitbuffer bitbanging bitwriter int4 int15

Add a description, image, and links to the int4 topic page so that developers can more easily learn about it.

To associate your repository with the int4 topic, visit your repo's landing page and select "manage topics."