Commits

Branch selector

 486f98e

User selector

All users

Datepicker

All time

Commit History

Commits on Jul 1, 2024

• zero-grad is async and part of backward call

  ngc92

  committed
  486f98e

  Copy full SHA for 486f98e

• Merge branch "ademeure-optimize_attproj_fcproj"

  karpathy

  committed
  79223b9

  Copy full SHA for 79223b9

• resolve merge conflict

  karpathy

  committed
  cee396a

  Copy full SHA for cee396a

• add edu fineweb sh download script, change paths to my HF repo, add hellaswag to starter pack

  karpathy

  committed
  48fee04

  Copy full SHA for 48fee04

• Merge pull request #653 from ademeure/cublaslt_refactor

  Show description for a876282
  karpathy

  authored
  a876282

  Copy full SHA for a876282

• Disable GELU fusion by deffault & fix potential argv[] out of bounds access

  ademeure

  committed
  00247dc

  Copy full SHA for 00247dc

• Merge pull request #663 from karpathy/fix/reshuffle_zero_grad

  Show description for 2e6a397
  karpathy

  authored
  2e6a397

  Copy full SHA for 2e6a397

• move things around a bit, a few comments, and also update python script to correspond to C more

  karpathy

  committed
  7b92930

  Copy full SHA for 7b92930

• Merge branch "fix_periodic_spikes" of https://github.com/gordicaleksa/llm.c into gordicaleksa-fix_periodic_spikes-2

  karpathy

  committed
  8253823

  Copy full SHA for 8253823

• Merge pull request #662 from karpathy/overfit-batch-actually

  Show description for 942fed5
  karpathy

  authored
  942fed5

  Copy full SHA for 942fed5

• fix the overfit single batch behavior to actually overfit batch, not microbatch, we do this more cleanly by simply resetting the dataloader every step

  karpathy

  committed
  7fd7717

  Copy full SHA for 7fd7717

• Add micro step guard

  gordicaleksa

  committed
  74e5ba7

  Copy full SHA for 74e5ba7

• Remove local args for testing

  gordicaleksa

  committed
  78e7627

  Copy full SHA for 78e7627

• Make the grad cleaning more robust

  gordicaleksa

  committed
  de8454d

  Copy full SHA for de8454d

• Fix periodic spikes - clean loss after val

  gordicaleksa

  committed
  d2f06ae

  Copy full SHA for d2f06ae

• only rank 0 creates log dir

  karpathy

  committed
  61625d1

  Copy full SHA for 61625d1

Commits on Jun 30, 2024

• merge fix

  ademeure

  committed
  20e450c

  Copy full SHA for 20e450c

• fix default for test_gpt2cu

  ademeure

  committed
  6e29dd0

  Copy full SHA for 6e29dd0

• Make GELU fusion default 2 for H100+ and 0 on older GPUs for now

  ademeure

  committed
  0f11a4e

  Copy full SHA for 0f11a4e

• Refactor matmul to only use cuBLASLt and optionally use fused GELU

  ademeure

  committed
  fa5a196

  Copy full SHA for fa5a196

• small cosmetic fixes to fineweb data file

  karpathy

  committed
  ea36d04

  Copy full SHA for ea36d04

• Remove per-layer attproj and fcproj activation tensors (reuse output as scratch buffer instead). Significant memory saving.

  ademeure

  committed
  ba4e2c0

  Copy full SHA for ba4e2c0

Commits on Jun 29, 2024

• Merge pull request #629 from rosslwheeler/feature/tests

  Show description for 2daaf6a
  karpathy

  authored
  2daaf6a

  Copy full SHA for 2daaf6a

Commits on Jun 28, 2024

• Merge branch "eliebak-add_edu_fineweb_support"

  karpathy

  committed
  e6e365a

  Copy full SHA for e6e365a

• Merge branch "add_edu_fineweb_support" of https://github.com/eliebak/llm.c into eliebak-add_edu_fineweb_support

  karpathy

  committed
  ff7a90b

  Copy full SHA for ff7a90b

Commits on Jun 27, 2024

• Merge branch "ngc92-mixed-dtypes-2"

  karpathy

  committed
  2cf7c5b

  Copy full SHA for 2cf7c5b

• handle unknown datatype to squash compiler warning

  karpathy

  committed
  012915c

  Copy full SHA for 012915c

• Merge pull request #652 from ademeure/cudnn_determinism

  Show description for 06cd83e
  karpathy

  authored
  06cd83e

  Copy full SHA for 06cd83e

• Make cuDNN deterministic for Flash Attention backward (requires frontend >=1.5)

  ademeure

  committed
  29a067f

  Copy full SHA for 29a067f

• Addressed review comments

  Show description for d6d08be
  rosslwheeler

  committed
  d6d08be

  Copy full SHA for d6d08be

• Add H100 and A100 ptx/sass files

  rosslwheeler

  committed
  eb0b74e

  Copy full SHA for eb0b74e

• Merge branch "karpathy:master" into feature/tests

  rosslwheeler

  authored
  267efc9

  Copy full SHA for 267efc9

• macro-based alternative

  ngc92

  committed
  98875dd

  Copy full SHA for 98875dd

Commits on Jun 26, 2024

• add some comments

  ngc92

  committed
  d38c614

  Copy full SHA for d38c614

• keep layernorm stats in fp32

  ngc92

  committed
  7beaa1e

  Copy full SHA for 7beaa1e

Pagination

Previous

Next