properly mark static fields

optimize the chances some google person will try shared key / values …

release working enwik8 training, thanks to @conceptofmind

oh, jax already has swish

fuse the attention and feedforward input projections, as in paper

fuse the attention and feedforward input projections, as in paper

attention and feedforward are processed in parallel and summed with r…

jit by default

fix rmsnorm in palm-lite

fix causal mask value, and add mask value to alibi positional bias to…