Skip to content

WebAssembly bits processing utilities 🏎️ (unpack, pack, xor)

License

Notifications You must be signed in to change notification settings

hazae41/naberius

Repository files navigation

Naberius

WebAssembly bits processing utilities

npm i @hazae41/naberius

Node Package πŸ“¦ β€’ Deno Module πŸ¦–

Algorithms

  • unpack: transform an array of bytes to an array of bits (aka bitfield)
  • pack_left: transform an array of bits to a left-padded array of bytes
  • pack_right: transform an array of bits to a right-padded array of bytes
  • xor_mod: apply in-place XOR to an array of bytes using a mask

Features

  • Reproducible building
  • Pre-bundled and streamed
  • Zero-copy memory slices

Benchmarks

unpack

Deno

cpu: Apple M1 Max
runtime: deno 1.30.0 (aarch64-apple-darwin)

file:///src/deno/bench/unpack.bench.ts
benchmark        time (avg)             (min … max)       p75       p99      p995
--------------------------------------------------- -----------------------------
wasm           3.31 Β΅s/iter     (3.25 Β΅s … 3.52 Β΅s)   3.33 Β΅s   3.52 Β΅s   3.52 Β΅s
js (array)    10.44 Β΅s/iter   (8.25 Β΅s … 121.83 Β΅s)  13.04 Β΅s  14.25 Β΅s  15.04 Β΅s
js (string)   51.27 Β΅s/iter  (49.75 Β΅s … 178.83 Β΅s)  50.08 Β΅s 109.71 Β΅s 114.38 Β΅s

summary
  wasm
   3.15x faster than js (array)
   15.49x faster than js (string)

Node

src/node/bench/unpack.bench.ts
cpu: Apple M1 Max
runtime: node v18.12.1 (arm64-darwin)

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   (index)   β”‚     average     β”‚  minimum   β”‚   maximum   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚    wasm     β”‚ '3.09 ΞΌs/iter'  β”‚ '2.04 ΞΌs'  β”‚ '169.21 ΞΌs' β”‚
β”‚ js (array)  β”‚ '17.64 ΞΌs/iter' β”‚ '16.71 ΞΌs' β”‚ '854.29 ΞΌs' β”‚
β”‚ js (string) β”‚ '41.94 ΞΌs/iter' β”‚ '40.04 ΞΌs' β”‚ '218.75 ΞΌs' β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Summary
- wasm is 5.71x faster than js (array)
- wasm is 13.58x faster than js (string)

pack_right

Deno

cpu: Apple M1 Max
runtime: deno 1.30.0 (aarch64-apple-darwin)

file:///src/deno/bench/pack_right.bench.ts
benchmark        time (avg)             (min … max)       p75       p99      p995
--------------------------------------------------- -----------------------------
wasm           2.75 Β΅s/iter      (2.73 Β΅s … 2.9 Β΅s)   2.76 Β΅s    2.9 Β΅s    2.9 Β΅s
js (array)   668.34 Β΅s/iter   (241.29 Β΅s … 1.21 ms)   1.02 ms   1.11 ms   1.16 ms
js (string)   43.83 Β΅s/iter  (42.58 Β΅s … 137.96 Β΅s)  43.04 Β΅s  78.25 Β΅s  80.33 Β΅s

summary
  wasm
   15.92x faster than js (string)
   242.72x faster than js (array)

Node

src/node/bench/pack_right.bench.ts
cpu: Apple M1 Max
runtime: node v18.12.1 (arm64-darwin)

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   (index)   β”‚     average      β”‚   minimum   β”‚   maximum   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚    wasm     β”‚  '2.44 ΞΌs/iter'  β”‚  '2.04 ΞΌs'  β”‚ '111.17 ΞΌs' β”‚
β”‚ js (array)  β”‚ '118.16 ΞΌs/iter' β”‚ '110.13 ΞΌs' β”‚ '464.25 ΞΌs' β”‚
β”‚ js (string) β”‚ '42.15 ΞΌs/iter'  β”‚ '40.62 ΞΌs'  β”‚  '1.14 ms'  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Summary
- wasm is 48.40x faster than js (array)
- wasm is 17.26x faster than js (string)

xor_mod

Deno

cpu: Apple M1 Max
runtime: deno 1.30.0 (aarch64-apple-darwin)

file:///src/deno/bench/xor_mod.bench.ts
benchmark      time (avg)             (min … max)       p75       p99      p995
------------------------------------------------- -----------------------------
wasm       888.48 ns/iter    (874.06 ns … 1.1 Β΅s) 885.63 ns    1.1 Β΅s    1.1 Β΅s
js           1.16 Β΅s/iter     (1.15 Β΅s … 1.17 Β΅s)   1.16 Β΅s   1.17 Β΅s   1.17 Β΅s

summary
  wasm
   1.3x faster than js

Node

src/node/bench/xor_mod.bench.ts
cpu: Apple M1 Max
runtime: node v18.12.1 (arm64-darwin)

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ (index) β”‚     average      β”‚   minimum   β”‚   maximum   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  wasm   β”‚ '880.48 ns/iter' β”‚ '750.00 ns' β”‚ '154.00 ΞΌs' β”‚
β”‚   js    β”‚ '17.71 ΞΌs/iter'  β”‚ '17.42 ΞΌs'  β”‚ '610.67 ΞΌs' β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Summary
- wasm is 20.11x faster than js

Usage

Concatening bits

import { Naberius, unpack, pack_right } from "@hazae41/naberius";

// Wait for WASM to load
Naberius.initSyncBundledOnce()

// Create a header of bits
const headerBits = new Uint8Array([0x00, 0x01, 0x00, 0x01])

// Create a body of bytes
const bodyBytes = new Uint8Array(256)
crypto.getRandomValues(bodyBytes)

// Unpack it
const bodyBitsSlice = unpack(bodyBytes)

// Concat both bits arrays
const fullBits = new Uint8Array(headerBits.length   bodyBitsSlice.bytes.length)
fullBits.set(headerBits, 0)
fullBits.set(bodyBitsSlice.bytes, headerBits.length)

bodyBitsSlice.free()

// Pack adding 0-padding to the right
const fullBytes = pack_right(fullBits).copyAndDispose()

Xoring with mask

const bytes = new Uint8Array(1024)
crypto.getRandomValues(bytes)

const mask = new Uint8Array(4)
crypto.getRandomValues(mask)

const xored = xor_mod(bytes, mask).copyAndDispose()
const unxored = xor_mod(xored, mask).copyAndDispose()

Building

Unreproducible building

You need to install Rust

Then, install wasm-pack

cargo install wasm-pack

Finally, do a clean install and build

npm ci && npm run build

Reproducible building

You can build the exact same bytecode using Docker, just be sure you're on a linux/amd64 host

docker compose up --build

Then check that all the files are the same using git status

git status --porcelain

If the output is empty then the bytecode is the same as the one I commited

Automated checks

Each time I commit to the repository, the GitHub's CI does the following:

  • Clone the repository
  • Reproduce the build using docker compose up --build
  • Throw an error if the git status --porcelain output is not empty

Each time I release a new version tag on GitHub, the GitHub's CI does the following:

  • Clone the repository
  • Do not reproduce the build, as it's already checked by the task above
  • Throw an error if there is a npm diff between the cloned repository and the same version tag on NPM

If a version is present on NPM but not on GitHub, do not use!