Have you ever wanted to benchmark a baby Llama2 models in 12 programming languages? No? Well, now you can!
- Clone this repo
- Clone & build hypertune & make it avaialble in your PATH
- Edit
config.sh
# set workdir that should contain directory `models` with `*.bin` models
export WORKDIR=~/opensource
# optionally set the number of benchmark rounds
export BENCHMARK_ROUNDS=30
# also it"s possible to adjust the number of threads
export THREADS=4- Edit
bench/compare2.shchange the VERSIONS definition
VERSIONS=(
"v1,https://github.com/tairov/llama2.mojo.git,master"
"v2,[email protected]:andresnowak/llama2.mojo.git,master"
)- Run
compare2.shwith bash
only bash supported for now, since compare2.sh uses exported bash functions for passing to hypertune
bash bench/compare2.sh
It will generate report in /tmp directory & open it in browser.