A lightweight benchmarking tool based inspired by oha and powered by Ratatui.
make install-benchmarkFirst, start text-generation-inference:
text-generation-launcher --model-id bigscience/bloom-560mThen run the benchmarking tool:
text-generation-benchmark --tokenizer-name bigscience/bloom-560m