This is a collection of command-line and GUI tools for capturing and analyzing audio data.

The most interesting tool is called keytap - it can guess pressed keyboard keys only by analyzing the audio captured from the computer's microphone.

Check this blog post for more details:

Keytap: description and some random thoughts

Video: short demo of Keytap in action

The keytap2 tool is another interesting tool for recovering text from audio. It does not require training data - instead it uses statistical information about the frequencies of the letters and n-grams in the English language. A more detailed description of the tool is available here: Keytap2 discussion

Online demo using WASM

Video: short demo of Keytap2 in action

CTF: can you guess the text being typed?

This version introduces significant algorithm improvements and better n-gram statistics. The attack is now fully automated and and does not require any manual intervation during the decoding process.

Check if your keyboard is vulnerable to Keytap

"This works incredibly well.
I hope you realize what you've created (and made available to every person in the world)." -- ffpip

"I just tried it and it works incredibly well. It kind of makes me want to stop using a mechanical keyboard." -- Karawebnetwork

"This attack and Van Eck phreaking are why Edward Snowden, while typing passwords and other sensitive information, would pull a blanket over himself and his laptop." -- aarchi

"This is what mechanical keyboard users deserve" -- super guy

"fuck.." -- Lluis Franco

Dependencies:

• SDL2 - used to capture audio and to open GUI windows libsdl

  [Ubuntu]
  $ sudo apt install libsdl2-dev
  
  [Mac OS with brew]
  $ brew install sdl2
  
  [MSYS2]
  $ pacman -S git cmake make mingw-w64-x86_64-dlfcn mingw-w64-x86_64-gcc mingw-w64-x86_64-SDL2
  

• FFTW3 (optional) - some of the helper tools perform Fourier transformations fftw

Linux, FreeBSD, Mac OS, Windows (MSYS2 MinGW)

git clone https://github.com/ggerganov/kbd-audio
cd kbd-audio
git submodule update --init
mkdir build && cd build
cmake ..
make

Short summary of the available tools. If the status of the tool is not stable, expect problems and non-optimal results.

• record-full

  Record audio to a raw binary file on disk

  ./record-full output.kbd [-cN]
  

  ---

• play-full

  Playback a recording captured via the record-full tool

  ./play-full input.kbd [-pN]
  

  ---

• record

  Record audio only while typing. Useful for collecting training data for keytap

  ./record output.kbd [-cN] [-CN]
  

  ---

• play

  Playback a recording created via the record tool

  ./play input.kbd [-pN]
  

  ---

• keytap

  Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool.

  ./keytap input0.kbd [input1.kbd] [input2.kbd] ... [-cN] [-CN] [-pF] [-tF]
  

  ---

• keytap-gui

  Detect pressed keys via microphone audio capture in real-time. Uses training data captured via the record tool. GUI version.

  ./keytap-gui input0.kbd [input1.kbd] [input2.kbd] ... [-cN] [-CN]
  

  **Live demo (WebAssembly threads required) **

  

  ---

• keytap2-gui record.kbd n-gram-dir [-pN] [-cN] [-CN]

  Detect pressed keys via microphone audio capture. Uses statistical information (n-gram frequencies) about the language. No training data is required. The 'record.kbd' input file has to be generated via the record-full tool and contains the audio data that will be analyzed. The 'n-gram-dir' folder file has to contain n-gram probability files for the corresponding language.

  ./keytap2-gui record.kbd ../data
  

  

  ---

• keytap3

  Fully automated recovery of unknown text from audio recordings.

  ./keytap3 input.kbd ../data [-cN] [-CN] [-pF] [-tF] [-FN] [-fN]
  

  Online demo: https://keytap3.ggerganov.com

  ---

• view-full-gui

  Visualize waveforms recorded with the record-full tool. Can also playback the audio data.

  ./view-full-gui input.kbd [-pN]
  

  

  ---

• view-gui

  Visualize training data recorded with the record tool. Can also playback the audio data.

  ./view-gui input.kbd [-pN]
  

  

  ---

Any feedback about the performance of the tools is highly appreciated. Please drop a comment here.