PromptFuzz is an automated tool that generates high-quality fuzz drivers for libraries via a fuzz loop constructed on mutating LLMs' prompts. The fuzz loop of PromptFuzz aims to guide the mutation of LLMs' prompts to generate programs that cover more reachable code and explore complex API interrelationships, which are effective for fuzzing.

PromptFuzz is currently regarded as the leading approach for generating fuzz drivers both in academia and industry. The fuzz drivers generated by PromptFuzz achieved a branch coverage of 40.12% on the tested libraries, which is 1.61x greater than OSS-Fuzz and 1.67x greater than Hopper. Besides, PromptFuzz detected 33 valid security bugs from 49 unique crashes.

• Multiply LLM support: Supports the general LLMs: Codex, Incoder, ChatGPT, and GPT4 (Currently tested on ChatGPT).
• Context-based Prompt: Construct LLM prompts with the automatically extracted library context.
• Powerful Sanitization: The program's syntax, semantics, behavior, and coverage are thoroughly analyzed to sanitize the problematic programs.
• Prioritized Mutation: Prioritizes mutating the library API combinations within LLM's prompts to explore complex interrelationships, guided by code coverage.
• Fuzz Driver Exploitation: Infers API constraints using statistics and extends fixed API arguments to receive random bytes from fuzzers.
• Fuzz engine integration: Integrates with grey-box fuzz engine: LibFuzzer.

The fuzz drivers generated by PromptFuzz can detect a wide range of bugs, most of which are security bugs. For instances, CVE-2023-6277, CVE-2023-52355 and CVE-2023-52356.

PromptFuzz detects uniquely interesting bugs:

You can use the Dockerfile to build the environment:

docker build -t promptfuzz .
docker run -it promptfuzz bash

Before you apply this fuzzer for a new project, you must have a automatic build script to build your project to prepare the required data (e.g., headers, link libraries, fuzzing corpus and etc.), like OSS-Fuzz. See Preparation.

We have prepared the build scripts for some popular open source libraries, you can refer to the data directory.

If you prefer to set up the environment locally instead of using Docker, you can follow the instructions below:

Requirements:

• Rust stable
• LLVM and Clang (built with compiler-rt)
• wllvm (installed by pip3 install wllvm)

You can download llvm and clang from this link or install by llvm.sh.

Explicit dependency see Dockerfile.

If you prefer build llvm manually, you can build clang with compiler-rt and libcxx from source code following the config:

cmake -S llvm -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS="clang;lld" \
 -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;compiler-rt;" \
 -DCMAKE_BUILD_TYPE=Release -DLIBCXX_ENABLE_STATIC_ABI_LIBRARY=ON \
 -DLIBCXXABI_ENABLE_SHARED=OFF  -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang   

Those custom LLMs have not been fully supported and tested in PromptFuzz. If you just want to use PromptFuzz, please choose ChatGPT or GPT4.

Currently, the primary programming language used for implementation is Rust, while a few Python scripts are utilized to invoke specific LLM models.

If you want to invoke the self-build LLMs (i.e., Incoder), the following is the requirements for building Python dependency:

- pytorch (pip3 install torch)
- transformers (pip3 install transformers)
- yaml (pip3 install PyYAML)
- fastapi (pip3 install fastapi[all])

Run the script in the prompt_fuzz/data directory, to prepare the required data of this library.

After the build process is finished, the data of this library is stored under prompt_fuzz/output/build/.

You must have an OPENAI_API_KEY in advance if you choice ChatGPT and GPT4. If you don't have that key, apply it from OpenAI in advance.

user@ubuntu$ export OPENAI_API_KEY=$(your_key)

If you encounter difficulties in accessing the OPENAI service from your IP location, you can utilize a proxy to redirect your requests as follows:

user@ubuntu$ export OPENAI_PROXY_BASE=https://openai.proxy.com/v1

Here, openai.proxy.com should be the location of your personal openai service proxy.

PromptFuzz generates fuzz drivers in a fuzz loop. There are several options that can be tuned in the configuration of promptfuzz.

Typically, the only options that need to be actively set are -c and -r. The -c option determines the number of cores to be used for sanitization. Enabling the -r option will periodically re-check the correctness of the seed programs, reducing false positives but also introducing some extra overhead.

For instance, the following command is sufficient to perform fuzzing on libaom:

cargo run --bin fuzzer -- libaom -c $(nproc) -r

The detailed configurations of promptfuzz:

user@ubuntu$ cargo run --bin fuzzer -- --help

Once the fuzz drivers generated finish, you should follow the follow steps to run the fuzz drivers and detect bugs.

Take libaom is an example, you can run this command to fuse the programs into a fuzz driver that can be fuzzed:

cargo run --bin harness -- libaom fuse-fuzzer

And, you can execute the fuzzers you fused:

cargo run --bin harness -- libaom fuzzer-run

Note that, promptfuzz implements the mechanism to detect the crashed program inside the fused fuzz driver. If a crash of a program has detected, promptfuzz will disable the code of the crashed program, which enables an continuously fuzzing. So, ensure that executing the fuzz drivers in PromptFuzz.

After 24 hours execution(s), you should deduplicate the reported crashes by PromptFuzz:

cargo run --bin harness -- libaom sanitize-crash

Then, you can collect and verbose the code coverage of your fuzzers by:

cargo run --bin harness -- libaom coverage collect

and

cargo run --bin harness -- libaom coverage report

We also provide a harness named harness to facilitate you access some core components of PromptFuzz.

Here is the command input of harness:

#[derive(Subcommand, Debug)]
enum Commands {
    /// check a program whether is correct.
    Check { program: PathBuf },
    /// Recheck the seeds whether are correct.
    ReCheck,
    /// transform a program to a fuzzer.
    Transform {
        program: PathBuf,
        #[arg(short, default_value = "true")]
        use_cons: bool,
        /// corpora used to perform transform check
        #[arg(short = 'p', default_value = "None")]
        corpora: Option<PathBuf>,
    },
    /// Fuse the programs in seeds to fuzzers.
    FuseFuzzer {
        /// transform fuzzer with constraints
        #[arg(short, default_value = "true")]
        use_cons: bool,
        /// the number of condensed fuzzer you want to fuse
        #[arg(short, default_value = "1")]
        n_fuzzer: usize,
        /// the count of cpu cores you could use
        #[arg(short, default_value = "10")]
        cpu_cores: usize,
        seed_dir: Option<PathBuf>,
    },
    /// Run a synthesized fuzzer in the fuzz dir.
    FuzzerRun {
        /// which fuzzer you want to run. default is "output/$Library/fuzzers"
        #[arg(short = 'u', default_value = "true")]
        use_cons: bool,
        /// the amount of time you wish your fuzzer to run. The default is 86400s (24 hours), the unit is second. 0 is for unlimit.
        time_limit: Option<u64>,
        /// whether minimize the fuzzing corpus before running
        minimize: Option<bool>,
    },
    /// collect code coverage
    Coverage {
        /// Coverage kind to collect
        kind: CoverageKind,
        /// -u means the exploit fuzzers
        #[arg(short = 'u', default_value = "true")]
        exploit: bool,
    },
    Compile {
        kind: Compile,
        #[arg(short = 'u', default_value = "true")]
        exploit: bool,
    },
    /// infer constraints
    Infer,
    /// Minimize the seeds by unique branches.
    Minimize,
    /// Sanitize duplicate and spurious crashes
    SanitizeCrash {
        #[arg(short = 'u', default_value = "true")]
        exploit: bool,
    },
    /// archive the results
    Archive { suffix: Option<String> },
    ///  Build ADG from seeds
    Adg {
        /// ADG kind to build: sparse or dense
        kind: ADGKind,
        /// The path of target programs to build the ADG.
        target: Option<PathBuf>,
    },
}

• Custom LLMs support: Support custom LLMs.
• Close-source libraries: Apply PromptFuzz to close-source libraries by fine tuning LLMs on private code corpus.
• Generalization: Generalize PromptFuzz to binary programs.