High-performance JSON masker

JSON masker library which can be used to mask sensitive values inside JSON corresponding to a set of keys (block-mode) or, alternatively, allow only specific values to be unmasked corresponding to a set of keys while all others are masked (allow-mode).

The library provides a modern and convenient Java API which offers a wide range of masking customizations. Furthermore, the implementation is focused on maximum (time) performance and minimal heap allocations.

Finally, no additional third-party runtime dependencies are required to use this library.

Features

Mask all primitive values by specifying the keys to mask, by default any string is masked as "***", any number as "###" and any boolean as "&&&"
If the value of a targeted key corresponds to an object, all nested fields, including nested arrays and objects will be masked, recursively
If the value of a targeted key corresponds to an array, all values of the array, including nested arrays and objects, will be masked, recursively
Ability to define different masking strategies, per value type
- (default) mask strings with a different string: "maskMe": "secret" -> "maskMe": "***"
- mask characters of a string with a different character: "maskMe": "secret" -> "maskMe": "*****" (preserves length)
- (default) mask numbers with a string: "maskMe": 12345 -> "maskMe": "###" (changes number type to string)
- mask numbers with a different number: "maskMe": 12345 -> "maskMe": 0 (preserves number type)
- mask digits of a number with a different digit: "maskMe": 12345 -> "maskMe": 88888 (preserves number type and length)
- (default) mask booleans with a string: "maskMe": true -> "maskMe": "&&&" (changes boolean type to string)
- mask booleans with a different boolean: "maskMe": true -> "maskMe": false (preserves boolean type)
Ability to define masking strategy per key
Target key case sensitivity configuration (default: false)
Use block-list (maskKeys) or allow-list (allowKeys) for masking
Limited support for JsonPath masking in block-list (maskJsonPaths) and allow-list (allowJsonPaths) modes
Masking a valid JSON will always return a valid JSON

Note: Since RFC-8529 dictates that JSON exchanges between systems that are not part of an enclosed system MUST be encoded using UTF-8, the library only support UTF-8 encoding.

JDK Compatibility

The json-masker baseline JDK requirement is JDK 17. However, we would be open to add a version which lowers this requirement to JDK 11, on request.

Usage examples

JsonMasker instance can be created using any of the following factory methods:

// block-mode, default masking config
var jsonMasker = JsonMasker.getMasker(Set.of("email", "iban"));

// block-mode, default masking config (using a builder)
var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskKeys(Set.of("email", "iban"))
                .build()
);

// block-mode, JSONPath
var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskJsonPaths(Set.of("$.email", "$.nested.iban"))
                .build()
);

// allow-mode, default masking config
var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .allowKeys(Set.of("id", "name"))
                .build()
);

// allow-mode, JSONPath
var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskJsonPaths(Set.of("$.id", "$.nested.name"))
                .build()
);

Using JsonMaskingConfig allows customizing the masking behaviour of types, keys or JSONPath or mix keys and JSON paths.

Note

Whenever a simple key (maskKeys(Set.of("email", "iban"))) is specified, it is going to be masked recursively regardless of the nesting, whereas using a JsonPath (maskJsonPaths(Set.of("$.email", "$.iban"))) would only mask those keys on the top level JSON

After creating the JsonMasker instance, it can be used to mask a JSON as following:

String maskedJson = jsonMasker.mask(json);

The mask method is thread-safe, and it is advised to reuse the JsonMasker instance as it pre-processes the masking (allowed) keys for faster lookup during the actual masking.

Default JSON masking

Example of masking fields (block-mode) with a default config

Usage

var jsonMasker = JsonMasker.getMasker(Set.of("email", "age", "visaApproved", "iban", "billingAddress"));

String maskedJson = jsonMasker.mask(json);

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "***",
    "age": "###",
    "visaApproved": "&&&"
  },
  "payment": {
    "iban": "***",
    "successful": true,
    "billingAddress": [
      "***",
      "***"
    ]
  },
  "companyContact": {
    "email": "***"
  }
}

Allow-list approach

Example showing an allow-list based approach of masking a JSON.

Usage

var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .allowKeys(Set.of("orderId", "id", "travelPurpose", "successful"))
                .build()
);

String maskedJson = jsonMasker.mask(json);

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "***",
    "age": "###",
    "visaApproved": "&&&"
  },
  "payment": {
    "iban": "***",
    "successful": true,
    "billingAddress": [
      "***",
      "***"
    ]
  },
  "companyContact": {
    "email": "***"
  }
}

Overriding default masks

The default masks can be overridden for any type.

Usage

var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskKeys(Set.of("email", "age", "visaApproved", "iban", "billingAddress"))
                .maskStringsWith("[redacted]")
                .maskNumbersWith("[redacted]")
                .maskBooleansWith("[redacted]")
                .build()
);

String maskedJson = jsonMasker.mask(json);

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[redacted]",
    "age": "[redacted]",
    "visaApproved": "[redacted]"
  },
  "payment": {
    "iban": "[redacted]",
    "successful": true,
    "billingAddress": [
      "[redacted]",
      "[redacted]"
    ]
  },
  "companyContact": {
    "email": "[redacted]"
  }
}

Masking with JsonPath

To have more control over the nesting, JsonPath can be used to specify the keys that needs to be masked (allowed).

Usage

var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskJsonPaths(Set.of(
                        "$.customerDetails.email",
                        "$.customerDetails.age",
                        "$.customerDetails.visaApproved",
                        "$.payment.iban",
                        "$.payment.billingAddress"
                ))
                .build()
);

String maskedJson = jsonMasker.mask(json);

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "***",
    "age": "###",
    "visaApproved": "&&&"
  },
  "payment": {
    "iban": "***",
    "successful": true,
    "billingAddress": [
      "***",
      "***"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Masking with preserving the type

The following configuration might be useful where the value must be masked, but the type needs to be preserved, so that the resulting JSON can be parsed again or if the strict JSON schema is required.

Usage

var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskKeys(Set.of("email", "age", "visaApproved", "iban", "billingAddress"))
                .maskNumbersWith(0)
                .maskBooleansWith(false)
                .build()
);

String maskedJson = jsonMasker.mask(json);

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "***",
    "age": 0,
    "visaApproved": false
  },
  "payment": {
    "iban": "***",
    "successful": true,
    "billingAddress": [
      "***",
      "***"
    ]
  },
  "companyContact": {
    "email": "***"
  }
}

Masking with preserving the length

Example showing masking where the length of the original value (string or number) is preserved.

Usage

var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskKeys(Set.of("email", "age", "visaApproved", "iban", "billingAddress"))
                .maskStringCharactersWith("*")
                .maskNumberDigitsWith(8)
                .build()
);

String maskedJson = jsonMasker.mask(json);

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "*******************************",
    "age": 88,
    "visaApproved": "&&&"
  },
  "payment": {
    "iban": "**********************",
    "successful": true,
    "billingAddress": [
      "*************",
      "*****************"
    ]
  },
  "companyContact": {
    "email": "*************"
  }
}

Masking with using a per-key masking configuration

When using a JsonMaskingConfig you can also define a per-key masking configuration, which allows to customize the way certain values are masked.

Usage

var jsonMasker = JsonMasker.getMasker(
        JsonMaskingConfig.builder()
                .maskKeys(Set.of("email", "age", "visaApproved", "billingAddress"))
                .maskKeys(Set.of("iban"), KeyMaskingConfig.builder()
                        .maskStringCharactersWith("*")
                        .build())
                .build()
);

String maskedJson = jsonMasker.mask(json);

Note

When defining a config for the specific key and value of that key is an object or an array, the config will apply recursively to all nested keys and values, unless the nested key(s) defines its own masking configuration.

If config is attached to a JsonPath it has a precedence over a regular key.

Input

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "[email protected]",
    "age": 29,
    "visaApproved": true
  },
  "payment": {
    "iban": "NL91 FAKE 0417 1643 00",
    "successful": true,
    "billingAddress": [
      "Museumplein 6",
      "1071 DJ Amsterdam"
    ]
  },
  "companyContact": {
    "email": "[email protected]"
  }
}

Output

{
  "orderId": "789 123 456",
  "customerDetails": {
    "id": 1,
    "travelPurpose": "business",
    "email": "***",
    "age": "###",
    "visaApproved": "&&&"
  },
  "payment": {
    "iban": "**********************",
    "successful": true,
    "billingAddress": [
      "***",
      "***"
    ]
  },
  "companyContact": {
    "email": "***"
  }
}

Dependencies

The library has no third-party runtime dependencies
The library only has a single JSR-305 compilation dependency for nullability annotations
The test/benchmark dependencies for this library are listed in the build.gradle

Performance

The json-masker library is optimized for a fast key lookup that scales well with a large key set to mask (or allow). The input is only scanned once and memory allocations are avoided whenever possible.

Benchmarks

For benchmarking, we compare the implementation against multiple baseline benchmarks, which are:

Counting the bytes of the JSON message without doing any other operation
Using Jackson to parse a JSON message into JsonNode and masking it by iterating over and replacing all values corresponding to the targeted keys
A naive regex masking (replacement) implementation.

Generally our implementation is ~15-25 times faster than using Jackson (besides the additional benefits of no runtime dependencies and a convenient API out-of-the-box).

Benchmark                              (characters)  (jsonPath)  (jsonSize)  (maskedKeyProbability)   Mode  Cnt     Score   Error  Units
BaselineBenchmark.countBytes                unicode         N/A         2mb                    0.01  thrpt    2  1525.281          ops/s
BaselineBenchmark.jacksonParseAndMask       unicode         N/A         2mb                    0.01  thrpt    2     3.353          ops/s
BaselineBenchmark.regexReplace              unicode         N/A         2mb                    0.01  thrpt    2     2.684          ops/s
JsonMaskerBenchmark.jsonMaskerBytes         unicode       false         2mb                    0.01  thrpt    2    79.309          ops/s
JsonMaskerBenchmark.jsonMaskerBytes         unicode        true         2mb                    0.01  thrpt    2    80.745          ops/s
JsonMaskerBenchmark.jsonMaskerString        unicode       false         2mb                    0.01  thrpt    2    48.488          ops/s
JsonMaskerBenchmark.jsonMaskerString        unicode        true         2mb                    0.01  thrpt    2    49.345          ops/s

Name		Name	Last commit message	Last commit date
Latest commit History 207 Commits
.github		.github
adr		adr
gradle		gradle
src		src
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
build.gradle.kts		build.gradle.kts
gradle.properties		gradle.properties
gradlew		gradlew
gradlew.bat		gradlew.bat
settings.gradle.kts		settings.gradle.kts

License

Sprokof/json-masker

Folders and files

Latest commit

History

Repository files navigation

High-performance JSON masker

Features

JDK Compatibility

Usage examples

Default JSON masking

Usage

Input

Output

Allow-list approach

Usage

Input

Output

Overriding default masks

Usage

Input

Output

Masking with JsonPath

Usage

Input

Output

Masking with preserving the type

Usage

Input

Output

Masking with preserving the length

Usage

Input

Output

Masking with using a per-key masking configuration

Usage

Input

Output

Dependencies

Performance

Benchmarks

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages