Skip to content

Fast word list cleaner and de-duplicator. Filters problem characters that play havoc with UTF-8 and removes duplicates to avoid inefficient use of resources. Allows removal of words below a specified length. Very handy when consolidating large wordlists or when combining multiple wordlists generated with cewl.

License

Notifications You must be signed in to change notification settings

blairjames/squeaky

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Squeaky

Fast word list cleaner and de-duplicator.

  • Filters problem characters that play havoc with UTF-8 and removes duplicates to avoid inefficient use of resources.
  • Allows removal of words below a specified length.
  • Very handy when optimizing and combining multiple large wordlists.

Usage:

git clone https://github.com/blairjames/squeaky.git
squeaky.py [-h] [-d] [-l LEN] [-u] input_file output_file

positional arguments:
--input_file
--output_file

options:
-h, --help - show this help message and exit.
-d, --dir - Input a directory to process for word lists. (".txt" files)
-l LEN, --len LEN - Minimum word length, words shorter than specified length will be removed.
-u, --unique - Delete duplicate words in word list.

About

Fast word list cleaner and de-duplicator. Filters problem characters that play havoc with UTF-8 and removes duplicates to avoid inefficient use of resources. Allows removal of words below a specified length. Very handy when consolidating large wordlists or when combining multiple wordlists generated with cewl.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published