Identifies and/or fixes inconsistent whitespace and line endings in text files, so that they don't clog up your commits to version control systems like Git, Mercurial, or Subversion.
- Whitespace Total Fixer
- Exciting origin story
- Whitespace issues addressed
- Reporting
- Todo
- Bugs
- Author
- License
- It's like an incomprehensible shell-script one-liner (e.g.
sed -e 's/LINENOISE/'
), but way better - It's similar to
git stripspace
, but more flexible and detailed. wtf.py
is a simple Python script with no dependencies beyond the standard Python library. It should run correctly with either Python 2.7 or 3.x.
curl https://raw.githubusercontent.com/dlenski/wtf/master/wtf.py > ~/bin/wtf.py && chmod 0755 !#:3
See below for options to control exactly which whitespace issues it fixes, but here are some examples:
# consistent whitespace from programs that generate text files
dump_database_schema | wtf.py -o clean_output.sql
# in-place editing
wtf.py -i file1.txt file2.txt file3.txt
wtf.py -I.bak file1.txt file2.txt file3.txt # ditto, with backups
# summarize a bunch of files without actually editing them (-0)
find . -name "*.txt" -exec wtf.py -0 {} \;
# more advanced: find interesting text files. pass options and a directory
wtf-find() {
find "${@: -1}" -not \( -name .svn -prune -o -name .git -prune \
-o -name .hg -prune \) -type f -exec bash -c \
'grep -Il "" "$1" &>/dev/null && wtf.py '"${*:1:$#-1}"' "$1"' _ {} \;
}
wtf-find -0 .
# exit status
wtf.py file1.txt file2.txt file3.txt > /dev/null
if (( $? == 10 )); then
echo "issues fixed"
elif (( $? == 20 )); then
echo "unfixed issues!"
fi
You can use Git pre-commit
hooks
to automatically run wtf
and cleanup whitespace in your repository
prior to every commit.
(You can bypass the hook, however, with git commit [--no-verify|-n]
.)
The pre-commit.careful hook
will run wtf
, with the default options, on all the to-be-committed
text files. You can easily use it as-is by creating a symlink, for example
by running the following in your repository's working directory:
ln -s /path/to/wtf/source/pre-commit.careful .git/hooks/pre-commit
If you want to modify it, create the file .git/hooks/pre-commit
in your
repository's working directory, and ensure that it is executable (with
chmod x .git/hooks/pre-commit
).
Features:
- This hook only modifies your commits, and does not touch Git's
working tree. If you understand and use Git's index ("staging area"),
this should make sense. It will play nicely with
git add --patch
. - It skips over files marked as
binary
or non-text (-text
) according to your repository's https://git-scm.com/docs/gitattributes - Note that when a file changes in your commit but remains unchanged in
your working tree, the file will be marked
modified
bygit status
immediately after the commit.
This version modifies Git's working tree as well as your commits. This version will not play nicely with
git add --patch
.
This version will run wtf -i
, with any other options you add to wtf_options
, on all the
to-be-committed text files. They will be cleaned up in the commit, as
well as in your working tree.
#!/bin/sh
wtf_options=''
# get a list of to-be-committed filepaths, EXCLUDING files considered
# by Git to be binary
committees=$(git diff --cached --numstat --diff-filter=ACMRTU|egrep -v ^-|cut -f3-)
# Run Whitespace Total Fixer in-place, and re-add files modified by it
for committee in $committees
do
wtf -i $wtf_options "$committee" || git add "$committee"
done
One day at work, I spent way too much time wrangling commits laden with whitespace issues generated by recalcitrant text-editing tools on multiple platforms.
That evening, I went home and spent way too much time writing this program instead!
WTF currently fixes, or simply reports, a few common types of whitespace issues. Most of these issues offer three possible command-line options enabling the user to fix, report, or ignore the issue.
-
Remove trailing spaces at the ends of lines (default is fix):
-t, --trail-space -T, --report-trail-space -It, --ignore-trail-space
-
Remove blank lines at the ends of files (default is fix):
-b, --eof-blanks -B, --report-eof-blanks -Ib, --ignore-eof-blanks
-
Ensure that a new-line character appears at the end of the file (default is fix):
-n, --eof-newl -N, --report-eof-newl -In, --ignore-eof-newl
-
Make sure that all lines have matching EOL markers (that is, no mixing of lf/crlf). Default is to fix non-matching EOL characters by making them all the same as the first line of the file. The desired EOL markers can also be set to a specific value (
lf
/crlf
/native
), in which case all lines will unconditionally receive this marker.-E ENDING, --coerce-eol ENDING -e ENDING, --expect-eol ENDING -Ie, --ignore-eol
-
Check for spaces followed by tabs in the whitespace at the beginning of a line; fixing this condition requires setting either
--change-tabs
or--change-spaces
. The default is to report and warn:-s, --tab-space-mix -S, --report-tab-space-mix -Is, --ignore-tab-space-mix
-
Change tabs in the whitespace at the beginning of a line to the specified number of spaces (default is not to change tabs at all). This option will not touch leading tabs when they are mixed with spaces, unless
--tab-space-mix
is also specified.-x SPACES, --change-tabs SPACES
-
Change the specified number of consecutive spaces in the whitespace at the beginning of a line to tabs (default is not to change spaces at all). This option will not touch leading tabs when they are mixed with spaces, unless
--tab-space-mix
is also specified.-y SPACES, --change-spaces SPACES
Unless the -q
/--quiet
option is used, WTF will summarize each file
processed in which any whitespace issues were found and/or fixed. With
-v
it will also report issue-free files.
$ wtf -0 nightmare.txt # -0 is equivalent to > /dev/null
nightmare.txt LINE 8: WARNING: spaces followed by tabs in whitespace at beginning of line
nightmare.txt:
CHOPPED 1 lines with trailing space
CHOPPED 0 blank lines at EOF
ADDED newline at EOF
CHANGED 1 line endings which didn't match crlf from first line
WARNED ABOUT 1 lines with tabs/spaces mix
WTF will return the following exit codes on successful operation:
0
: no issues seen (or-X
/--no-exit-codes
specified)10
: issues fixed20
: unfixed issues seen
- Stability tests?
- Unicode tests?
Corrupts source code files written in the Whitespace programming language.
Anything else?
If you are having problems with tab completion in Bash,
it is probably because of default completion rules for the unrelated
utility wtf(6)
.
You can override this by adding the following to your ~/.bashrc
:
complete -o default -o bashdefault wtf
© Daniel Lenski <[email protected]> (2014-2020)