This release updates ABN
, InPlaceABN
and InPlaceABNSync
to feature parity with recent versions of Pytorch's BatchNormNd layers:
- Add a
track_running_stats
parameter to enable / disable computation of running statistics independently from the layer's training state - Add a
num_batches_tracked
buffer, and allow passingmomentum=None
to support cumulative moving average for tracking running stats instead of exponential moving average - As a side-effect, now support loading parameters from standard BatchNorm without work-arounds. Still, if the loaded parameters contain negative weight elements the output will differ compared to standard BatchNorm
Additional changes:
- Fix backward pass in eval mode: it was not properly accounting for the activation function
- Refactor code to follow more sensible formatting standards
- Add type annotations
- Improve docstrings
- Update installation instructions, pointing to the PyPI package