Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add cuDNN 4 support #19

Open
MichaelHirn opened this issue Mar 1, 2016 · 2 comments
Open

add cuDNN 4 support #19

MichaelHirn opened this issue Mar 1, 2016 · 2 comments

Comments

@MichaelHirn
Copy link
Member

This issue tracks full cuDNN4 support with rust-cudnn.

cuDNN 4 does not introduce many new things and rust-cudnn works already pretty well with the cuDNN 4 library. (see notable update below) Mostly it makes improvements to the Normalization API and internally improves the convolution performance. From the cuDNN 4 release notes.

_New Features_
- Batch Normalization routines have been added.
- Convolution forward and backward now supports NHWC tensor format.
- FFT Tiling algorithm has been added for cudnnConvolutionForward and cudnnConvolutionBackwardData routines
- cudnnConvolutionForward now supports computation in FP16 when run on GPU
with a compute capability >= 5.3
- cudnnConvolutionForward has been optimized for batch size = 1
- Pooling and activation routines have a descriptor option to propagate NaN numbers.

One notable update of cuDNN 4 is

Performance of cudnnConvolutionBackwardFilter when using Algo 1 has been
improved for some cases. This code path now also requires a workspace.

which affects collenchyma-nn as it makes the convolution algorithm inconsistent when switching from cuDNN3 to cuDNN4.

@hobofan
Copy link
Member

hobofan commented Mar 2, 2016

#21 adds "passive" cuDNNv4 support, meaning that it knows about the FFT Tiling algorithm, so it won't break when cuDNNv4 is used, but doesn't implement any of the other new features.

@jramapuram
Copy link

Would be nice to have forward/backward pass for LSTM's.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants