Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LSTM, GRU and RNN implementation for ChainerX #7764

Merged
merged 67 commits into from
Aug 19, 2019
Merged
Changes from 1 commit
Commits
Show all changes
67 commits
Select commit Hold shift click to select a range
c1ed7a5
implementing simple lstm function
dido1998 May 16, 2019
f88460e
implementation lstm activation function
dido1998 May 18, 2019
f6954d2
implementation of n_step_lstm
dido1998 May 25, 2019
2f142de
fix formatting
dido1998 May 25, 2019
0db2e42
fix formatting
dido1998 May 25, 2019
3124124
merge with master
dido1998 May 25, 2019
a11cb63
fix compile errors
dido1998 May 25, 2019
7adc07e
fix python formatting
dido1998 May 25, 2019
3830e68
fix errors in test_connection.py
dido1998 May 25, 2019
c449375
fix formatting
dido1998 May 25, 2019
e6755ca
fix syntax errors
dido1998 May 25, 2019
0a77595
fix return error
dido1998 May 25, 2019
b66996b
fix c formatting
dido1998 May 25, 2019
5692af0
implemetation of n_step_bi_lstm
dido1998 May 26, 2019
d01d703
fix c formatting
dido1998 May 27, 2019
b0ee988
added cuddn rnn forward implementation
dido1998 Jun 6, 2019
14b2d0b
cudnn rnn implementation
dido1998 Jun 13, 2019
405bc36
remove unnecessary debug messages
dido1998 Jun 13, 2019
6736a02
adding remaining files
dido1998 Jun 14, 2019
3a0ec64
Merge branch 'LSTM' of https://github.com/dido1998/chainer into LSTM
dido1998 Jun 14, 2019
e9cdba8
cudnn code fixed
dido1998 Jun 19, 2019
e5ab97b
adding incomplete test code for cuda LSTM
dido1998 Jun 20, 2019
74085fa
completed cudnn error-free code
dido1998 Jun 23, 2019
6d48e97
changing usage of AsContiguous
dido1998 Jun 24, 2019
da42d96
adding deleted file
dido1998 Jun 27, 2019
87c7678
using the same descriptor for forward and backward pass
dido1998 Jun 27, 2019
fdcaa1a
merge with chainerx master
dido1998 Jun 28, 2019
9914ee8
fixing compile errors
dido1998 Jun 28, 2019
f38147a
working backward pass for cudnn n_step_lstm
dido1998 Jun 29, 2019
ca6a79d
fixing cpplint errors
dido1998 Jun 29, 2019
c6b726b
fixing clang errors
dido1998 Jun 29, 2019
a03b9a1
fix clang errors
dido1998 Jun 29, 2019
84ed04d
adding GRU base function
dido1998 Jun 30, 2019
3cf04fc
native implementation of n_step_gru and n_step_bigru
dido1998 Jul 2, 2019
91a9b8c
cudnn implementation of n_step_gru and n_step_bigru
dido1998 Jul 3, 2019
1a363d2
fix clang errors
dido1998 Jul 3, 2019
9cb248a
Merge pull request #18 from chainer/master
dido1998 Jul 4, 2019
e8d9135
fix incorrect include
dido1998 Jul 4, 2019
8e1092a
fix clang errors
dido1998 Jul 4, 2019
698ee3c
Merge branch 'GRU' of https://github.com/dido1998/chainer into GRU
dido1998 Jul 4, 2019
06fcd9c
fix incorrect include
dido1998 Jul 4, 2019
0993771
minor fixes
dido1998 Jul 4, 2019
5c2a912
fix minor clang error
dido1998 Jul 4, 2019
5653d91
fix minor clang errors
dido1998 Jul 4, 2019
0243f76
vanilla rnn implementation
dido1998 Jul 14, 2019
5fbf512
minor changes
dido1998 Jul 15, 2019
3022131
fix clang errors
dido1998 Jul 15, 2019
d129f81
changing tolerance for float16 and handling non-differentiable point …
dido1998 Jul 18, 2019
5623a03
adding documentation and making minor changes
dido1998 Aug 1, 2019
5b2ea53
fix clang errors
dido1998 Aug 5, 2019
233026b
Merge pull request #30 from chainer/master
dido1998 Aug 7, 2019
1cec24f
integrating chainer with chainerx
dido1998 Aug 7, 2019
2fc9110
Merge pull request #33 from chainer/master
dido1998 Aug 8, 2019
8063ba4
integrating chainerx into chainer for GRU
dido1998 Aug 8, 2019
5a95348
integrating chainerx into chainer for lstm
dido1998 Aug 8, 2019
fd6347c
fix clang errors
dido1998 Aug 13, 2019
3dbce9d
Merge pull request #34 from chainer/master
dido1998 Aug 13, 2019
4b7295b
integrating chainer with chainerx for n_step_lstm
dido1998 Aug 13, 2019
3c5b822
converting names to camelcase and minor changes
dido1998 Aug 15, 2019
4911c33
making cosmetic changes
dido1998 Aug 15, 2019
74a7346
fix clang errors
dido1998 Aug 15, 2019
83a1c6c
fix clang errors
dido1998 Aug 16, 2019
becf72c
Merge pull request #36 from chainer/master
dido1998 Aug 16, 2019
9d292f7
cosmetic fixes
dido1998 Aug 17, 2019
07fb9ff
Merge branch 'VanillaRNN' of https://github.com/dido1998/chainer into…
dido1998 Aug 17, 2019
2767db3
minor changes
dido1998 Aug 19, 2019
a8d9806
fix clang-tidy errors
dido1998 Aug 19, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
adding GRU base function
  • Loading branch information
dido1998 committed Jun 30, 2019
commit 84ed04d1ffa0dacc430d2e5e278c255ac2c49210
21 changes: 21 additions & 0 deletions chainerx_cc/chainerx/routines/n_step_rnn.cc
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 47,27 @@ Array _stack_weight(const std::vector<Array>& ws) {
return w;
}

std::vector<Array> _gru(
const Array& x, const Array& h, const nonstd::optional<Array>& c, const std::vector<Array>& ws, const std::vector<Array>& bs) {
Array xw = Concatenate({ws[0], ws[1], ws[2]}, 0);
Array hw = Concatenate({ws[3], ws[4], ws[5]}, 0);
Array xb = Concatenate({bs[0], bs[1], bs[2]}, 0);
Array hb = Concatenate({bs[3], bs[4], bs[5]}, 0);

Array gru_x = Linear(x, xw, xb);
Array gru_h = Linear(h, hw, hb);

std::vector<Array> split_w = Split(gru_x, 3, 1);
std::vector<Array> split_h = Split(gru_h, 3, 1);
Array r = Sigmoid(split_w[0] split_h[0]);
Array z = Sigmoid(split_w[1] split_h[1]);
Array h_bar = Tanh(split_w[2] r * split_h[2]);
std::vector<Array> out{};
out.push_back((Array)NULL);
out.push_back((1 - z) * h_bar z * h);
return out;
}

std::vector<Array> _lstm(
niboshi marked this conversation as resolved.
Show resolved Hide resolved
const Array& x, const Array& h, const nonstd::optional<Array>& c, const std::vector<Array>& ws, const std::vector<Array>& bs) {
std::vector<Array> ws_0_4{ws[2], ws[0], ws[1], ws[3]};
Expand Down