-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to improve training accuracy? #8
Comments
Hi, sorry about the delay in responding. Here's some inputs:
|
Your help was very much appreciated. |
Yeah, the train function in secondary.cpp should be the right one. And you're right, the division function is yet to be implemented in full generality. Thanks for working on it, feel free to check in your edits. Finally, about the learning rate, each layer should use the This is all I can think of for now. My suggestions for your third question initially (quoted below) should be useful. If you have more issues/questions, feel free to open up this issue again.
|
Hi snwagh, Now I am trying to use this codebase to realize the training of the "MiniONN" over the MNIST dataset. |
About that TODO, it must be something left over from one of the commits. I think I have completed that piece of code but that only thing that is left is a thorough testing of that component. So effectively, if you backprop through a Conv layer over plaintext (say through PyTorch), does the output of the MPC code agree with that of the PyTorch code (this has to be tested). What do you mean by realization? Do you mean any pointers on how to get it trained with good accuracy? I think the general principles for that will be the same for all the smaller networks. One thing you can do is try to run one forward backward pass on the plaintext code and then compare that with one iteration of MPC code. If they agree, even one epoch should give you about 95% accuracy (because that is how much the plaintext gives and my hunch is that MPC approximations will not change it that much). |
thanks for the reply! |
Now I am trying to use this codebase to realize the training of the "SecureML" neural network over the MNIST dataset, but I have encountered some problems. Please could you advise me what I should do?
I fixed the data importing of the codebase, and the training label and testing label used the one-hot representation. could you help me confirm that this input format is correct? In addition, do I need to perform other preprocessing on the training data before importing the data?
I used the He-Initialization method to initialize the parameters(weights).
The following figure shows the training accuracy of 10 epochs, and the result is very unsatisfactory. Could you give me some valuable suggestions?
In addition, Can functions "backward","computeDelta" and "updateEquations" in the codebase be used directly?
The text was updated successfully, but these errors were encountered: