Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize myNetwork.standalone() #139

Open
MariasStory opened this issue Sep 9, 2016 · 7 comments
Open

Optimize myNetwork.standalone() #139

MariasStory opened this issue Sep 9, 2016 · 7 comments
Labels
enhancement

Comments

@MariasStory
Copy link

Hi.
Is there something wrong with myNetwork.standalone()?

I did test on:

var myNetwork = new synaptic.Architect.Perceptron(4, 4, 1);

var trainingSet = [
{input: [1,1,1,1], output: [0]},
{input: [1,0,1,1], output: [0]},
{input: [1,1,0,1], output: [0]},
{input: [1,0,0,1], output: [0]},
{input: [0,1,1,0], output: [1]},
{input: [0,0,1,0], output: [1]},
{input: [0,1,0,0], output: [1]},
{input: [0,0,0,0], output: [1]}
];`

The output of standalone is:

F = {
0: 0,
1: 1,
2: 0,
3: 1,
11: 0.5783836367920991, //overwritten
4: 3.269995990621269, //overwritten - newer used
5: 0.31614157133734766, //overwritten
6: -2.06530892462214,
7: 2.9538544192839216,
8: -0.4061988197735862,
9: -0.40684976444775095,
10: 2.787649315733074,
12: 0.2438560054832443, //overwritten - newer used
43: -6.256601162852674,
20: 0.3938994608292939, //overwritten
13: -2.8713889221212012, //overwritten - newer used
14: -0.43095017735574337, //overwritten
15: 1.6619867843729375,
16: -2.4404387447654576,
17: 0.309646396095047,
18: 0.35497706787677674,
19: -2.402583357823728,
21: 0.23874267558768547, //overwritten - newer used
44: 4.9879005895254105,
29: 0.3727089749699747, //overwritten
22: -2.134857031543589, //overwritten - newer used
23: -0.5206127005542267, //overwritten
24: 1.0077975174345466,
25: -1.614244330989362,
26: 0.18699113445097487,
27: 0.08223132493968356,
28: -1.7154013524397482,
30: 0.23379699494680548, //overwritten
45: 3.036628305475426,
38: 0.6530008622033913, //overwritten
31: 1.713750783555073, //overwritten - newer used
32: 0.6322561544405128, //overwritten
33: -0.33435249254426874,
34: 1.0814946291145602,
35: -0.09321356394988159,
36: -0.07216057316493636,
37: 1.0598222109346631,
39: 0.22659073616501887, //overwritten
46: -1.681959162436592,
47: 0.2868782728298269, //overwritten
40: -6.15432430661483, //overwritten - newer used
41: -0.9108631418793808, //overwritten
42: 0.7099323999282806,
48: 0.2045791294080023 //overwritten
};
F[0] = input[0];
F[1] = input[1];
F[2] = input[2];
F[3] = input[3];
F[4] = F[5];
F[5] = F[6];
F[5] = F[0] * F[7];
F[5] = F[1] * F[8];
F[5] = F[2] * F[9];
F[5] = F[3] * F[10];
F[11] = 1 / (1 Math.exp(-F[5]));
F[12] = F[11] * (1 - F[11]);
F[13] = F[14];
F[14] = F[15];
F[14] = F[0] * F[16];
F[14] = F[1] * F[17];
F[14] = F[2] * F[18];
F[14] = F[3] * F[19];
F[20] = 1 / (1 Math.exp(-F[14]));
F[21] = F[20] * (1 - F[20]);
F[22] = F[23];
F[23] = F[24];
F[23] = F[0] * F[25];
F[23] = F[1] * F[26];
F[23] = F[2] * F[27];
F[23] = F[3] * F[28];
F[29] = 1 / (1 Math.exp(-F[23]));
F[30] = F[29] * (1 - F[29]);
F[31] = F[32];
F[32] = F[33];
F[32] = F[0] * F[34];
F[32] = F[1] * F[35];
F[32] = F[2] * F[36];
F[32] = F[3] * F[37];
F[38] = 1 / (1 Math.exp(-F[32]));
F[39] = F[38] * (1 - F[38]);
F[40] = F[41];
F[41] = F[42];
F[41] = F[11] * F[43];
F[41] = F[20] * F[44];
F[41] = F[29] * F[45];
F[41] = F[38] * F[46];
F[47] = 1 / (1 Math.exp(-F[41]));
F[48] = F[47] * (1 - F[47]);
var output = [];
output[0] = F[47];

I did a simple test by removing the "//overwritten - newer used" values and setting the "//overwritten" values to zero. This steps removed some unnecessary calculations and made the parameters smaller. The output was basically identical to the original model.

Can someone remove this dead code and unnecessary values?
@Jabher
Copy link
Collaborator

Jabher commented Sep 9, 2016

well, PRs are always welcome. Bad thing is that there's no tests for .standalone right now, so they are required too for a nice PR.

@MariasStory
Copy link
Author

MariasStory commented Sep 11, 2016 •
edited
Loading

OK. I'll propose an optimization function.

Maybe, someone will change the function to an object with local variables and a calculations sub-function:

function themodel(){
var F = {
0: 0,
...
};
this.calculate=function(input){
F[0] = input[0];
F[1] = input[1];
F[2] = input[2];
F[3] = input[3];
F[4] = F[5];
F[5] = F[6];
...
output[0] = F[47];
return output;
}
}

Then the object is than called like this:

myModel = new themodel(); // initialization

result = myModel.calculate([1,0,1,1]); // calculation

This way we avoid initializing the local variables every time. This should improve calculations.

@Jabher
Copy link
Collaborator

Jabher commented Sep 11, 2016

Marias, AFAIK this fn was aimed to deliver code as small as possible - otherwise it's more sane to use JSON-encoded network and Network.fromJSON, so possibly using something like higher-order function can be good, still making an actual code generator with big API is not very reasonable. Let's wait @cazala to get more info about this fn, he'll be available in early October

@MariasStory
Copy link
Author

MariasStory commented Sep 11, 2016 •
edited
Loading

So my simple solution is:

function minify(inputFunction){
    var calcRe, theMinFunction;
    var paramRe = new RegExp("((\\d ):\\s[-\\d\\.,] )", "g");
    theMinFunction = inputFunction;
    var matches, matches1, res1 = [], replFunk = [], replaceRegex, overwriteVar;
    while (matches = paramRe.exec(inputFunction)) {
        calcRe = new RegExp("(.*F\\[" matches[2] "\\].*)", "g");
        res1=[];
        while (matches1 = calcRe.exec(inputFunction)){
            res1.push(matches1[1]); // get captured group value
        }
        replaceRegex = new RegExp("F\\[" matches[2] "\\] = F\\[\\d \\];", "g");
        if (replFunk.indexOf(res1[0] "\n")>-1){
            overwriteVar = replaceRegex.exec(res1[1]);
        }else{
            overwriteVar = replaceRegex.exec(res1[0]);
        }
        if(overwriteVar){
            if(res1.length==1){
                theMinFunction = theMinFunction.replace(matches[1] "\n", "");
                theMinFunction = theMinFunction.replace(res1[0] "\n", "");
                replFunk.push(res1[0] "\n");
            }else{
                theMinFunction = theMinFunction.replace(matches[0],  matches[2] ": 0,");
            }
        }
    }
    return theMinFunction;
}

@Jabher
Copy link
Collaborator

Jabher commented Sep 11, 2016

What's the point if this solution?

@MariasStory
Copy link
Author

Given a string representation of the function it removes unused parameters and sets overwritten parameters to 0.
Just a brute force solution.
I did not test the result on correctness yet. Also, there could be another simplifications that could be made.

@Jabher
Copy link
Collaborator

Jabher commented Sep 12, 2016

We need tests first anyway 😞 Some of moving parts here are not working as expected, and I'm even not sure that this one is acting properly right now - it's not covered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Jabher @MariasStory