-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TF python vs TFJS models generating significantly different results #8025
Comments
Hi, @danielgoldelman Thank you for bringing this issue to our attention and I was trying to replicate the same issue from my end if possible, could you please help me with code-snippet/code-example which you're using to display above results for |
Hello @gaikwadrahul8, here is a link to a repo that can display these results. The results shared in the screenshots above can be generated via the |
I compared the CPU execution vs WebGL (you can reproduce this by running You could try converting the model without grappler graph optimization. This shouldn't be the problem, but it's possible something went wrong during optimization. In any case, I would probably do this before trying to compare intermediate tensors between the TF version and TFJS version. Unfortunately, we don't have a flag for this yet, but if you comment out these lines and replace them with |
@mattsoulanille Sorry, but I would like some help understanding where to place the BUILD file in the tfjs filestructure. I am getting this error:
Could you help me get on track? |
Hello, I have been working on the same task as @danielgoldelman (same model, setup, etc.), and I seem to be running into a similar issue. I've tried @mattsoulanille 's suggestion to disable optimization, but that didn't seem to have any impact on the performance.
I'm not really sure where to go from here, does anybody have any suggestions? |
System information
Describe the current behavior
This model sees a significant drop in performance between a Tensorflow in Python TFBertForSequenceClassification model and a related Tensorflow in JS Graph Model, created using tensorflow_converter.
Describe the expected behavior
The expected behavior would be that the TF python model and the TF js model would produce similar or identical results. Please let us know if the conversion was done incorrectly.
Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/CodePen/any notebook.
Converter scripts:
Model location:
https://github.com/danielgoldelman/modelrepo/tree/main/MultitaskModel
Results for the Python model:
Results for the JS model (tested both in node and in the browser)
The text was updated successfully, but these errors were encountered: