-
Notifications
You must be signed in to change notification settings - Fork 981
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some unimplemented unary expressions? #1697
Comments
Isn't reciprocal square root supported by eltwise pow algorithm with alpha =1 and beta = -1/2? Will adding sin/cos algorithms to eltwise primitive be enough to cover embeddings case? |
Thanks |
1 for adding cos/sin elementwise! I would also greatly appreciate it! |
@WilliamTambellini @lucasalavapena Thank you for the request, @WilliamTambellini I see that positional encoding does not depend on the actual data, is it pre-computed prior to inference/training and remains constant during the process? |
@igorsafo I am still relatively new to oneDNN, so hopefully what I am saying makes sense 😄 . I was more thinking of having it as an op for dnnl::graph, so you can get some benefits you normally can get with the graph extension (memory layout and use of graph compiler?). Aren't some of the existing element wise functions part of the standard math library too? |
Hi
Here are some unary ops currently not implemented in 1dnn (or I missed them):
https://stackoverflow.com/questions/10732027/fast-sigmoid-algorithm
https://en.wikipedia.org/wiki/File:Gjl-t(x).svg
Best
The text was updated successfully, but these errors were encountered: