You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
value residual learning (lucidrains#312)
* cite
* add value residual learning
* oops
* slip in value residual learning for pairformer stack
* also cite Nguyen, whose initial paper led here
a new paper claims there is a free lunch by setting model weights to …
…ema weights every epoch. allow researchers to experiment with this, conveniently already available in EMA-pytorch due to hare and tortoise paper