Hi, nice work for VFI.
According to the paper, to avoid the decoder using features only from the “faster” training encoder, you train the encoders for motion spline estimator, fusion encoders, and the entire network sequentially. However, how about training all modules together with different learning rates? Set a smaller learning rate for the event-encoder and a normal one for the image-based encoder.
Hope you can help me. Thx a lot.