1. Write the program in a reversible style.
2. Reverse the program.
3. Insert gradient codes.
Yeah, I think I need to read the paper…
From glance of it, seems it sugests you don’t need to keep a stack of previous values, therefore, reduces the memory usage (no need of auxiliary memory to store the forward pass computed values). Seems plausible until some computations cannot be reversed?
Thats also my concern. What are actual use cases of this technology? I guess only computation where the amount of information stays constant are allowed, which would discard many use case. You cant create something from nothing.
May be a single or multi body physics simulation?
This repository definitely needs a Tenet reference of some kind. Code inversion might be cooler than time inversion or will lead one to the other? :D Very cool project!