Before consideration this is going to require a good bit of empirical validation more than what's in the arXiv paper, especially with regard to varying dimensions and curvatures. To be open I am hesitant about the robustness of automatically switching between diagonal and dense given how small the early windows are, and how noisy those off-diagonal estimates are (not to mention the eigenvalue approximations). For something like this goes to go in it will have to be verified to work properly for diagonally-dominant problems, dense-dominated problems, and everything in between without a significant increase in cost.

Keep in mind that we're trying to minimize the sampler variants in the code base and not have "experimental" versions on dev/master. At some point we'll clean up the other samplers in there.

Before consideration this is going to require a good bit of empirical validation more than what's in the arXiv paper, especially with regard to varying dimensions and curvatures. To be open I am hesitant about the robustness of automatically switching between diagonal and dense given how small the early windows are, and how noisy those off-diagonal estimates are (not to mention the eigenvalue approximations). For something like this goes to go in it will have to be verified to work properly for diagonally-dominant problems, dense-dominated problems, and everything in between without a significant increase in cost.

Yup, hopefully we can find some models that break and learn stuff!

In general I recommend working through the validation before creating a pull request and using up testing resources. Instead a branch can be discussed on Discourse.

Because of how this proposal modifies warmup it will require studying, at the very least,

• sensitivity to initial conditions
• sensitivity to heavy tails
• sensitivity to dimension
• warmup time
• models with spatially-varying covariances

Jenkins Console Log
Blue Ocean
Commit hash: 473a28c

Jenkins Console Log
Blue Ocean
Commit hash: da00166