Is my understanding of the approaches to uncertainty modeling correct?

Yes

Which approach do you think is most appropriate?

This doesn't have an answer. It depends on your problem. Choose your metric to measure how "good" the policy is, and then pick the model that achieves the best policy for your problem. Note that how you model the uncertainty is a modeling choice. It's not the real word, but that doesn't matter.

As for the 5th approach mentioned above, does SDDP.jl support such problem formulation?

Sure, you can make a policy graph with the Markov transition matrices:

[ [1/3 1/3 1/3], [1 0 0; 0 1 0; 0 0 1], ..., [1 0 0; 0 1 0; 0 0 1]

but this means that when we see the realization in stage 1, we know the realization of the uncertainty in all future stages.

Again, this is a modeling choice. Whether it is suitable is up to you.

Should we try to avoid using the first bold approach? Or in another word, how much return can we get from modeling the uncertainty?

I don't have an answer for you because this is problem-dependent.

if we model the uncertainty using the first approach and train the SDDP model as long as possible, it should achieve the same effect as when we model the uncertainty using other better methods. Is this true?

No!!!

Training an "optimal" policy to a poor choice of model will probably perform poorer in practice than a model with a better representation of the world trained for fewer iterations.

Thank you very much, @odow ! I'm very excited every time I see your reply and I always learn a lot!

Now I feel I have a deeper understanding on SDDP.jl. But I still have one question. If we take the 5th approach to train a policy based on the scenarios by introducing a Markovian policy graph, then, how should we evaluate a decision rule when we observe a new uncertainty realization in practice? (Since we don't know which node now we have arrived at.)

how should we evaluate a decision rule when we observe a new uncertainty realization in practice

Now you see the problem with this approach 😄

It's up to you. You might want to pick the scenario that is most similar.

Or you might realize that this isn't a very appropriate model for sequential decision making under uncertainty.

Thank you @odow ! Thank you very much! 😀🤝🤝

But I think, if I understand correctly, perhaps this problem can be circumvented by pre-specifying some structured decision rule functions for each stage (rather than node), is that right? (Although this may be a very rough approximation).

There are other ways to find (sub-optimal) policies for sequential decision problems, such as (linear) decision rules.

SDDP.jl does not support custom decision rules, although it has come up in discussion: #696

I got it!