Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different sampling strategy for Sobol' analysis #626

Open
Rishitha-28 opened this issue Jul 9, 2024 · 10 comments
Open

Different sampling strategy for Sobol' analysis #626

Rishitha-28 opened this issue Jul 9, 2024 · 10 comments

Comments

@Rishitha-28
Copy link

Hello,

I have a question regarding the use of SALib for Sobol analysis and wasn't sure where to post it, so I'm posting it here.

Can SALib's sobol.analyze function calculate Sobol indices for a non-uniform sample set? Specifically, is it possible to use other sampling techniques instead of Saltelli or Sobol methods?

Additionally, I would like to know if SALib can compute Sobol indices for a subset of the sample set. For instance, can we apply a condition or threshold to the model output and then calculate the indices for this subset?

Thank you in advance for your assistance!

@tupui
Copy link
Member

tupui commented Jul 9, 2024

Hi 👋

Thanks for the question.

When you define your problem, you can define any distribution you want for the parameters. Then it will work just fine (and is also correct from a statistical standpoint.)

You can use your own sample to pass to the analysis part. But you must use the correct matrix format with A, B, AB, BA. So I would not recommend doing it. Missing a single point would make the analysis wrong.

@tupui tupui added the question label Jul 9, 2024
@tupui
Copy link
Member

tupui commented Jul 9, 2024

We had plans for deprecating the Saltelli method.

What we can do is to go forward but also propose a new generic API that would basically be the new Sobol' but with a sampling method parameter.

@Rishitha-28
Copy link
Author

Hi
Thanks for the response.
What do you mean by 'use the correct matrix format with A, B, AB, BA'?

@tupui
Copy link
Member

tupui commented Jul 9, 2024

I mean that the analysis function expects to be constructed in a very specific way. It's not just a list of samples. First it takes two independent samples: A, B. Then it makes a matrix AB (and BA if second order) by mixing A and B. And it then interleaves the matrices together. So when the analysis gets the input sample, it decomposes it into these matrices and if one point is out of place, it will not work and results will be random.

@Rishitha-28
Copy link
Author

In the case of only two parameters (i.e., no second order), isn't it enough if Y.size is divisible by (D 2) [which is 4 in our case]?

image
Doesn't this take care of the matrices?

@tupui
Copy link
Member

tupui commented Jul 9, 2024

Sorry I don't understand your question.

In case of 2 parameters, you would only have A, B and AB. But it's not just about being divisible. As you can see in the code, the input sample Y is decomposed into A, B, AB in a specific order. A, AB, B, A, AB, B, etc. And Y was constructed by scrambling the inputs that way.

You cannot take an existing random sample and split it in 3 to get A, B and AB. You must start with 2 matrices in your parameter space: A, B. Then AB is (for 2 parameters–don't quote me on the order, just to show it's not that straightforward): [A[:, 0], B[:, 1], A[:, 1], B[:, 0]]. Then you use this sample with your model to get f_A, f_B, f_AB. And the concatenation of these 3 gives you Y.

@Rishitha-28
Copy link
Author

Correct me if I am wrong, but I see that Y is the model output array. And the code splits the output array into A, B, AB matrices.
Y is evaluated initially over the total sample space, and later, inside the function sobol.analyze, the matrices are formed.

Ref:

image

@tupui
Copy link
Member

tupui commented Jul 10, 2024

The matrices are not formed in sobol.analyze but "retrieved".

Y is the model output array and X is the sample leading to Y: f(X) -> Y

saltelli.sample (or sobol.sample) is creating X in a very specific way as I explained. saltelli.sample makes at least three matrices A, B, AB and concatenate them together to give X.

There is of course a direct relationship between samples in X and model outputs in Y.

All that to say that you cannot take an arbitrary sample X (from a random sampling for instance) and then get Y and pass it to sobol.analyze. This is incorrect. If you want to construct yourself X, you must follow the rules I outlined above (A, B, AB with the correct column re-ordering to get AB.)

@Rishitha-28
Copy link
Author

Thank you for your time! Appreciate it!

@tupui tupui reopened this Jul 10, 2024
@tupui
Copy link
Member

tupui commented Jul 10, 2024

Of course, glad I could help.

I am re-opening since there is a possible enhancement with the addition of a method parameter in saltelli.sample (or the function that would supersede it.)

@tupui tupui changed the title Can SALib Handle Non-Uniform Samples and Conditional Sobol Analysis? Different sampling strategy for Sobol' analysis Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants