Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discrepancy between 3ML and Xspec residuals of spectral fitting #572

Open
rahulc157 opened this issue Sep 1, 2022 · 13 comments
Open

Discrepancy between 3ML and Xspec residuals of spectral fitting #572

rahulc157 opened this issue Sep 1, 2022 · 13 comments

Comments

@rahulc157
Copy link

Hi,

I am doing the joint spectral fitting of Fermi GBM and ASIM data using threeML and Xspec. I am getting consistent spectral parameters from both (threeML and Xspec) the tools. However, I am facing issues with the residual of the model fit. 3ML is showing a large residual for the ASIM data (although the data is good fitting), however, Xspec is showing a very low residual for the ASIM data. I am using display_spectrum_model_counts() of threeML to show the model fit and residual. Is there any other way to plot residuals in 3ML of the model fit, similar to delchi command of Xspec? What could be the possible reasons for this discrepancy between 3ML and Xspec residuals?

@grburgess
Copy link
Collaborator

I am not sure how XSPEC computes the residuals, but it my be a naive chi2. We use the proper residuals for the likelihood of the data and this may be the result of the discrepancy.

@rahulc157
Copy link
Author

Many thanks for the prompt response @grburgess.

But I am surprised, that the joint spectral model fitting is well fitting the ASIM data but the residual plot of 3ML is very large (close to 10 sigma). What could be the possible cause of this? Count/plugin/etc issues in the spectrum or some limitation of the 3ML itself?

@grburgess
Copy link
Collaborator

Can you provide comparison plots to demonstrate the issue?

@rahulc157
Copy link
Author

rahulc157 commented Sep 1, 2022

TRS_band_bin2_GBM_ASIM.pdf using 3ML
Xspec

Okay. Please see the attachment. Pdf file is obtained using 3ML and png file is obtained using Xspec.

@grburgess
Copy link
Collaborator

There does seem to be an issue with how the residuals are plotted. Can you provide the code used to generate the plot?

@rahulc157
Copy link
Author

Okay, I have sent you a tar folder with code and data files to your email ([email protected]). Many thanks.

@grburgess
Copy link
Collaborator

The issue was in the scale factor of the ASIM background. How is this derived?

@grburgess
Copy link
Collaborator

@rahulc157 any update?

@rahulc157
Copy link
Author

rahulc157 commented Sep 7, 2022

Sorry for the delayed reply. Below is the response to your query:

The BACKSCAL keyword in the ASIM background file (the scale factor) is set to 1, as it should be according to the explanation provided here: https://heasarc.gsfc.nasa.gov/docs/asca/abc_backscal.html. It is set to 1 also in Fermi data, so I don’t understand what is the problem in 3ML while reading the background file. I see that in ASIM the keyword is written as 1. While in Fermi files, it is 1.0 but it should not matter, maybe the 3ML parser does not read properly the ASIM value? While Xspec is correct reading the same background file.

I have another query...I tried to fit the joint Fermi and ASIM data without the background file of ASIM data and got better residuals in the 3ML. But, I think it is not correct to model without a background file. Please suggest further to resolve the issue. I will be thankful to you.

@grburgess
Copy link
Collaborator

The background has an exposure of:

TSTART  =            106.50000 / DOY observation start time                     
TSTOP   =            106.50000 / DOY observation stop time                      
EXPOSURE=           0.42000000 / Integration time in seconds fro the PHA data   
AREASCAL=                   1. /  Area scaling factor, no field AREASCAL        
BACKSCAL=                   1. /  Background scale factor, no field BACKSCAL 

which does not line up with the start and stop time.

This is much shorter than the source exposure time:

TSTART  =            106.50000 / DOY observation start time                     
TSTOP   =            108.20000 / DOY observation stop time                      
EXPOSURE=           0.70000000 / Integration time in seconds fro the PHA data   

and moreover the background and source times overlap. I"m not familiar with the details of the data reduction, but the significance calculation which also does the residuals is derived for a background scaling that is larger than one (this also makes since when using a profile likelihood otherwise there is not enough data to perform a stable profile).

The issue is not that the file is not being read correctly, it is that the residuals in XSPEC are not Poisson residuals, they are simply an L1 norm from the data, which is incorrect for the data you have.

@rahulc157
Copy link
Author

Hi,

I agree that the TSTART and TSTOP keyword are not consistent with the EXPOSURE keyword. This comes from the fact that ASIM is not designed to detect GRB, and the pipeline for producing the fit files is not bullet-proof. However, as far as I know, XSPEC uses the EXPOSURE keywords only, this is why we did not bother fixing the TSTART and TSTOP keywords.

The background is, in fact, only 0.42 sec exposure, because this is all the data that we have before the trigger. We do not have all photon data as for GBM. But, of course, the background time interval does not overlap with the GRB time interval.

Sorry, but I don’t understand when you write: ‘they are simply an L1 norm from the data, which is incorrect for the data you have.’
Can you please explain me better?

What is plotted above was obtained with the XSPEC command ‘plot data delchi’, which, according to the XSPEC manual, is supposed to provide residuals between observations and the best fit model in terms of sigmas.

@grburgess
Copy link
Collaborator

The formula on the y axis of the xspec plot is incorrect for poisson data.

@omodei
Copy link
Collaborator

omodei commented Sep 27, 2022

Can we close this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants