Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BF is NA #81

Closed
mattansb opened this issue Jul 21, 2024 · 2 comments
Closed

BF is NA #81

mattansb opened this issue Jul 21, 2024 · 2 comments

Comments

@mattansb
Copy link

Revisiting #68 & easystats/bayestestR#627, we're still not getting BFs.

(I'm comparing the same model to itself, so log(BF) should be 0, but getting NA):

library(blavaan)

data("PoliticalDemocracy", package = "lavaan")

model <- "
  # latent variable definitions
  dem60 =~ y1   a*y2
  dem65 =~ y5   a*y6

  # regressions
  dem65 ~ dem60

  # residual correlations
  y1 ~~ y5
"

bfit <- blavaan::bsem(model,
                      data = PoliticalDemocracy,
                      n.chains = 1, burnin = 50, sample = 100
)
#> Warning messages:
#> 1: The largest R-hat is NA, indicating chains have not mixed.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#r-hat 
#> 2: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#bulk-ess 
#> 3: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#tail-ess 
#> 4: blavaan WARNING: As specified, the theta covariance matrix is neither diagonal nor unrestricted, so the actual prior might differ from the stated prior. See
#>  https://arxiv.org/abs/2301.08667 

blavaan::blavCompare(bfit, bfit)
#> 
#> WAIC estimates: 
#>  object1:  1436.048 
#>  object2:  1436.048 
#> 
#>  ELPD difference & SE: 
#>     0.000    0.000 
#> 
#> LOO estimates: 
#>  object1:  1435.986 
#>  object2:  1435.986 
#> 
#>  ELPD difference & SE: 
#>     0.000    0.000 
#> 
#> Laplace approximation to the log-Bayes factor
#> (experimental; positive values favor object1):       NA
@ecmerkle
Copy link
Owner

Thanks for following up, I think this is a different issue than the one described in #68.

I think the issue now is related to the fourth warning message about the theta covariance matrix. The y1 ~~ y5 residual covariance leads to a covariance matrix with one free off-diagonal element and others fixed to 0. In such a case, the prior for the covariance matrix can be influenced by the positive definite constraint, and blavaan cannot necessarily evaluate the correct prior (which throws off the BF calculation) so an NA is reported. For more detail, see the "positive definite constraints" section here.

In your specific case, I think it is a false alarm because there can be one free covariance without worrying about positive definiteness. I can improve the check here. And if you remove the residual covariance, you should get the BF with the current version of blavaan.

@ecmerkle
Copy link
Owner

You should now be able to obtain the Bayes factor for your model. If you were to add a second residual covariance to your model involving either y1 or y5, then the Bayes factor would again become NA due to the positive definite issue from my first reply. And the Bayes factor is also currently turned off for models that have ordinal observed variables. In general, I am trying to be careful about only returning the Bayes factor when it is clear that we can reliably evaluate the priors in the model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants