Workflow to evaluate and tune a deep learning model using the uncertainty model goodness metric. https://doi.org/10.1016/j.petrol.2021.108975
Deep learning models focus on prediction accuracy and minimizing prediction error. Yet, for spatial models, accurate predictions along with accurate and precise uncertainty models are critical.
Due to limited data veracity, data sparsity, and feature heterogeneity in the spatial setting, uncertainty is significant; therefore, predicting a single estimate must be replaced with a prediction of the uncertainty distribution.
We propose a novel and easy to use objective function to summarize uncertainty model performance. The proposed function tunes deep learning models for optimum uncertainty accuracy and precision.
For this notebook example we have a problem that consists on predicting permeability from porosity and acoustic impedance data. We have selected this problem because we are primarily interested in capturing the uncertainty related to predictions of permeability based on existing data.
This problem can be expanded to any prediction problem.
- Executive summary
- Objective
- Introduction
- Library importing
- Functions
- Dataset exploration
- Dataset normalization and split
- Deterministic prediction
- Model goodness evaluation
- Uncertainty model generation
- Model goodness evaluation
- Hyperparameter space exploration
- Summary of the training
- Conclusions and observations
NOTE: The hyperparameter space exploration section might be too long to run, we have included the results of the grid search in Search-Space.pkl. Feel free to skip this section and load the results in Summary of the training subsection