Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge new exogenous cache logic into our big mega-PR #23

Merged
merged 12 commits into from
Jan 14, 2025

Conversation

GondekNP
Copy link
Contributor

This new logic is spurred on by @mfisher87 's work on #20 (within branch #22) - I refactored out the logic that creates the elevation layer cache (called here the 'exog_cache`), and created some abstractions that should allow us to add more and more exogenous data caches - here elevation, but perhaps soil lithography, average windspeed, initial occupancy conditions, etc.

Ordinarily we would merge this right to dev - but since my mega-merge encompasses so many changes, I will merge here to address conflicts within #17, merging into dev only when all of the latest changes work together.

mfisher87 and others added 12 commits January 11, 2025 12:33
* Extract the pystac client to a cached property; it'll only be
  initialized when it's accessed, and subsequent accesses return the
  same object as the first access
* Extract cache existence check to own function for readability

However I still receive an error trying to start the app:

```
  File "/home/robatt/Projects/dse/mesa_abm_poc/.pixi/envs/default/lib/python3.11/site-packages/stackstac/rio_reader.py", line 352, in _open
    vrt = WarpedVRT(
          ^^^^^^^^^^
  File "rasterio/_warp.pyx", line 1045, in rasterio._warp.WarpedVRTReaderBase.__init__
  File "rasterio/crs.pyx", line 786, in rasterio.crs.CRS.from_user_input
  File "rasterio/crs.pyx", line 596, in rasterio.crs.CRS.from_epsg
rasterio.errors.CRSError: The EPSG code is unknown. PROJ: internal_proj_create_from_database: /home/robatt/Projects/dse/mesa_abm_poc/.pixi/envs/default/share/proj/proj.db contains DATABASE.LAYOUT.VERSION.MINOR = 2 whereas a number >= 3 is expected. It comes from another PROJ installation.
```
It would be great if we could get the needed versions of mesa-geo up on
conda-forge! The source of the problem seems to be mixing pip and conda
deps. Rasterio >=1.4b1 is required by mesa-geo 0.9:

https://github.com/projectmesa/mesa-geo/blob/bb1cfca2d799d4d5012d45578cc99dbb1c61af87/pyproject.toml#L46

And we specified a conda dependency of rasterio 1.3.*.
Use /tmp/ as cache base directory because we can expect it to exist on
the system and be writeable as non-root.

However, now we're stuck in a loop of writing to the same file over and
over!
…ion model to get RasterLayer.to_file kicks because we don' have the cache, but that's the whole point. Really though we just need to use rio to export whatever this cache is meant to look like (or I guess fail silently or show a warning in Vegetation - since it won't be a problem until we step
…. solved the Vegetation instantiation problem by creating an method, which likely will have other knock on benefits down the line:
@GondekNP
Copy link
Contributor Author

@lucialayr , @amandersonyou and @mayazomer - this is a good example of the sort of conflicts we expect to see when merging super divergent branches. I figured I'd describe it here to help clarify since it was a bit tangential / rambly this AM!

My mega-PR feature branch (feat/paint_some_trees, see #17 ) was pulled from dev a while back. I made a lot of changes across a lot of areas (frontend visualizations, interactivity, cache changes, parallel processing, batch jobs, etc). Along the way I refactored things that seemed like they would cause problems down the line.

When @mfisher87 jumped in a couple days ago, he didn't know that feat/paint_some_trees was essentially our dev branch at that point (in the sense that it had the latest changes, was relatively stable), so he created his new branch off of dev. Thus, when he started working on this stuff, his branch was already out of date and he was making changes that conflict with my feature branch changes (in some cases, addressing the same problems I had tried to address through my refactors).

To address the mismatch, I created this PR to merge the new cache logic into my mega-PR, with the assumption that I can address the merge conflicts and get it all working before merging to dev. But all this said, worth noting that this is not particularly good practice - I would have been better off creating new branches for each new feature, or ideally merged back to dev periodically.

@GondekNP GondekNP merged commit 992c738 into feat/paint_some_trees Jan 14, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants