Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept path-like HREF arguments in I/O methods #728

Merged
merged 8 commits into from
Feb 4, 2022

Conversation

duckontheweb
Copy link
Contributor

@duckontheweb duckontheweb commented Jan 24, 2022

Related Issue(s):

Description:

  • Implements the os.PathLike interface for pystac.Link objects. This implementation always returns a str representation (never bytes). See PEP 519 for details on the file system path protocol.

  • Updates the package-level pystac.read_file & pystac.write_file methods to accept os.PathLike objects for their HREF arguments in addition to strings.

  • Updates the StacIO I/O methods to accept os.PathLike objects for their HREF arguments.

    This represents a breaking change in terms of typing since users who are creating custom StacIO classes will need to update the type hints in these methods as well. However, this is not a breaking change in terms of runtime behavior, so I am inclined to include it in a minor version release.

@TomAugspurger I'd like to get your feedback on the overall approach, the idea of implementing os.PathLike for Links, and whether we should go farther and accept os.PathLike objects in more methods that work with HREFs (e.g. Catalog.save, etc.).

PR Checklist:

  • Code is formatted (run pre-commit run --all-files)
  • Tests pass (run scripts/test)
  • Documentation has been updated to reflect changes, if applicable
  • This PR maintains or improves overall codebase code coverage.
  • Changes are added to the CHANGELOG. See the docs for information about adding to the changelog.

@duckontheweb duckontheweb requested a review from lossyrob January 24, 2022 17:11
@duckontheweb duckontheweb changed the title Add/469 pathlike support Accept path-like HREF arguments in I/O methods Jan 24, 2022
@duckontheweb duckontheweb requested review from gadomski and m-mohr and removed request for m-mohr January 24, 2022 17:13
Copy link
Member

@gadomski gadomski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues from me.

Only minor comment is that Union[str, "os.PathLike[AnyStr]"] ends up everywhere and is a lot to parse for a casual reader, is it worth type aliasing that to Href or something?

@duckontheweb
Copy link
Contributor Author

No issues from me.

Only minor comment is that Union[str, "os.PathLike[AnyStr]"] ends up everywhere and is a lot to parse for a casual reader, is it worth type aliasing that to Href or something?

Yeah, I agree that's pretty hard to interpret. Maybe we should have a types.py module where we can keep some PySTAC-/STAC-specific types that we use throughout the library.

@TomAugspurger
Copy link
Collaborator

Looks good to me to.

I vaguely recall a discussion around whether __fspath__ was intended to represent a "proper" filesystem, or whether it could be used for potentially remote urls like we're doing here for Links. But IMO it's ok. If you do something like open(item.links[0]), where the link happens to be remote, you're still going to get an error.

duckontheweb added a commit to duckontheweb/pystac that referenced this pull request Jan 28, 2022
duckontheweb added a commit to duckontheweb/pystac that referenced this pull request Jan 28, 2022
@duckontheweb duckontheweb force-pushed the add/469-pathlike-support branch from b510828 to cb60f2e Compare January 28, 2022 11:49
@duckontheweb duckontheweb added this to the 1.4.0 milestone Jan 28, 2022
@duckontheweb
Copy link
Contributor Author

No issues from me.
Only minor comment is that Union[str, "os.PathLike[AnyStr]"] ends up everywhere and is a lot to parse for a casual reader, is it worth type aliasing that to Href or something?

Yeah, I agree that's pretty hard to interpret. Maybe we should have a types.py module where we can keep some PySTAC-/STAC-specific types that we use throughout the library.

I created a pystac.types module that currently has the HREF type alias (from this PR) and the ItemLike type alias that used to be in pystac.item_collection. I think we could probably do some more work on creating more readable type aliases and moving them into pystac.types, but that seems better left for a separate PR.

@l0b0 Do you have any thoughts on the approach of storing type aliases in a single module?

duckontheweb added a commit to duckontheweb/pystac that referenced this pull request Jan 31, 2022
@duckontheweb duckontheweb force-pushed the add/469-pathlike-support branch from 400d1c6 to b3ac17d Compare January 31, 2022 17:18
duckontheweb added a commit to duckontheweb/pystac that referenced this pull request Jan 31, 2022
@duckontheweb duckontheweb force-pushed the add/469-pathlike-support branch from b3ac17d to 39e5802 Compare January 31, 2022 18:32
@l0b0
Copy link
Contributor

l0b0 commented Jan 31, 2022

@l0b0 Do you have any thoughts on the approach of storing type aliases in a single module?

I'd treat that the same as utility functions, and refer to my answer to Is putting general-use functions in a "helpers" file an anti-pattern or code smell? 😁

@duckontheweb
Copy link
Contributor Author

@l0b0 Do you have any thoughts on the approach of storing type aliases in a single module?

I'd treat that the same as utility functions, and refer to my answer to Is putting general-use functions in a "helpers" file an anti-pattern or code smell? 😁

Thanks. It does feel like overkill to create a separate types sub-module for just 2 type aliases, especially since 1 of them is only used by the ItemCollection class. I'm going to move ItemLike back where it came from and put HREF into the link sub-module since that seems like the most logical place for it.

duckontheweb added a commit to duckontheweb/pystac that referenced this pull request Feb 1, 2022
@duckontheweb duckontheweb force-pushed the add/469-pathlike-support branch from 39e5802 to dcdadbc Compare February 1, 2022 00:58
@codecov-commenter
Copy link

Codecov Report

Merging #728 (3e9da85) into main (4b30217) will increase coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #728       /-   ##
==========================================
  Coverage   94.33%   94.39%    0.05%     
==========================================
  Files          77       77              
  Lines       11285    11292        7     
  Branches     1347     1343       -4     
==========================================
  Hits        10646    10659       13     
  Misses        459      455       -4     
  Partials      180      178       -2     
Impacted Files Coverage Δ
pystac/__init__.py 100.00% <ø> (ø)
pystac/link.py 91.16% <100.00%> ( 0.30%) ⬆️
pystac/stac_io.py 87.38% <100.00%> ( 4.48%) ⬆️
tests/test_link.py 100.00% <100.00%> (ø)
tests/utils/stac_io_mock.py 85.18% <100.00%> ( 1.18%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b30217...3e9da85. Read the comment docs.

@duckontheweb duckontheweb force-pushed the add/469-pathlike-support branch from 3e9da85 to 48f77ba Compare February 4, 2022 11:05
@duckontheweb duckontheweb merged commit dfc1cc2 into stac-utils:main Feb 4, 2022
@duckontheweb duckontheweb deleted the add/469-pathlike-support branch February 4, 2022 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants