@dukecat0 @uranusjr I've got a preliminary POC of this feature which I'm quite happy with, but before I sink any more time into this, I'm hoping to get some feedback and make sure I'm on the right track.

Is this something you would like me to keep working on? do you have any notes on my implementation?

Thanks for working on this! I personally find this feature very useful. 👍 Probably it will be better if there's an environment variable that allows users to disable this behavior.

Looks like a helpful feature, thanks!

Probably it will be better if there's an environment variable that allows users to disable this behavior.

I agree with that – or even a flag to toggle the behaviour.

Good stuff!

I would argue that this feature should be opt-in, rather than opt-out. I would be very confused if I accidentally mistype a python version and pipx just starts downloading it unprompted.

opt-in, rather than opt-out

I agree – that's why I suggest a flag (with negative default value).

Excellent idea! I’m more than happy to implement that. Not sure what to call the flag though… something succinct?

ideally also an environment variable to flip the default

maybe something like —fetch-as-needed? It’s a bit wordy, but I can’t think of anything more succinct that gets the idea across

ideally also an environment variable to flip the default

1

maybe something like —fetch-as-needed? It’s a bit wordy, but I can’t think of anything more succinct that gets the idea across

fetch-missing-python? fetch-as-needed could also refer to the package to install/run itself.

Please add a news fragment, too. And we'll need tests.

Hi All, this feature is now, to the best of my ability, complete.

I've added tests, a news entry, and adopted / implemented just about every change that's been requested.

The tests are all now passing on all platforms and python versions.

I've spent a not insignificant amount of time and energy making this feature implementation as good as it can be in the hopes that it will be accepted and merged. I am running out of time and energy to devote to this now.

I think it's ready to review and/or merge. I'm going to step away from it now. If there are any additional changes you would like to make, feel free to make them before or after merging the feature.

Thank you

Would be nice if there was a command to clean up the python versions, e.g. when a downloaded python version does not have a installed app that uses it anymore. We can treat that as a separate feature request though IMO.

Would be nice if there was a command to clean up the python versions, e.g. when a downloaded python version does not have a installed app that uses it anymore. We can treat that as a separate feature request though IMO.

Also a command to update one or all standalone versions (ie someone uses --python=3.9 --fetch-missing-python and wind up with v3.9.10 installed to PIPX_HOME/py/3.9. several months later, they come back and install another package with --python=3.9 --fetch-missing-python, and that package gets installed with python 3.9.10, even though v3.9.18 is now available)

That's a hard problem to solve automatically, so i think a manual process / command would be the way to go

This latest error in the windows unit test is... interesting, to say the least.

is PIPX_HOME cached in CI between test runs?

I'm not quite sure how else to explain why PIPX_HOME/py/3.11 already exists on the windows CI server...

Probably we should use pytest's tmp_path fixture?

Probably we should use pytest's tmp_path fixture?

to override the PIPX_STANDALONE_PYTHON_CACHEDIR constant?

I suppose so, but that somehow feels wrong. the unit test should as closely as possible reflect what will actually happen on a user's machine

ordinarily, this error we're seeing should be impossible. if standalone python 3.11's already been downloaded, the function should return early without trying to redownload it. this error seems to indicate that a previous run failed and produced an empy PIPX_HOME/py/3.11 folder...

Well, apart from that rate limit error on the windows test, I think we're now good to go.

Are there any additional changes you need, or can this be merged?

I'd like to raise awareness for #1243 (comment) that should still be implemented IMHO, as it tests basic functionality.

As for the rate limit, I see that as a possible issue, as these tests will be run many times. I propose skipping this test if standalone_python.py isn't modified (don't know how to achieve that, yet), or scheduling it.

I'd like to raise awareness for #1243 (comment) that should still be implemented IMHO, as it tests basic functionality.

As for the rate limit, I see that as a possible issue, as these tests will be run many times. I propose skipping this test if standalone_python.py isn't modified (don't know how to achieve that, yet), or scheduling it.

The first suggestion from that comment (ensuring the target python isn't found on dev's machines) has already been implemented (by mocking out which in both the windows test and the unix test.

The second comment, adding a python_version variable to these two unit tests, doesn't make sense to me. none of the other tests in that test suite define such a variable, and i don't understand why we would want to add such an inconsistency or why the lack of this variable is a blocker towards getting this change merged?

I'd like to raise awareness for #1243 (comment) that should still be implemented IMHO, as it tests basic functionality.

As for the rate limit, I see that as a possible issue, as these tests will be run many times. I propose skipping this test if standalone_python.py isn't modified (don't know how to achieve that, yet), or scheduling it.

regarding the rate limit, that is definitely a concern. I'm going to try and write a fixture to mock out the API call specifically, that should take care of that problem

stand by

The first suggestion from that comment (ensuring the target python isn't found on dev's machines) has already been implemented (by mocking out which in both the windows test and the unix test.

However, implementing it would ensure the exception is raised correctly, which isn't entirely useless.

However, implementing it would ensure the exception is raised correctly, which isn't entirely useless.

oh, i see, you're suggesting i write a test for the failure mode. gotcha, will do

I'd like to raise awareness for #1243 (comment) that should still be implemented IMHO, as it tests basic functionality.

hi @chrysle, I've implemented test cases for versions of python which aren't available in the python-build-standalone project

in the future, we might want to expand the logic of the standalone_python module to collect data from multiple releases of the python-build-standalone project and merge it all together so that we can support fetching more python versions than are available in the current release (ie. 20231002 was the last release to include a version of python 3.8), but that's a massive increase in complexity for very little gain, and far outside the scope of this PR

Thank you for all the work you have invested into this contribution!

Would be nice if there was a command to clean up the python versions, e.g. when a downloaded python version does not have a installed app that uses it anymore. We can treat that as a separate feature request though IMO.

Also a command to update one or all standalone versions

Additional concerns, as a pipx clean (?) command, shall be addressed in follow-ups.

Thank you for bearing with us through the review process @alextremblay. Great contribution!

We also moved logging the exceptions from the interpreter resolution outside of the scope of this issue, so that also needs a follow-up.