-
-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shared redesign of embedded requirements feature #44
Comments
Option 1: Python ObjectsThe current implementation achieves the goals by soliciting special variables from the user. This feature was inspired by the setuptools technique from 15 years ago, in which requirements were embedded into the In this approach, the user specifies their requirements in the script using Python syntax and special variable names exposing static values. Example:
The tool parses the Python and extracts the requirement specifications from the declared variables and incorporates those during the install phase. Pros
Cons
Variant 1: single variableTo limit polluting the namespace, the tool could solicit all possible values through a single variable, something like:
This approach might prove easier to extend. It also encapsulates the values into a namespace, avoiding proliferation of dunder-names. It is probably more tedious to use, though. |
Option 2: Requirements.txt in commentsThis approach re-uses the requirements.txt syntax but as a multi-line comment in the script. Example:
Some features of Pros
Cons
Variant 1: Install using
|
Option 3: embedded command-line argumentsThe tool could solicit arbitrary arguments to the
Pros
Cons
Variant 1: Shebang inlineInstead of including the command-line options in the Python portion of the file, including it in the Unix shebang line.
Pros
Cons
Variant 2: Constrain the allowed argumentsIt may be possible/desirable to constrain the allowed arguments select ones, but that of course requires maintaining that constraint. Variant 3: Inline variableMuch like in Option 1, the command-line options could be solicited through a single variable:
This approach would simplify the parsing of arguments, but has many of the same cons as option 1 but doesn't provide a structured syntax for soliciting the values. It compromises both the intuition of the user (who's thinking "command line parameters") and the tool execution. |
I'm not a fan of the magic variable syntax, so I guess I'll toss my own proposal in. Option 4: Restricted Requirements.txt in commentsThis approach is as Option 2, but only allows PEP 440 specifiers. In addition, a single-line syntax is provided for brevity. The Pros
Cons
ExampleSingle line example: #! /usr/bin/env python3
# Generate visualization
# Requires: numpy pandas~=1.0 seaborn~=0.10 Multiline example: #! /usr/bin/env python3
# Generate visualization
# Requires:
# numpy~=1.18
# pandas~=1.0
# seaborn~=0.10 Variant TangentSomething that I'd like is the ability to pretty-format. This is probably impractical, but here's what that'd look like: #! /usr/bin/env python3
# Generate visualization
# Requires:
# - numpy~=1.18
# - pandas~=1.0
# Look, a comment!
# - seaborn~=0.10
# - matplotlib>=3.2.1 # See GH#1234 Overall ratings: Option 1: Anyway, these are my 2¢ for Proposals 1-3:
And another comment for thought: what about the Notebook use case? This would be very useful for notebook distribution. |
My preference is for option 4. I don't see any value in supporting the "full" requirements syntax (which is arcane, and not standardised). Being able to specify a list of projects the script depends on, possibly with a version restriction is IMO sufficient. I'm not too concerned about the precise syntax. Multi-line, single-line or both doesn't bother me, and allowing the requirements to go after the imports isn't the end of the world either (I can see arguments for putting them at the end of the file). But anything works. I'm more concerned with semantics - that the data looks like a comment to Python, and is treated as a list of (project plus optional PEP 440 version specifier) values by I'm not clear why being able to specify an alternative index is important. In my view, "what index to use" is a property of the environment, not of the script, so it doesn't actually belong in the script metadata. There's also complex questions about whether to allow The same argument (it's a property of the environment, not the script) also applies to the various suggestions to let the script specify other To summarise my objections to the other options: Option 1: I don't like "magic variables", and as you say users will expect to be able to build the value at runtime. Apart from this, it's the best of the options I rejected. I've not commented on any of the variants. None of them have any significant impact on my overall opinion of the options. |
I'm looking at adding a similar feature to pipx - see pypa/pipx#913. My proposal there is
This is essentially option 4, with the following differences:
Using a common format would obviously be beneficial. But that can be fixed after the fact, so I'm only mentioning this for context (although if |
The original case is that pkg_resources inspects But I can imagine other scenarios where introspection could be useful:
The use-case that motivated this behavior was I wished to satisfy the need where the script itself has dependencies it knows cannot be satisfied from PyPI and wishes to communicate that in its requirements. It's for the same reason that pip supports Consider for example a repro that demonstrates a fix with privately-published packages: __requires__ = ['cherrypy', 'cheroot==6.5.6.dev28 g9413ed9c']
__index_url__ = 'https://m.devpi.net/jaraco/dev'
# ... demonstrate some behavior only present in cheroot 6.5.6.dev28 g9413ed9c It violates a key principle of pip-run, that invocations should be one-liners or they should be executed as a script. pip-run is trying to get away from the multi-step instructions required for invocation.
Yes. The syntax comes directly from pip (and is passed directly to pip), likely the most used command-line in Python.
The notebook use-case is implemented (#45, example). Of course, the notebook use-case could parse |
Since there's some uncertainty about the future of pip-run, here's what I plan to do - I'll first implement option 1 4, so either format is supported. That way there's compatibility with the proposed pipx format as well as backward compatibility for the existing format. This dual-mode will give the formats a chance to compete for mind share and after some time perhaps one can be deprecated and retired. |
Background
In pypa/pip#3971, key members of the PyPA have leveled critiques of the embedded requirements feature, where a user can supply requirements and other common installer directives to signal to the tool how (best) to run the script. This feature emerged from the use-case where a script author would like to distribute a single file and have that file be runnable by any number of end users with minimal guidance, allowing the file to be published to a gist or alongside other scripts in a directory without needing additional context for executability.
Goals
--extra-index-url
or--quiet
.The text was updated successfully, but these errors were encountered: