Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there no preferred approach to single-sourcing the project version? #182

Closed
nchammas opened this issue Nov 9, 2015 · 50 comments
Closed
Labels
type: question A user question that needs/needed an answer

Comments

@nchammas
Copy link

nchammas commented Nov 9, 2015

There are many approaches offered in the current documentation, but no approach seems to be "recommended" or otherwise suggested as the preferred approach to take. For a new developer looking to package their project, this can be confusing.

Looking at a few of the projects I'm familiar with, it seems that approach 3 is common.

Set the value to a __version__ global variable in a dedicated module in your project (e.g. version.py), then have setup.py read and exec the value into a variable.

Would it make sense to present this approach as more than just one among many? Or is there really no preferred way to single-source the project version, even for new projects?

@pfmoore
Copy link
Member

pfmoore commented Nov 9, 2015

Typically any attempt to recommend a preferred solution is met with debate. There really doesn't seem to be a consensus.

FWIW, if I recall the common objection to the approach you mention here is that it doesn't work properly if your project build directory isn't on sys.path (which it may not be in certain situations).

The objections to the other options seem to me to be no stronger (but no weaker either).

@nchammas
Copy link
Author

nchammas commented Nov 9, 2015

FWIW, if I recall the common objection to the approach you mention here is that it doesn't work properly if your project build directory isn't on sys.path (which it may not be in certain situations).

Is that a common occurrence? I'm not trying to start a debate about this now; I just ask as a total newcomer to packaging. :)

I suppose if there really is no consensus, or even no approach that we can offer as "works for most people", then we can close this issue.

@pfmoore
Copy link
Member

pfmoore commented Nov 9, 2015

I don't know, it's not my objection...

@qwcode
Copy link
Contributor

qwcode commented Nov 9, 2015

It think approach #3 is becoming more common (I use it now btw), but it's hard for me to say what is globally "more common", w/o doing some kind of analysis of what's being posted to pypi these days.

let's keep this issue open, and maybe #3 can become preferred at some point.

@pfmoore
Copy link
Member

pfmoore commented Nov 9, 2015

BTW, my explanation of the objection I've seen is wrong (if you exec the file,you don't need it on sys.path).

One real issue is getting encodings right. The execfile approach is Python 2 only. The exec version needs an open() but Python 2 doesn't support supplying an encoding to open(), and the default encoding depends on the user's PC settings. The best approach is only using ASCII in your version.py and relying on the fact that nobody's going to have a default encoding that isn't ASCII-compatible. But all it takes is a non-ASCII character in a comment and the read gets a decoding error :-(

But it's a corner case, and not likely to affect newcomers (um, unless they don't speak English as their native language, I guess...)

@dstufft
Copy link
Member

dstufft commented Nov 9, 2015

You could always just add myproject/_version which is just a text file, then have:

# myproject/__init__.py
import pkgutil

__version__ = pkgutil.get_data("myproject", "_version").strip()

and

# setup.py
import os.path

with open(os.path.join(os.path.dirname(__file__), "myproject", "_version")) as fp:
    version = fp.read().strip()

setup(
    ...
    version = version,
    ...
)

@dstufft
Copy link
Member

dstufft commented Nov 9, 2015

Also, codecs.open is like open except it supports an encoding parameter and works on 2 and 3.

@qwcode
Copy link
Contributor

qwcode commented Nov 9, 2015

You could always just add myproject/_version which is just a text file

btw, this is already presented as one of the options... #4

@Changaco
Copy link
Contributor

Note: #190 proposes changes to the document that would affect this discussion.

@Changaco
Copy link
Contributor

The problem I have with all the solutions currently listed in the document is that if your project uses Git tags to mark releases, then there isn't a single source of truth anymore: there's the version number in the code and the one in the tags.

@nchammas
Copy link
Author

That's correct, but I think that falls outside of the domain of what the PUG should cover. If you want your single-source version to be in Git, then you're coupling how Python versioning is done to something outside of Python. What about users who use Mercurial or SVN? Should the guide cover those as well?

As far as the PUG is concerned, I think agreeing on the best approach to having a single source for the version across your Python code is what we should shoot for. Once we have that, it's easier to make additional recommendations on how to add an additional layer of indirection to source that version from outside of Python.

So to provide an example, say the guide evolves to converge on option 3. From there, it's much easier to add tips on how to source that version in turn from a VCS.

For example:

Set the value to a __version__ global variable in a dedicated module in your project (e.g. version.py), then have setup.py read and exec the value into a variable.
You can set __version__ automatically by querying your VCS. For example, in Git...

@xavfernandez
Copy link
Member

Why option 3 and not any other one ? :-)
Note also that all these discussions relate to the use of distutils/setuptools as build tool.

@nchammas
Copy link
Author

Per the discussion earlier in this thread, option 3 seems to be the option that new projects are converging on. Do you observe otherwise?

I think the PUG serves the community best when it recognizes already-existing conventions, or when it helps nudge the community towards emerging conventions and accelerates their adoption. I don't think anyone here is trying to invent something new and impose it on people by diktat.

Personally, which option we go with is less important to me than the fact that we converge on a single "recommended" option. It's better for users when there are fewer choices on display for implementing such a basic pattern.

@nchammas
Copy link
Author

To expand a bit: The PUG is a guide, not an encyclopedia. Its purpose is to present common advice to users facing common problems, not to document every possible permutation that people have found useful.

At least, that's how I see the PUG. Perhaps I've misunderstood its purpose though.

@xavfernandez
Copy link
Member

Per the discussion earlier in this thread, option 3 seems to be the option that new projects are converging on. Do you observe otherwise?

I must admit I usually don't check the version hack used by the library I use but at work we tend to use something similar to pip: https://github.com/pypa/pip/blob/develop/setup.py#L34-L39 so that would be option 1.

@pfmoore
Copy link
Member

pfmoore commented Nov 26, 2015

@nchammas That sounds pretty much correct.

@Changaco
Copy link
Contributor

Since #190 has become tangled with this issue I've turned it into the following proposal:

  • simplify option 1 (__version__ in __init__) and use the same coding style in option 3 (__version__ in __about__)
  • move option 2 (external tools) to the end
  • remove options 4 (complicated variant of option 3) and 5 (unreliable use of pkg_resources)
  • remove option 6 (importing __version__ from __init__) and mention why it's not recommended (in the note of option 1)
  • put the VCS tag solution in its own item, because it's not a release tool so it doesn't fit into option 2

@Changaco
Copy link
Contributor

I've kept both option 1 and option 3 because I can't find a good argument to declare that one is better than the other.

@Changaco
Copy link
Contributor

In #190 @pfmoore points out that option 1 is less simple and solid because it uses regexps. That could justify dropping it in favour of option 3. Opinions?

@pfmoore
Copy link
Member

pfmoore commented Nov 27, 2015

No, I was pointing out that your edits omitted the note that the fact that it used regexps is a factor. I don't think there's anything new in the fact that option 1 uses regexps. So -1 on dropping it "because it uses regexps".

Remember that the original question here was "Is there no preferred approach to single-sourcing the project version?" My answer to that is still "No there isn't, sorry". (But for my personal projects, I don't actually care about single-sourcing the version, having it in a couple of places isn't an issue to me).

@Changaco
Copy link
Contributor

Sorry, I didn't mean to misrepresent what you said. Your comment about the missing note made me realize that option 1 is less simple and solid.

@pfmoore
Copy link
Member

pfmoore commented Nov 27, 2015

No problem

@Changaco
Copy link
Contributor

Changaco commented Dec 3, 2015

Sorry #190 failed to solve this issue, @nchammas.

Considering the reluctance that was expressed to removing options from the page, maybe splitting the options in two sections (e.g. "recommended" and "other") would be a better approach. I won't be the one making that PR though.

@ionelmc
Copy link
Contributor

ionelmc commented Feb 15, 2016

As I understood it, but correct me if I'm wrong, the point of not being opinionated is to prevent the inevitable disagreements and bike-shedding. And that's why PPUG don't want to recommend one way to do it.

That being said, I still believe it should be easier for the user to pick something. Currently you'd be inclined to pick something based on ordering - so there's already some implied recommendation there ;-)

I propose to have a list of pros and cons for each option, and then the user can look over those and decide what's important for him. Cause no one has the same needs.

@ncoghlan
Copy link
Member

ncoghlan commented Sep 24, 2016

Belatedly revisiting this, I do agree there's value in offering folks a prescriptive "I don't care about the details, just give me a 'not wrong' approach that covers absolutely everything I need to do".

However, I also think it's problematic for the PyPA specifically to publish such a guide, since it's easy for folks to interpret "this is one good approach" as "all other possible approaches are bad" and try to use it as a lever to coerce maintainers of existing projects into changing the way they do things even when there's nothing wrong with their current approach (cf. folks wanting to retroactively apply modern versions of PEP 8 to code bases that predate those updates by many years).

Accordingly, perhaps we could take the expedient route and offer a link at the start of the Packaging & Distribution guide referencing @hynek's walk-through at https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/ ?

That kind of "This is a good, comprehensive, example of doing it right" endorsement is softer than actually providing such a prescriptive guide directly on packaging.python.org, but still meets the need for a step-by-step introduction that doesn't ask first-time publishers to make any decisions they may not be comfortable making yet.

@pfmoore
Copy link
Member

pfmoore commented Sep 24, 2016

One minor thing - @hynek's post says "Sorry, no Windows", which in't needed as (as far as I can see) there's basically nothing in the post that's OS-specific.

But that nit aside, I agree with the principle of pointing to articles that we endorse to break the logjam of "we don't want to say anything as it makes it too official".

@hynek
Copy link

hynek commented Sep 24, 2016

If someone can confirm that everything works on Windows as described, I’ll be more than delighted to remove it. :)

@pfmoore
Copy link
Member

pfmoore commented Sep 24, 2016

I haven't run all of the commands, but I have reviewed everything and the only points I'd make are:

  1. The read() function in setup.py assumes UTF-8. Windows users will have to specifically ensure that they get their text editor to save in UTF-8. But honestly, it's the only sane cross-platform choice, so that's not a problem.
  2. In the "You can test whether both install properly" sections, the commands are Unix, but there's no way you could avoid some level of platform specificity in an example like this. I'd just add a comment "the examples use Unix, Windows would be similar".

(Honestly, as a Windows user, I'm used to reading articles on the net that use Unix terms, and mentally translating. I wouldn't even have thought about the issue if you hadn't explicitly mentioned it).

@iMichka
Copy link

iMichka commented Dec 11, 2016

Just adding my 2 cents to the discussion. I stumbled over this a few days ago, as I also have 4 places where I need to change the version number in my library, and failed to do so correctly already twice.

I really like method 1, as the __version__ is defined in your main __init__.py as a string, which I would expect from most packages. It is the first place where I would look for it. If the problem is that regexes are perceived as hard / obscure for some people, why could setuptools not propose the find_version method ? It really looks like some boilerplate code that could benefit to everybody. That would hide the complexity away from the end user, and allow the setup tools maintainer to take care of the details.

About method 4, I find that saying:

An advantage with this technique is that it’s not specific to Python. Any tool can read the version

is probably not really an advantage. It's just more direct. You can probably extract that version from a .py file with any other language or tool without too much hassle.

And I think that having 7 options is way too many. Besides, at the end you end up reading this thread to get even more opinions :), which is great but ... not realistic for most of the users.

@flying-sheep
Copy link
Contributor

flying-sheep commented Aug 2, 2019

None of these is single-source, as pretty much everything is under version control these days, and people tag versions. I prefer using the VCS as source for the version using get_version or versioneer hatch-vcs.

@ChrisBarker-NOAA
Copy link
Contributor

PIng! This is a really old issue, and a lot of work (including at least two PRs merged has been done -- close it?

Note: there's some work still to be done (things move fast!) on the Single-sourcing the package version page, but that's for another issue or PR.

@flying-sheep
Copy link
Contributor

hatch-vcs is a good solution. To switch to it, one simply has to do

 [build-system]
 build-backend = "hatchling.build"
 requires = [
   "hatchling",
   "hatch-vcs",
 ]

 [project]
 ...
-version = "..."
 dynamic = ["version"]

 [tool.hatch.version]
 source = "vcs"

@pfmoore
Copy link
Member

pfmoore commented Jul 11, 2023

I think this issue can be closed. In the end, there will always be multiple approaches, even if only because people don't all agree on which source should be the single source.

The packaging guide could do with updating - personally, I'd structure it by starting from "where do you want to hold the single copy of your version?" and discuss options for each of the potential choices from there - but that's separate, and really just needs people to do the work of writing new text, rather than more discussions of what options exist.

hatch-vcs is a good solution.

... if you're using hatch, and you want to use a git tag as your master version 🙂

@flying-sheep
Copy link
Contributor

I fully agree with everything you said, just wanted to give some practical advice for people stumbling upon this thread.

@wimglenn
Copy link
Contributor

wimglenn commented Jul 12, 2023

Can we just remove this page entirely? It's terribly quaint.

The modern ways as far as I can see are a version derived from VCS, or just written statically/directly in pyproject.toml like:

[project]
version = "1.0"

Duplication of a __version__ attribute in the module namespace (whether that was "burnt-into" the source by the build, or fetched from the source during the build) has been unnecessary and convoluted approach ever since importlib.metadata made it to stdlib (Python 3.8), and the PEP396 rejection notice from 2021 says similar:

The packaging ecosystem has changed significantly in the intervening years since this PEP was first written, and APIs such as importlib.metadata.version() provide for a much better experience.

The whole page seems like a hangover from distutils days and we'd be in a better place just removing such outdated advice.

@ChrisBarker-NOAA
Copy link
Contributor

The whole page seems like a hangover from distutils days and

Well, yes, which is why I did WIP PR #1273 -- in that PR I didn't actually remove anything, but it may be time to do that.

we'd be in a better place just removing such outdated advice.

I take issue with some of this -- despite the SC's ruling on PEP396, I really like version, and it absolutely can be done with single source, as recommend, and as setuptools will do for you.

NOTE: PEP 396 was complicated by ideas about putting Version in the standard library -- I still think it's excellent advise for third party packages: KISS

That's a debate for somewhere else -- in this thread, we've specifically said that these docs are not going to bless any particular method.

@hynek
Copy link

hynek commented Jul 13, 2023

JFTR, you can have it both ways: https://github.com/hynek/stamina/blob/main/src/stamina/__init__.py

Simply add this to your __init__.py:

def __getattr__(name: str) -> str:
    if name != "__version__":
        msg = f"module {__name__} has no attribute {name}"
        raise AttributeError(msg)

    from importlib.metadata import metadata

    return metadata("YOUR-PKG")["version"]

I used to raise a deprecation warning but turns out users really like __version__.

@ChrisBarker-NOAA
Copy link
Contributor

Exactly -- MTOWTDI -- which is the point of this thread -- no definitive recommendation.

The question is what the Docs should say -- let's have that discussion in the PR.

@flying-sheep
Copy link
Contributor

flying-sheep commented Jul 13, 2023

turns out users really like __version__.

I’m not sure that’s a good reason to keep it.

importlib.metadata.version('distname') is the standardized way to do this and always works, while __version__ is just a convention that is often correctly followed, but not always.

@hynek
Copy link

hynek commented Jul 13, 2023

see python-attrs/attrs#1136 for the concrete issue. As far as I understand, it's impossible to reason from an imported module to the package that installed it (happy to be corrected tho). And looks like pydoc looks at __version__ too.

@flying-sheep
Copy link
Contributor

flying-sheep commented Jul 13, 2023

it's impossible to reason from an imported module to the package that installed it (happy to be corrected tho)

Happy to oblige: importlib.metadata.packages_distributions() gives you a mapping.

@wimglenn
Copy link
Contributor

@hynek Is there a reason why you went with importlib.metadata.metadata("stamina")["version"] instead of importlib.metadata.version("stamina")?

@hynek
Copy link

hynek commented Jul 13, 2023

Three theories:

  1. Copy paste.
  2. It's the first I found and it was good enough.
  3. In legacy projects I mapped all the data and it was easier to map dunders like this (see eg attrs)

@wimglenn
Copy link
Contributor

I offer #1276 as an alternative to #1273.

@sinoroc
Copy link
Contributor

sinoroc commented Jul 22, 2023

I feel like it is a matter of deciding what we want the scope of packaging.python.org to be. From one of the latest discuss.python.org thread (a few months ago already), I was under the impression that the direction was towards minimizing the scope as much as possible to focus mostly on the packaging specifications and the things that are part of Python and its standard library (in other words the things that are under our/PyPA's control).

So in this case, I guess it would be enough to mention importlib.metadata.version() and how to use it. The other techniques such as those involving build back-ends plugins would be out of scope. We could mention that those exist but without naming them in particular. If we name them then we have to ensure over time that these techniques and plugins still exist and are still up-to-date with the latest specifications, otherwise we are misleading users.

@wimglenn
Copy link
Contributor

Right, the approaches shown in #1273 would be more at home in the setuptools docs. They're features quite specific to a setuptools-based build backend. The Python Packaging User Guide should strive not to be so closely wedded to setuptools.

@sinoroc
Copy link
Contributor

sinoroc commented Jul 23, 2023

I also wonder about the possible confusion between distribution package and import package. Should import something; print(something.__version__) always report the same value as importlib.metadata; print(importlib.metadata.version('Something')).

Even in the cases where both are strictly the same (and importing the package is not too costly in terms of resources), aren't we being misleading by recommending to obtain the version string from the import package instead of from the installed distribution package metadata.

@webknjaz
Copy link
Member

webknjaz commented Sep 6, 2023

I offer #1276 as an alternative

FTR I commented on this one if anybody from this thread wants to take a look.

@pradyunsg pradyunsg added the type: question A user question that needs/needed an answer label Nov 5, 2023
@ncoghlan
Copy link
Member

#1612 resolves this by explaining that pkg.__version__ and importlib.metadata.version("pkg") are reporting the version of two different things (an import package or module and a distribution package) that happen to share a name.

The old setuptools-specific guide is now just a HTTP redirect to the discussion page that mostly defers to "consult the documentation of your chosen build system".

@ChrisBarker-NOAA
Copy link
Contributor

two different things (an import package or module and a distribution package) that happen to share a name

They may not have the same name -- one reason why both are useful.

I'll take another look at #1612 and see if there's anymore clarification needed there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question A user question that needs/needed an answer
Projects
None yet
Development

No branches or pull requests