Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datetime does not support timestamps after 2038 on 32-bit #101069

Open
bmwiedemann opened this issue Jan 16, 2023 · 8 comments
Open

datetime does not support timestamps after 2038 on 32-bit #101069

bmwiedemann opened this issue Jan 16, 2023 · 8 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@bmwiedemann
Copy link
Contributor

Bug report

cpython on 32-bit platforms cannot handle dates beyond 2038-01-19

python3.10 -c "import datetime; datetime.datetime.fromtimestamp(3000000000)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: timestamp out of range for platform time_t

Your environment

  • CPython versions tested on: 3.8, 3.9, 3.10
  • Operating system and architecture: openSUSE-Tumbleweed 20230110 i586

This comes from
https://github.com/python/cpython/blob/8e9d08b0/Modules/_datetimemodule.c#L4982
https://github.com/python/cpython/blob/8e9d08b0/Python/pytime.c#L173

These could use __time64_t if available or we compile everything with __TIMESIZE=64

@sobolevn
Copy link
Member

It works on my machine (macOS 10.14) on main:

(.venv) ~/Desktop/cpython  main ✔                                                    
» ./python.exe -c "import datetime; datetime.datetime.fromtimestamp(3000000000)" 

(.venv) ~/Desktop/cpython  main ✔                                                    
» echo $?                                                                        
0

The same for 3.9:

Python 3.9.15 (main, Nov 22 2022, 17:18:20) 
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime; datetime.datetime.fromtimestamp(3000000000)
datetime.datetime(2065, 1, 24, 8, 20)

I don"t say that this is not a bug, I say that this is totally system-specific.

@mcepl
Copy link
Contributor

mcepl commented Jan 16, 2023

It works on my machine (macOS 10.14) on main:

Is your system 32bit? I doubt it with Mac.

@sobolevn
Copy link
Member

No, it is 64 bit

@bmwiedemann bmwiedemann changed the title datetime does not support timestamps after 2038 datetime does not support timestamps after 2038 on 32-bit Jan 16, 2023
@mcepl
Copy link
Contributor

mcepl commented Feb 6, 2023

No, it is 64 bit

Then your comment is unfortunately irrelevant.

@sobolevn
Copy link
Member

sobolevn commented Feb 6, 2023

@mcepl yeap, that"s what I said :)

I don"t say that this is not a bug, I say that this is totally system-specific.

@bmwiedemann
Copy link
Contributor Author

I guess, this could be solved outside of python by rebuilding our 32-bit Linuxes with
-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64

So that their (or glibc"s) time_t would have 8 bytes.

@calimeroteknik
Copy link

I came across this issue as well on my raspberry pi"s Cortex A7 (32-bit CPU). Two remarks:

This works until year 10000 everywhere (incl. 32bit), buying us some extra time:

rasberrypi ~ $ python3.10 -c "import datetime; print(datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=3000000000))"
2065-01-24 05:20:00

The Y10K problem here is still a limitation if you want to handle timestamps in the (distant) future. I know they said 2000 was far away and it came, I know they said 2038 was far away and now it"s at our door, but Y10K feels sufficiently far away for representing the current time of reality (it will cause problems in Y30K science-fiction — and yes, I"m not too happy when the SF program that ran fine on my laptop crashes on my raspberry pi).

I found a heterogenous mix of errors when going past year 9999:

anyplatform ~ $ python3.10 -c "import datetime; datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=300000000000)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: date value out of range

anyplatform ~ $ python3.10 -c "import datetime; datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=100000000000000)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: days=1157407407; must have magnitude <= 999999999

anyplatform ~ $ python3.10 -c "import datetime; datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=300000000000000)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: Python int too large to convert to C int

Okay, enough goofing around, let"s get to the second remark:
Perl suffers from none of this.

raspberrypi ~ $ perl -le "print scalar localtime 300000000000000"
Thu Jul 21 06:20:00 9508591

None of the Y10K bug, D1G bug ( a Python original) or "C int" shenanigans that we saw with Python above.
It doesn"t seem to mind that the platform is 32bit, or about any -D_SOMETHING_BITS either.

Maybe Perl is enough of a low bar that it"s not egregious to say that Python"s time library is antiquated, buggy, and in need of an urgent fix. Important: I"m not demanding anyone volunteer to fix this; however I do seek unmitigated agreement that it"s indeed Python"s datetime that"s broken on 32bit, not anything else.

@bmwiedemann
Copy link
Contributor Author

however I do seek unmitigated agreement that it"s indeed Python"s datetime that"s broken on 32bit, not anything else.

AFAIU, the problem originates in glibc"s time_t implementation that currently defaults to 32-bit on 32-bit archs and is not trivial to switch for compatibility reasons. It can be solved at the level of distributions/glibc outside of python. But as shown by perl, solutions could be made within python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Development

No branches or pull requests

5 participants