Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: cannot pickle Language object with v2.0.1 #199

Open
tboye opened this issue Nov 29, 2023 · 6 comments
Open

TypeError: cannot pickle Language object with v2.0.1 #199

tboye opened this issue Nov 29, 2023 · 6 comments
Milestone

Comments

@tboye
Copy link

tboye commented Nov 29, 2023

After switching to v2.0.1 from v1.3.4 I'm facing TypeError: cannot pickle 'builtins.Language' object when running something along the lines of

import copy
from lingua import Language

l = copy.copy(Language.ENGLISH)

I've checked it with Python 3.10 and Python 3.11.

Am I missing something?

@pemistahl pemistahl changed the title TypeError: cannot pickle 'builtins.Language' object with v2.0.1 TypeError: cannot pickle Language object with v2.0.1 Nov 29, 2023
@pemistahl
Copy link
Owner

Pickling does not work out-of-the-box anymore because you are dealing with a Rust object now, not a Python enum. Pickle support with PyO3 is possible but complicated, that's why I have left it out so far until people start to complain which happened quicker now than I expected.

Why do you need pickle support in the first place? If you are using the multiprocessing module, it is not needed anymore since Lingua now brings its own parallelization methods. If you want to persist Language objects to disk, have you thought about persisting their string representations instead, e.g. Language.ENGLISH.name?

@tboye
Copy link
Author

tboye commented Nov 29, 2023

Thanks for the explanation.

I'm using a framework that pickles its settings and one of the settings happened to be a lingua.Language object in my case. But yeah, I can work it around by having a string representation in the settings and then creating an object from it.

Thanks for a nice library!

@pemistahl
Copy link
Owner

You are welcome. It's good that you have opened this issue. I will try to implement pickle support for the next major release as it is definitely useful to have.

@pemistahl pemistahl modified the milestones: Lingua 2.0.2, Lingua 2.1.0 Dec 6, 2023
@vrdn-23
Copy link

vrdn-23 commented Dec 22, 2023

I would like to see this issue cause we're looking to implement something that doesn't block an async event loop. So we get this same error when trying to put it into a ProcessPoolExecutor. Looking forward to having this ASAP! Thanks a lot and great work!

@DanielHabenicht
Copy link

We came across the same problem in combination with supplying a parameter based on lingua.Languages to python-dependency-injection

@pemistahl A similar error occurs if one wants to use normal enum conversion:

>>> from lingua import IsoCode639_1, Language
>>> IsoCode639_1["en"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type 'builtins.IsoCode639_1' is not subscriptable
>>> 

Should I open another one for this?

@pemistahl
Copy link
Owner

@DanielHabenicht I will fix these errors shortly. I'm currently preparing a new major release 1.4.0 for the pure Python edition of the library. After this is done, I will concentrate on the Rust edition again.

Thanks for your patience everyone. With more kids and less time than last year, further development of my projects has slowed down but not stopped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants