Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not working in heroku #62

Closed
umlx5h opened this issue Mar 20, 2018 · 15 comments
Closed

Not working in heroku #62

umlx5h opened this issue Mar 20, 2018 · 15 comments
Labels
heroku question

Comments

@umlx5h
Copy link

umlx5h commented Mar 20, 2018 •
edited
Loading

Hi.
Can this gem work in heroku environment?
I use python 3.6, ruby 2.5.0 and rails 5.1.4 version in heroku.

I set PYCALL_DEBUG_FIND_LIBPYTHON=1 variable and got an following error.

Loading production environment (Rails 5.1.4)
irb(main):001:0> require 'pycall/import'
=> false
irb(main):002:0> include PyCall::Import
=> Object
irb(main):003:0> pyimport :math
DEBUG(find_libpython) find_libpython(".heroku/python/bin/python")
DEBUG(find_libpython) investigate_python_config(".heroku/python/bin/python")
DEBUG(find_libpython) libs: ["libpython3.6m.a", "libpython3.6m", "libpython3.6", "libpython"]
DEBUG(find_libpython) libpaths: ["/app/.heroku/python/lib", "/app/.heroku/python/lib", "", "/app/.heroku/python", "/app/.heroku/python/lib"]
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /libpython3.6m.a
DEBUG(find_libpython) Unable to find /libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /libpython3.6m
DEBUG(find_libpython) Unable to find /libpython3.6m.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6m.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /libpython3.6
DEBUG(find_libpython) Unable to find /libpython3.6.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.6.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /libpython
DEBUG(find_libpython) Unable to find /libpython.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.6m.a: invalid ELF header
DEBUG(find_libpython) Fiddle::DLError: libpython3.6m: cannot open shared object file: No such file or directory
DEBUG(find_libpython) Fiddle::DLError: libpython3.6: cannot open shared object file: No such file or directory
DEBUG(find_libpython) Fiddle::DLError: libpython: cannot open shared object file: No such file or directory
Traceback (most recent call last):
        1: from (irb):3
PyCall::PythonNotFound (PyCall::PythonNotFound)

Python can work by using command.

Loading production environment (Rails 5.1.4)
irb(main):001:0> system('python --version')
Python 3.6.4
=> true
irb(main):002:0> system('which python')
/app/.heroku/python/bin/python
=> true
irb(main):003:0> system('echo $PATH')
/app/vendor/bundle/ruby/2.5.0/bin:/app/bin:/app/vendor/bundle/bin:vendor/yarn-v1.0.2/bin/:/app/.heroku/python/bin:/usr/local/bin:/usr/bin:/bin
=> true
irb(main):004:0>

thanks.
@umlx5h umlx5h changed the title not working in heroku Not working in heroku Mar 20, 2018
@mrkn
Copy link
Owner

mrkn commented Apr 2, 2018

@umlx5h pycall.rb requires that Python is configured with --enable-shared.

But heroku's python seems not configured with --enable-shared.
See https://github.com/heroku/heroku-buildpack-python/blob/master/builds/runtimes/python-3.6.4#L13

@richgong
Copy link

richgong commented Jul 3, 2019

I've forked it to see if this fixes the problem: https://github.com/richgong/heroku-buildpack-python

@raphaelfarzankashani
Copy link

Hi @mrkn - how can I access this repo and this link? https://github.com/heroku/heroku-buildpack-python/blob/master/builds/runtimes/python-3.6.4#L13

Any way to contact you directly?

@mrkn
Copy link
Owner

mrkn commented Aug 19, 2019

@raphaelfarzankashani I think @richgong's this commit fixes this issue.

Why do you want to contact me directly?

@mrkn
Copy link
Owner

mrkn commented Aug 19, 2019

I close this issue because this is not pycall.rb's responsibility.

@mrkn mrkn closed this as completed Aug 19, 2019
@dsounded
Copy link

Hello, @mrkn and thank you for the gem

I am trying to make it work on Heroku and had no luck...

Still PyCall::PythonNotFound: PyCall::PythonNotFound

This is my fork which should patch --enable-shared issue.

dsounded/heroku-buildpack-python@43cf2de

Did anything change before this topic was discussed for the last time ?
It even can't start the Puma now because of that error

(works perfectly locally)

@dsounded dsounded mentioned this issue Jan 25, 2021
Closed
@dsounded
Copy link

Here is the output:

DEBUG(find_libpython) find_libpython(nil)
DEBUG(find_libpython) investigate_python_config("python3")
DEBUG(find_libpython) libs: ["libpython3.9.a", "libpython3.9", "libpython"]
DEBUG(find_libpython) libpaths: ["/app/.heroku/python/lib", "/app/.heroku/python/lib", "", "/app/.heroku/python", "/app/.heroku/python/lib"]
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /libpython3.9.a
DEBUG(find_libpython) Unable to find /libpython3.9.a.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.a.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /libpython3.9
DEBUG(find_libpython) Unable to find /libpython3.9.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython3.9.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /libpython
DEBUG(find_libpython) Unable to find /libpython.so
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/libpython.so
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython
DEBUG(find_libpython) Unable to find /app/.heroku/python/lib/x86_64-linux-gnu/libpython.so
DEBUG(find_libpython) Fiddle::DLError: /app/.heroku/python/lib/libpython3.9.a: invalid ELF header
DEBUG(find_libpython) Fiddle::DLError: libpython3.9: cannot open shared object file: No such file or directory
DEBUG(find_libpython) Fiddle::DLError: libpython: cannot open shared object file: No such file or directory
Traceback (most recent call last):
	4: from -e:1:in `<main>'
	3: from /app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall.rb:13:in `builtins'
	2: from /app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall/init.rb:16:in `const_missing'
	1: from /app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall/init.rb:35:in `init'
/app/vendor/bundle/ruby/2.7.0/gems/pycall-1.3.1/lib/pycall/libpython/finder.rb:95:in `find_libpython': PyCall::PythonNotFound (PyCall::PythonNotFound)

@mrkn
Copy link
Owner

mrkn commented Jan 25, 2021 •
edited
Loading

The debug log showed that pycall couldn't find libpython.so in all the candidate locations.

Where did you install the python built with --enable-shared? You should specify the location of the appropriate python command in PYTHON environment variable.

@dsounded
Copy link

Yeah. actually wanted to leave a comment with the solution...
For Heroku it works with the fork I provided you need to specify LIBPYTHON as /app/.heroku/python/bin/python3.9 in my case (I wanted to use 3.9.0) since default python and python3 execs are infrastructures ones.

To find needed executable path you can run heroku run whereis python that helped me to find a path for needed version and now it looks good.

Thank you again for your time and this library!
Maybe this comment will help someone in the future :)

@ferrisoxide
Copy link
Contributor

I realise this issue is closed, but if it helps anyone I managed to get pycall working Heroku working using this buildpack:

https://github.com/ReforgeHQ/heroku-buildpack-python.git

@mrkn
Copy link
Owner

mrkn commented Jan 30, 2021

Is there anyone who can write the description in README to explain how to use pycall on Heroku?

@ferrisoxide ferrisoxide mentioned this issue Feb 2, 2021
Merged
@ferrisoxide
Copy link
Contributor

ferrisoxide commented Feb 2, 2021 •
edited
Loading

Is there anyone who can write the description in README to explain how to use pycall on Heroku?

@mrkn Took a stab at it here (#134). Might be worth someone else running their eyes over the delta, as I was only going by my notes from a project a few months back. Hope this is of some use.

@Tshamp7
Copy link

Tshamp7 commented Jun 7, 2021

Yeah. actually wanted to leave a comment with the solution...
For Heroku it works with the fork I provided you need to specify LIBPYTHON as /app/.heroku/python/bin/python3.9 in my case (I wanted to use 3.9.0) since default python and python3 execs are infrastructures ones.

To find needed executable path you can run heroku run whereis python that helped me to find a path for needed version and now it looks good.

Thank you again for your time and this library!
Maybe this comment will help someone in the future :)

when I attempt to specify LIBPYTHON with the correct path from heroku I get:

Fiddle::DLError: /app/.heroku/python/bin/python3.9: cannot dynamically load position-independent executable

@dsounded do you know how to fix this? thanks!

@Tshamp7
Copy link

Tshamp7 commented Jun 10, 2021

Yeah. actually wanted to leave a comment with the solution...
For Heroku it works with the fork I provided you need to specify LIBPYTHON as /app/.heroku/python/bin/python3.9 in my case (I wanted to use 3.9.0) since default python and python3 execs are infrastructures ones.
To find needed executable path you can run heroku run whereis python that helped me to find a path for needed version and now it looks good.
Thank you again for your time and this library!
Maybe this comment will help someone in the future :)

when I attempt to specify LIBPYTHON with the correct path from heroku I get:

Fiddle::DLError: /app/.heroku/python/bin/python3.9: cannot dynamically load position-independent executable

@dsounded do you know how to fix this? thanks!

I had to clear my build cache on heroku. Then it worked fine.

@buncis
Copy link
Contributor

buncis commented Jul 28, 2021 •
edited
Loading

@dsounded @Tshamp7 hello I have same problem and still can't make it work even with dsounded branch

I'm trying to use python -3.9.6 with --enable-shared
actually how to specify the LIBPYTHON that you guys talk about ?

I've set it on my config var
as

heroku config:set LIBPYTHON = /app/.heroku/python/bin/python3.9

or

heroku config:set libpython = /app/.heroku/python/bin/python3.9

and clear my build cache still it gives error when my rails app checking the rake tasks

my runtime.txt
python-3.9.6

and I'm using heroku-20 stack

apparently after checking my python3 on heroku its not builded with --enabled-shared even though I have using buildpacks that have --enable-shared options

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
heroku question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@mrkn @ferrisoxide @richgong @dsounded @buncis @umlx5h @raphaelfarzankashani @Tshamp7