Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first try for multi-stage build #4

Merged
merged 3 commits into from
Mar 18, 2020
Merged

first try for multi-stage build #4

merged 3 commits into from
Mar 18, 2020

Conversation

nilsnolde
Copy link
Collaborator

Fixes #3

Does now a multi-stage build for the final image:

  1. build stage: installs all dependencies to build vroom and clones both repos
  2. run stage: only installs needed linked libs, copies vroom binary and vroom-express repo from build stage, and installs (and runs) the node project

However, even now the final image size is close to 1 GB. The libboost-all-dev libs are HUGE! If I leave that out of the image, the final size is only 215 MB, even with libssl-dev!

So, I'm wondering @jcoupey, can we do it similar to osrm and only install the boost components vroom needs to run? They do this in their Dockerfile:

apt-get install -y --no-install-recommends libboost-program-options1.62.0 libboost-regex1.62.0 \
        libboost-date-time1.62.0 libboost-chrono1.62.0 libboost-filesystem1.62.0 \
        libboost-iostreams1.62.0 libboost-thread1.62.0 expat liblua5.2-0 libtbb2 &&\

Or would that entail a change in the makefile? I'm hopeless with C builds..

@nilsnolde nilsnolde added the enhancement New feature or request label Mar 11, 2020
@nilsnolde nilsnolde added this to the v1.6.0 milestone Mar 11, 2020
@nilsnolde nilsnolde self-assigned this Mar 11, 2020
@jcoupey
Copy link
Contributor

jcoupey commented Mar 12, 2020

We use two components from boost: asio and optional. They're typically included with:

#include <boost/asio.hpp>
#include <boost/optional.hpp>

My understanding is that those are header-only libraries required at the build stage so you actually don't need any boost-related package at the run stage.

I think that if it weren't for the boost::optional stuff, we would only have to install libasio-dev at the build stage. Would this change the need for a multi-stage build?

@nilsnolde
Copy link
Collaborator Author

I tried using it without the boost dependency in the running stage, but then requests fail with a Internal error vroom error.

I tried to only include libboost-iostreams1.67.0 of the boost suite (earlier versions, e.g. 1.62.0, don't seem to be distributed for debian:buster), but again the same error. Maybe it doesn't include the optional stuff or 1.67.0 is not compatible with vroom?!

I saw the vroom #312 you put in. That would be really cool if boost wouldn't be a dependency at all anymore. Not only for Docker.

Then I won't do any more on this and we can fix once vroom was updated. Ok for you?

@jcoupey
Copy link
Contributor

jcoupey commented Mar 12, 2020

That would be really cool if boost wouldn't be a dependency at all anymore

This won't happen totally because there is still the need for boost::asio (but then installing libasio-dev should be enough for the build stage).

I'm puzzled and not sure the problem comes from either boost::asio or boost::optional because:

  • asio is header-only (see installed files)
  • if think there is no system library required for the boost::optional part (though there don't seem to be a simple way to install only the boost/optional headers without pulling the whole libboost-dev stuff).

We may need a bit of digging here, can you spot the error logged by vroom-express for the Internal error you mention?

@krypt-n
Copy link

krypt-n commented Mar 12, 2020

asio is not exactly header only, since it requires libboost-system (https://www.boost.org/doc/libs/1_72_0/doc/html/boost_asio/using.html) or at least vroom is compiled so that it requires it. So installing

        libboost-system1.67.0
        libssl1.1

should be enough, if vroom is compiled without linking against libosrm.

The uses of boost::system in vroom are pretty limited though and could potentially be removed, if removing libboost-system as a runtime requirement is important.

@jcoupey
Copy link
Contributor

jcoupey commented Mar 12, 2020

@krypt-n thanks for stepping in!

The uses of boost::system in vroom are pretty limited though and could potentially be removed

Could you expand a bit on how we could achieve this? Would this be as simple as replacing the boost::system::error_code in routing/http_wrapper.cpp by some other custom exception?

if vroom is compiled without linking against libosrm.

I don't think we should worry about this (at least for now) in the context of a simple Docker image.

@krypt-n
Copy link

krypt-n commented Mar 12, 2020

Ah sorry, I was mistaken. It seems like boost::system is a hard requirement for asio and thus libboost-system needs to be present at runtime. However, this just adds a at most few MB to the docker image, so I think it can be ignored

@nilsnolde
Copy link
Collaborator Author

nilsnolde commented Mar 13, 2020

Oh great, thanks so much for the insight @krypt-n ! I'll amend that next week!

I don't think we should worry about this (at least for now) in the context of a simple Docker image

Ýeah I agree. In the future, maybe we can extend the Dockerfile to optionally build libosrm with a build argument, so an additional routing container wouldn't be needed.

@nilsnolde
Copy link
Collaborator Author

Yep, that worked @krypt-n, thanks again:) Image size now down to 209MB uncompressed, great!

@nilsnolde nilsnolde marked this pull request as ready for review March 14, 2020 02:50
@nilsnolde nilsnolde requested a review from jcoupey March 14, 2020 02:50
@nilsnolde nilsnolde merged commit 5117030 into master Mar 18, 2020
@nilsnolde nilsnolde deleted the make-multistage branch July 9, 2020 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Decrease final image size
3 participants