Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile multiphase build cache issues #3116

Open
4 tasks
ilyapoz opened this issue Mar 13, 2018 · 9 comments
Open
4 tasks

Dockerfile multiphase build cache issues #3116

ilyapoz opened this issue Mar 13, 2018 · 9 comments

Comments

@ilyapoz
Copy link
Contributor

ilyapoz commented Mar 13, 2018

High Level Spec

Original Post

Description of problem:
Let's say I use this Dockerfile:

Dockerfile.txt

report.txt

Environment details (AWS, VirtualBox, physical, etc.):
vps

How was Dokku installed?:
znz.dokku ansible galaxy role

How reproducible:
every time

Steps to Reproduce:

  1. push an app consisting only of the Dockerfile attached
  2. push some meaningless commit to trigger new build

Actual Results:
RUN echo in builder phase is not cached

Expected Results:
RUN echo is cached as in docker build . run

Additional info:
main phase builds are cached correctly by content! I.e. changing the exact method of creating some_file.txt in build phase doesn't affect caching in the main phase.

@josegonzalez
Copy link
Member

Can you gist the output of a deploy before and after running dokku trace on? I have a feeling that our cleanup step is removing the intermediate build layer since its not used, but getting confirmation of that would be great.

@ilyapoz
Copy link
Contributor Author

ilyapoz commented Mar 30, 2018

@josegonzalez
Copy link
Member

Can you run dokku trace off and run the deploy?

@ilyapoz
Copy link
Contributor Author

ilyapoz commented Mar 30, 2018

@alexquick
Copy link
Contributor

@josegonzalez I took a look and the image from the builder phase is getting removed at: https://github.com/dokku/dokku/blob/master/plugins/common/functions#L650 as you suspected.

@josegonzalez
Copy link
Member

Not sure there is a great way around that, given our deployment methodology. Any image cleanup will break this, so perhaps having folks manage that on their own when they are using multi-stage builds is the best way?

@josegonzalez
Copy link
Member

Looks like we might be able to leverage dockfmt to list all stages in an image, and then build the image by specifying targets/tagging each layer. We'd then avoid removing tagged layers during the cleanup stage.

@josegonzalez
Copy link
Member

So my original plan won't work unless each stage is named, meaning we would have to either drop the cache-fix for non-named stages or rewrite the dockerfile to ensure each stage is named. I don't know how I feel about either, though the later method is probably possible using a fun regex and the post-extract hook.

@hiepxanh
Copy link
Contributor

hiepxanh commented Jun 4, 2021

hello, I think i have solution for this, I'm looking a way to cache my package manager dont have to reinstall every time have CI build or dokku build on server and now I have it https://github.com/dokku/dokku/blob/c1df31b106da6eb8c7897deea67c143e2b5e611a/docs/deployment/builders/dockerfiles.md
I have make an PR for the docs on here
#4635

image

in my case I'm using pnpm (the same like npm or yarn) and I spend about 10 minutes or 20 minutes for each time package.json file change, and with the new command, It can save me about 5 minutes build time, which save alot of money sir :


FROM node:14-alpine as builder
ENV CYPRESS_INSTALL_BINARY=0
RUN --mount=type=cache,target=/root/.pnpm-store/v3
RUN pnpm install --frozen-lockfile --prod

image

@josegonzalez josegonzalez added this to the v1.0.0 milestone Aug 14, 2021
@josegonzalez josegonzalez added the estimate: 3h Estimated time: 3 hours label Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants