Caching is an important part of Docker and any build system. Docker currently does not provide sufficient tools to control caching. Here are some things we need to think about:

Caching layers

When we execute something like run(f"git clone ${URL}"), the layer will be reused as long as ${URL} does not change, even if the git repository gets updated. In principle, we can add something like run(f"git clone ${URL}")::no_cache, but then the cache will be invalidated every time, even if the repository is not updated. BuildKit provides an ad-hoc support for git, but this will not work for other VCS and any other scenarios. So, it would be nice to have a more fine-grained control over caching of layers.

Caching images

In Docker, the registry of images and the build cache are separate things. Thus, it is unclear what our "minimal build tree" means w.r.t. caching, i.e. do we compute the minimal tree w.r.t the images in the database, or w.r.t. to the cached data?

Ideally, we should pick a principled and intuitive approach that is also not too far from what Docker and BuildKit do, so that it was realistic to implement and maintain.

@barr , @maowtm , @thevirtuoso1973 , any thoughts?