Skip to content

A Cloud Native Buildpack to download tarballs from private S3 buckets or public URLs

License

Notifications You must be signed in to change notification settings

fagiani/s3-tarball-buildpack

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

S3 Tarball Cloud Native Buildpack

This is a Cloud Native Buildpack that can download tarballs from private Amazon S3 buckets. It gives you a way of adding private files outside the main git repository such as certificates, and more complex attributes that won't fit on environment variables to the container at build time without making it publicly accessible.

How it works

This buildpack aims to allow you to write files in build time in any path within the root application directory (/app or /workspace which is an alias). Therefore, with a tar archive you can achieve that by defining the paths desired that will be expanded when downloaded. A second optional benefit is compression when your archives have a significant size they can benefit of a faster download.

Usage

$ cat <<EOF > S3file
s3://my-private-bucket/path/to/tarball.tgz
s3://my-other-bucket/path/to/somethingelse.tgz
s3://S3_AWS_BUCKET_NAME/path/to/somethingelse.tgz
http://my-public-domain.com/tarball.tgz
https://my-other-public-domain.com/path/theother.tgz
EOF

$ pack build my-app --builder heroku/buildpacks:20 --buildpack fagiani/[email protected] \
  --env AWS_ACCESS_KEY_ID=AKIA000000000000000 \
  --env AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxx \
  --env S3_AWS_BUCKET_NAME ...

Alternatively you can use S3_AWS_ACCESS_KEY_ID and S3_AWS_SECRET_ACCESS_KEY to avoid IAM conflicts when using AWS containers to run pack build. If found, S3_AWS_BUCKET_NAME within the S3file will get replaced with the variable's value in case it is set. You can also use AWS IAM temporary keys by setting AWS_SESSION_TOKEN environment variable.

You probably want to use an IAM key with limited access. This code only requires s3:GetObject access to files.

In most cases you'll use this buildpack in conjunction with other buildpacks.

Please notice that public tarball URLs are also accepted and for that no credentials are required.

See also

Contributing

Feel free to contribute by opening a issue or sending a PR.

Licence

MIT license, see LICENSE.txt for details.

About

A Cloud Native Buildpack to download tarballs from private S3 buckets or public URLs

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Languages

  • Shell 100.0%