Skip to content

stv0g/gose

Repository files navigation

GoSƐ logo

GoSƐ - A terascale file-uploader

goreportcard Codacy grade libraries.io License GitHub go.mod Go version Go Reference

GoSƐ is a modern file-uploader focusing on scalability and simplicity. It only depends on an S3 storage backend and can scale horizontally without the need of an additional database or cache. GoSƐ aims at keeping its deployment simple and by bundling both front end backend components in a single binary and Docker image. GoSƐ has been tested with AWS S3, Ceph's RadosGW and Minio. Pre-built binaries and Docker images of GoSƐ are available for all major operating systems and architectures at the release page.

Features

  • De-duplication of uploaded files based on their content-hash
    • Uploads of existing files will complete in no-time without re-upload
  • S3 Multi-part uploads
    • Resumption of interrupted uploads
  • Drag & Drop of files
  • Browser notifications about failed & completed uploads
  • User-provided object expiration/retention time
  • Copy URL of uploaded file to clip-board
  • Detailed transfer statistics and progress-bar / chart
  • Installation via single binary or container
    • JS/HTML/CSS Frontend is bundled into binary
  • Scalable to multiple replicas
    • All state is kept in the S3 storage backend
    • No other database or cache is required
  • Direct up & download to Amazon S3 via presigned-URLs
    • Gose deployment does not see an significant traffic
  • UTF-8 filenames
  • Multiple user-selectable buckets / servers
  • Optional link shortening via an external service
  • Optional notification about new uploads via shoutrrr
    • Mail notifications to user-provided recipient

Roadmap

Checkout the Github issue tracker.

Installation

Pre-compiled binaries from GitHub releases

Take the download link for your OS/Arch from the Releases Page and run:

export RELEASE_URL=https://github.com/stv0g/gose/releases/download/v0.0.2/gose_0.0.2_linux_amd64
wget "${RELEASE_URL}" -O gose
chmod  x gose
mv gose /usr/local/bin

Kubernetes / Kustomize

  1. Copy default configuration file: cp config.yaml kustomize/config.yaml
  2. Adjust config: nano kustomize/config.yaml
  3. Apply configuration: kubectl apply -k kustomize

Docker

Via environment variables in .env file:

docker run --env-file=.env --publish=8080:8080 ghcr.io/stv0g/gose

or via a configuration file:

docker run -v$(pwd)/config.yaml:/config.yaml --publish=8080:8080 ghcr.io/stv0g/gose -config /config.yaml

Configuration

Gose can be configured via a configuration file and/or environment variables

File

For reference have a look at the example configuration file.

Environment variables

All settings from the configuration file can also be set via environment variables:

Variable Example Value Description
GOSE_LISTEN ":8080" Listen address and port of Gose
GOSE_BASE_URL "http://localhost:8080" Base URL at which Gose is accessible
GOSE_STATIC "./dist" Directory of frontend assets if not bundled
GOSE_BUCKET gose-uploads Name of S3 bucket
GOSE_ENDPOINT s3.0l.de Hostname of S3 server
GOSE_REGION s3 Region of S3 server
GOSE_PATH_STYLE true Prepend bucket name to path
GOSE_NO_SSL false Disable SSL encryption for S3
GOSE_ACCESS_KEY S3 Access Key
GOSE_SECRET_KEY S3 Secret Key
AWS_ACCESS_KEY_ID alias for GOSE_S3_ACCESS_KEY
AWS_SECRET_ACCESS_KEY alias for AWS_SECRET_ACCESS_KEY
GOSE_S3_MAX_UPLOAD_SIZE 5TB Maximum upload size
GOSE_S3_PART_SIZE 5MB Part-size for multi-part uploads
GOSE_S3_EXPIRATION_DEFAULT_CLASS 1week # one of the tags below Default expiration class
GOSE_SHORTENER_ENDPOINT "https://shlink-api/rest/v2/short-urls/shorten?apiKey=<your-api-token>&format=txt&longUrl={{.UrlEscaped}}" API Endpoint of link shortener
GOSE_SHORTENER_METHOD GET HTTP method for link shortener
GOSE_SHORTENER_RESPONSE raw Response type of link shortener
GOSE_NOTIFICATION_URLS pushover://shoutrrr:<api-token>@<user-key>?devices=laptop1&title=Upload Service URLs for shoutrrr notifications
GOSE_NOTIFICATION_TEMPLATE "New Upload: {{.URL}}" Notification message template
GOSE_NOTIFICATION_MAIL_URL smtp://user:password@host:port/[email protected] Service URLs for shoutrrr notifications
GOSE_NOTIFICATION_MAIL_TEMPLATE "New Upload: {{.URL}}" Notification message template

Author

GoSƐ has been written by Steffen Vogel.

License

GoSƐ is licensed under the Apache 2.0 license.