Image Copyright © SAWARATSUKI. All rights reserved.
Image used under permission from the copyright holder.
This project provides a boilerplate for building RESTful APIs using Go. It is designed to a quick start with best practices, easy configuration, and a clear project structure.
Warning
This boilerplate is specifically tailored for extensive codebases and scalable RESTful API applications, along with frontend websites that may span multiple sites. Its design ensures scalability and straightforward maintenance, making it highly suitable for complex projects. However, it may not be the ideal choice for smaller-scale RESTful API applications or single-website frontends where such robust architecture is not required.
Tip
Given that this boilerplate focuses on high performance (e.g., it can handle substantial workloads), it is recommended to run and isolate it on Kubernetes. It has been tested alongside the worker package and has proven to be fully stable for low to high workloads (concurrency). Running and isolating it on Kubernetes allows for easy scaling when additional resources, such as vCPUs, are needed.
- Use Fiber Framework
- Pre-configured MySQL and Redis integration
- Middleware examples for logging and authentication
- Scalable project structure
- Dual-licensed under BSD 3-Clause and MIT (specific files)
- High Performance (Thanks to the
Sonic JSON serializing & deserializing library) - Cryptography Techniques
- High Quality Go Codes
- Rich Presence TUIs using
charm.sh Boring TLS 1.3 Protocol (WIP)-> EOL (No longer actively developed due to time constraints)Certificate Transparency (CT) Log-> EOL (No longer actively developed due to time constraints)
Note
Boring TLS 1.3 Protocol This project utilizes the industry-standard TLS 1.3 protocol, known for its exceptional security and reliability. We call it Boring TLS because its robust security makes it almost boring.
Note
Certificate Transparency (CT) Log This project includes built-in support for submitting certificates to Certificate Transparency logs. It supports certificates with both RSA and ECDSA keys. By submitting certificates to CT logs, it enhances the security and transparency of the TLS ecosystem, allowing domain owners to monitor and detect fraudulent certificates or other bad certificates. The CT log functionality is implemented using the SubmitToCTLog function, which takes the certificate, private key, and CT log details as input. It encodes the certificate, calculates the hash, creates a JSON payload, sends an HTTP request to the CT log server, and verifies the signed certificate timestamp (SCT) received in the response. The VerifySCT method is responsible for verifying the SCT based on the public key type (RSA or ECDSA) and ensures the integrity of the timestamp. With this feature, it is easy to integrate certificate transparency into the TLS setup and contribute to a more secure web.
Warning
Some features might not work as expected when running this repo on certain cloud providers. For example, the Rich Presence TUIs feature requires a tty.
The reason why some features might not work as expected after implementation is because this repo is designed to be top-level and follow the Unix philosophy. Plus, most of the Go code in this repo follows many best practices and idioms from Effective Go.
Tip
Due to the large codebase of this repository, which includes many sub-packages such as helper functions (lazy splitting into another repository), consider implementing your own or an alternative main Go mechanism (follow best practices here). This can be used across multiple containers in a single deployment for infrastructure purposes. Personally, I have created many such mechanisms for running in Kubernetes (k8s) based on this starter repository.
Move here
Note
Additional features will be added later as my primary focus is on using this with the Fiber framework and Cryptography Techniques.
Note
The following list includes additional features that I've been using before for extremely scalable and high-performance applications:
The motivation for sharing this RESTful API Boilerplate on GitHub is to streamline my development process, ensuring I don't have to start from scratch each time. It's a robust, secure foundation that adheres to best practices, designed for quick and efficient project initiation.
This boilerplate is grounded in the Unix philosophy, emphasizing simplicity, modularity, and reusability. Each component is designed to do one thing and do it well, with the ability to combine components seamlessly to perform complex tasks. This approach ensures that the boilerplate remains lightweight, efficient, and easy to maintain.
MySQLwithoutRedis100KPOSTRequest (Outdated-go1.22.2)
Note
The Resource Memory Usage section demonstrates how Go has stable and low memory overhead compared to other languages, especially Java (See this article for more information on Java memory management Lmao.) hahaha.
Idle(Outdated-go1.22.2)
Note
The Idle section demonstrates the memory usage when there is no request. The maximum average memory usage is around 21.5MB. Go routines (100 goroutines) along with a semaphore are used to automatically handle high traffic situations (e.g., lots of requests).
Also Note that even with high incoming traffic (e.g., 1 million requests), the maximum average memory usage is still relatively low, around 100MB (e.g., 50MB), due to the use of the semaphore.
Idle(Outdated-go1.22.3)
sample#memory_total=8.01MB
sample#memory_rss=7.92MB
sample#memory_cache=0.09MB
sample#memory_swap=0.00MB
sample#memory_pgpgin=0pages
sample#memory_pgpgout=0pages
sample#memory_quota=1024.00MBNote
Go Memory Usage Improvements:
- The
Idlememory usage in the latest version ofGohas been optimized, reducing it to approximately7.92MBcompared to the previous version.
Average Memory Usage Breakdown:
- The average memory usage of
11.1MBincludes:- Go routine scheduler for database operations (separate from the 100 goroutines used for handling high incoming traffic automatically with a semaphore)
- Load for the
front-endwebsite - Load for the
REST APIs(Full Stack)
- The application remains stable without any performance issues, such as bottlenecks or memory leaks, and has zero vulnerabilities.
Go Performance Comparison:
- Compared to other languages, particularly for full-stack development,
Godemonstrates superior performance.
Standard TLS 1.3(Latest Go Version-go1.22.5)
Note
The screenshots provided demonstrate a Standard TLS 1.3 connection using the Closed networks or intranets method, which involves Cloudflare (Frontend) and Heroku (Backend). Without Cloudflare, any browser or tools like curl won't be able to access the backend server.
Error Page(Include Wildcard Handler)
Note
The screenshot provided demonstrates the performance of an Error Page, and it can easily be optimized to achieve all green metrics.
Tip
Also note that when optimized to achieve all green metrics which is easily, especially in the SEO category, it can be beneficial for business logic purposes. (e.g, Search engines like Google and Microsoft Bing tend to favor websites with good performance metrics, which can lead to improved search rankings and increased visibility.)
Latest Go Version
Note
The grafana dashboard is not shareable due to it being bound to my security configurations for real-world monitoring in other production environments.
Below is the architecture of this boilerplate and how it looks. I created this for REST APIs about volcano 🌋 monitoring used by the government (has been done before), so it can easily monitor volcanoes in the real world.
- Backend (Pure
Go)
backend/
|-- cmd/
| `-- server/ (main application entry point)
| `-- main.go
|-- pkg/
| |-- restapis/ (API route handlers)
| |-- any/ (any related code)
|-- internal/ (private application and library code)
| |-- any/ (any related code)
|-- .env (optional environment variables since it can placed anywhere)
`-- go.mod (dependencies)
- Frontend (Any Framework
TSe.g, React,NextJS or pureJSforHTMX)
frontend/
|-- pages/
| |-- api/ (optional, for Next.js API routes if needed)
| |-- _app.js (global page layouts and state)
| |-- index.js (home page)
| `-- [...other pages]
|-- public/ (static files like images, fonts)
|-- src/
| |-- components/ (shared React components)
| | `-- [various components]
| |-- styles/ (global styles, theme)
| |-- hooks/ (custom React hooks)
| |-- utils/ (utility functions)
| |-- lib/ (libraries and configurations)
| `-- context/ (React context files for state management)
|-- .env.local (environment variables)
|-- next.config.js (Next.js configuration)
`-- package.json (dependencies and scripts)
├── Dockerfile
├── LICENSE
├── README.md
├── SECURITY.md
├── backend
│ ├── cmd
│ │ └── server
│ │ ├── run.go
│ │ ├── run_heroku.go
│ │ ├── run_immutable.go
│ │ └── run_tls_fips.go
│ ├── internal
│ │ ├── database
│ │ │ ├── auth.go
│ │ │ ├── backup.go
│ │ │ ├── cloudflare-kv
│ │ │ │ └── setup.go
│ │ │ ├── constant.go
│ │ │ ├── helper.go
│ │ │ ├── mysql_redis.go
│ │ │ ├── redis_json.go
│ │ │ ├── setup.go
│ │ │ ├── sql_injection_test.go
│ │ │ └── tls.go
│ │ ├── logger
│ │ │ ├── constant.go
│ │ │ └── logger.go
│ │ ├── middleware
│ │ │ ├── authentication
│ │ │ │ ├── crypto
│ │ │ │ │ ├── bcrypt
│ │ │ │ │ │ ├── bcrypt.go
│ │ │ │ │ │ ├── bcrypt_test.go
│ │ │ │ │ │ ├── compare_password.go
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ └── hash_password.go
│ │ │ │ │ ├── cipher.go
│ │ │ │ │ ├── crypto.go
│ │ │ │ │ ├── crypto_test.go
│ │ │ │ │ ├── deadcode.go
│ │ │ │ │ ├── decrypt.go
│ │ │ │ │ ├── encrypt.go
│ │ │ │ │ ├── gopherpocket
│ │ │ │ │ │ └── keyrotation
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ ├── gopherkey.go
│ │ │ │ │ │ └── gopherkey_test.go
│ │ │ │ │ ├── gpg
│ │ │ │ │ │ ├── benchmark_test.go
│ │ │ │ │ │ ├── config.go
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ ├── encrypt.go
│ │ │ │ │ │ ├── encrypt_test.go
│ │ │ │ │ │ ├── file.go
│ │ │ │ │ │ ├── key_info.go
│ │ │ │ │ │ ├── key_ring.go
│ │ │ │ │ │ ├── keybox.go
│ │ │ │ │ │ ├── keybox_test.go
│ │ │ │ │ │ └── new.go
│ │ │ │ │ ├── helper.go
│ │ │ │ │ ├── hybrid
│ │ │ │ │ │ ├── decryptcookie.go
│ │ │ │ │ │ ├── encryptcookie.go
│ │ │ │ │ │ ├── hybrid.go
│ │ │ │ │ │ ├── hybrid_stream.go
│ │ │ │ │ │ ├── hybrid_test.go
│ │ │ │ │ │ └── stream
│ │ │ │ │ │ ├── benchmark_test.go
│ │ │ │ │ │ ├── chunk.go
│ │ │ │ │ │ ├── decrypt_stream.go
│ │ │ │ │ │ ├── digest.go
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ ├── encrypt_stream.go
│ │ │ │ │ │ ├── new_stream.go
│ │ │ │ │ │ ├── nonce.go
│ │ │ │ │ │ └── stream_test.go
│ │ │ │ │ ├── keyidentifier
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ ├── ecdsa_sign.go
│ │ │ │ │ │ ├── generate.go
│ │ │ │ │ │ ├── keyidentifier_test.go
│ │ │ │ │ │ └── new.go
│ │ │ │ │ ├── rand
│ │ │ │ │ │ ├── benchmark_test.go
│ │ │ │ │ │ ├── fixed_size.go
│ │ │ │ │ │ ├── rand_test.go
│ │ │ │ │ │ └── uuid.go
│ │ │ │ │ ├── tls
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ └── setup.go
│ │ │ │ │ ├── vault
│ │ │ │ │ │ ├── new.go
│ │ │ │ │ │ └── transit.go
│ │ │ │ │ ├── web3
│ │ │ │ │ │ └── eth
│ │ │ │ │ │ ├── docs.go
│ │ │ │ │ │ └── new.go
│ │ │ │ │ └── webauthn
│ │ │ │ │ ├── docs.go
│ │ │ │ │ ├── login.go
│ │ │ │ │ ├── protocol.go
│ │ │ │ │ └── registration.go
│ │ │ │ ├── helper
│ │ │ │ │ ├── constant.go
│ │ │ │ │ └── keyauth.go
│ │ │ │ └── keyauth
│ │ │ │ ├── constant.go
│ │ │ │ ├── error.go
│ │ │ │ ├── success.go
│ │ │ │ └── validator.go
│ │ │ ├── constant.go
│ │ │ ├── csp
│ │ │ │ ├── config.go
│ │ │ │ ├── csp_test.go
│ │ │ │ └── new.go
│ │ │ ├── custom_logger_tag.go
│ │ │ ├── custom_next.go
│ │ │ ├── filesystem
│ │ │ │ └── crypto
│ │ │ │ └── signature
│ │ │ │ ├── hmac_sign.go
│ │ │ │ ├── hmac_test.go
│ │ │ │ └── hmac_verify.go
│ │ │ ├── frontend_routes.go
│ │ │ ├── helper.go
│ │ │ ├── init.go
│ │ │ ├── lb_test.go
│ │ │ ├── monitor
│ │ │ │ ├── docs.go
│ │ │ │ └── prometheus.go
│ │ │ ├── restapis_routes.go
│ │ │ ├── restime
│ │ │ │ ├── config.go
│ │ │ │ └── new.go
│ │ │ ├── router
│ │ │ │ └── domain
│ │ │ │ ├── config.go
│ │ │ │ └── new.go
│ │ │ ├── routes.go
│ │ │ ├── routes_immutable.go
│ │ │ ├── routes_non_immutable.go
│ │ │ ├── storage.go
│ │ │ └── utils.go
│ │ ├── server
│ │ │ ├── boringtls.go
│ │ │ ├── boringtls_cert.go
│ │ │ ├── boringtls_cert_test.go
│ │ │ ├── boringtls_test.go
│ │ │ ├── constant.go
│ │ │ ├── ct_verifier.go
│ │ │ ├── ct_verifier_test.go
│ │ │ ├── helper.go
│ │ │ ├── helper_tls_test.go
│ │ │ ├── k8s
│ │ │ │ ├── docs.go
│ │ │ │ └── metrics
│ │ │ │ └── prometheus.go
│ │ │ ├── mount_routes.go
│ │ │ ├── register_routes.go
│ │ │ └── startup_async.go
│ │ └── translate
│ │ └── language.go
│ └── pkg
│ ├── gc
│ │ ├── docs.go
│ │ ├── reduce_http_client_overhead.go
│ │ ├── reduce_overhead.go
│ │ └── unique.go
│ ├── header
│ │ └── htmx
│ │ ├── constant.go
│ │ └── docs.go
│ ├── mime
│ │ ├── ascii.go
│ │ ├── docs.go
│ │ └── mime.go
│ └── restapis
│ ├── helper
│ │ ├── auth
│ │ │ ├── apikey.go
│ │ │ └── constant.go
│ │ ├── generate_apikey.go
│ │ ├── generate_apikey_test.go
│ │ ├── json
│ │ │ └── sonic
│ │ │ ├── config.go
│ │ │ └── docs.go
│ │ ├── numeric.go
│ │ ├── numeric_test.go
│ │ ├── restapis_error.go
│ │ └── restapis_error_test.go
│ └── server
│ └── health
│ ├── cache.go
│ ├── constant.go
│ ├── db.go
│ ├── helper.go
│ ├── mysql.go
│ └── redis.go
├── env
│ ├── docs.go
│ ├── env.go
│ └── getenv.go
├── frontend
│ ├── htmx
│ │ ├── error_page_handler
│ │ │ ├── 400.templ
│ │ │ ├── 400_templ.go
│ │ │ ├── 401.templ
│ │ │ ├── 401_templ.go
│ │ │ ├── 403.templ
│ │ │ ├── 403_templ.go
│ │ │ ├── 404.templ
│ │ │ ├── 404_templ.go
│ │ │ ├── 500.templ
│ │ │ ├── 500_templ.go
│ │ │ ├── 503.templ
│ │ │ ├── 503_templ.go
│ │ │ ├── base.templ
│ │ │ ├── base_templ.go
│ │ │ ├── page_handler.go
│ │ │ ├── render_frontend.go
│ │ │ └── static_handler.go
│ │ ├── public
│ │ │ └── assets
│ │ │ └── css
│ │ │ └── base-tailwind.css
│ │ └── site
│ │ ├── footer.templ
│ │ ├── footer_templ.go
│ │ ├── head.templ
│ │ ├── head_templ.go
│ │ ├── header.templ
│ │ ├── header_templ.go
│ │ ├── script.templ
│ │ └── script_templ.go
│ └── public
│ ├── assets
│ │ ├── css
│ │ │ ├── base-tailwind.css
│ │ │ └── raw.css
│ │ ├── images
│ │ │ ├── android-chrome-192x192.png
│ │ │ ├── android-chrome-512x512.png
│ │ │ ├── apple-touch-icon.png
│ │ │ ├── browserconfig.xml
│ │ │ ├── favicon-16x16.png
│ │ │ ├── favicon-32x32.png
│ │ │ ├── favicon.ico
│ │ │ ├── http_error_codes
│ │ │ │ ├── 403-Forbidden.png
│ │ │ │ ├── 404-NotFound.png
│ │ │ │ ├── 500-InternalServerError.png
│ │ │ │ └── 503-ServiceUnavailable.png
│ │ │ ├── logo
│ │ │ │ └── gopher-run.png
│ │ │ ├── mstile-150x150.png
│ │ │ ├── safari-pinned-tab.svg
│ │ │ └── site.webmanifest
│ │ └── js
│ │ ├── htmx.indicator.min.js
│ │ ├── htmx.min.js
│ │ └── tailwind.min.dark.js
│ └── magic_embedded.go
├── gcloud-builds.yaml
├── go.mod
├── go.sum
├── k8s-deployment
│ ├── MySQL.md
│ ├── README.md
│ ├── REDIS.md
│ ├── RESTAPIs.md
│ ├── component
│ │ ├── coredns
│ │ │ └── coredns-hpa.yaml
│ │ └── secrets
│ │ └── create_k8s_secret.sh
│ ├── mysql-deploy-cpu-boost.yaml
│ ├── mysql-deploy.yaml
│ ├── mysql-storage-eks.yaml
│ ├── mysql-storage-gke.yaml
│ ├── prometheus_portable.yaml
│ ├── prometheus_portable_rules_record.yaml
│ ├── redis-insight.yaml
│ ├── restapis-deploy.yaml
│ ├── restapis-ingress.yaml
│ └── traffic
│ └── nginx
│ ├── ingress-nginx-configmap.yaml
│ ├── ingress-nginx-hpa.yaml
│ ├── ingress-nginx-priority.yaml
│ └── ingress-nginx-vpa.yaml
├── tailwind.config.js
├── translate.json
└── worker
├── config.go
├── do_work.go
├── docs.go
├── jobs.go
├── token_bucket.go
└── worker_test.go
72 directories, 230 files
Note
The Current Boilerplate Tree is designed as a modular framework, and it is easily maintainable even if it reaches 1K files. Personally, I've been maintaining over 1k files as well
from this boilerplate, and it runs smoothly on Kubernetes (K8s) ⛵ ☸.
Server Location: Singapore (Stable and very low latency for Southeast Asian Regions, ranging from 10ms ~ 50ms)
git clone https://git.b0zal.io/H0llyW00dzZ/My-RESTAPIs-Boilerplate.gitNote
Server Management and Isolation by Kubernetes:
- The
Storageis secure and fully encrypted (end-to-end), designed with flexibility in mind. It is suitable forautomated attach/detachprocesses within acluster. - The
Networkutilizes anetwork load balancercontrolled byIngress Nginx, optimizing latency for theAPACregion, ensuring smooth sailing ⛵ ☸. - The
Git SSHshould function properly as it utilizes theTCP Service Nginx. It isfully secured, making it resistant tobrute-force attacksandexploitsdue to itsunderlying logic, which incorporatesElliptic Curve Cryptography (ECC).
Tip
For those in Indonesia, if you are unable to clone repositories (e.g., using git clone) from the Git Mirror (Auto Synced), try using a VPN. This issue may be related to the new load balancer, which could be blocked (by internet provider) or filtered by my firewall mechanism.
It is also more faster with the new load balancer, even when using a VPN (e.g., VPN in Singapore, Indonesia, Malaysia, and other APAC regions), and this improvement applies to both mobile devices and desktops, which incorporate Elliptic Curve Cryptography (ECC) for HTTPS/TLS.
Due to the use of Sonic JSON for encoding/decoding, this boilerplate supports only the following architectures:
AMD64ARM64
Note
It can also be ideal for multi-architecture workloads.
This project is dual-licensed under the BSD 3-Clause License and the MIT License - see the LICENSE file for details.