ADOT (AWS Distro for OpenTelemetry) Implementation for Simple Golang RESTful API Application (Bookstore)

Image name	Size
devopscorner/bookstore-adot:latest
devopscorner/bookstore-adot:alpine
devopscorner/bookstore-adot:alpine-latest
devopscorner/bookstore-adot:alpine-3.17

Image name	Size
devopscorner/bookstore-adot:cicd-alpine
devopscorner/bookstore-adot:cicd-alpine-latest
devopscorner/bookstore-adot:cicd-alpine-3.17

Image name	Size
devopscorner/bookstore-adot:cicd-codebuild
devopscorner/bookstore-adot:cicd-codebuild-latest
devopscorner/bookstore-adot:cicd-codebuild-4.0
devopscorner/bookstore-adot:cicd-latest

• Index Documentation, go to this link
• Build, Tag & Push container image to DockerHub, go to this link
• Build, Tag & Push container image to Amazon ECR (Elastic Container Registry), go to this link
• Workflow CI/CD Pipeline, go to this link
• Deployments:
  • AWS Developer Tools (AWS CodeCommit, AWS CodeBuild & AWS CodePipeline), detail here link
  • Jenkins CI/CD, detail here link
  • Terraform AWS CodeBuild, AWS CodePipeline & Amazon SNS, detail here link

• Folder Structure

.
├── .env
├── .env.example
├── config
│   ├── config.go
│   ├── config_test.go
│   ├── const.go
│   ├── logger.go
│   └── value.go
├── controller
│   ├── book_controller.go
│   ├── book_controller_test.go
│   ├── login_controller.go
│   └── login_controller_test.go
├── driver
│   ├── db.go
│   ├── dynamodb.go
│   ├── mysql.go
│   ├── psql.go
│   └── sqlite.go
├── go.mod
├── go.sum
├── main.go
├── main_test.go
├── middleware
│   ├── auth_middleware.go
│   └── auth_middleware_test.go
├── migrate_book.go.example
├── migrate_book_dynamo.go.example
├── model
│   └── book.go
├── observability
│   ├── metrics.go
│   ├── provider.go
│   ├── tracing-otel.go
│   └── tracing-xray.go
├── repository
│   └── book_repository.go
├── routes
│   ├── book_routes.go
│   ├── main_routes.go
│   ├── telemetry_routes.go
│   └── tracing_routes.go
└── view
    ├── book_view.go
    ├── error_view.go
    └── login_view.go

10 directories, 37 files

• Amazon Elastic Container Registry (ECR)
• Amazon Managed Service for Prometheus (AMP)
• Amazon Managed Service for Grafana (AMG)
• AWS Distro for OpenTelemetry (ADOT)
• AWS CloudWatch
• AWS X-Ray
• AWS Systems Manager Parameter Store
• AWS Developer Tools (AWS CodeCommit, AWS CodePipeline, AWS CodeBuild)
• Amazon EKS
• Amazon OpenSearch -- (Optional)
• Amazon DynamoDB

• Docker
• Docker-Compose
• Golang
• HelmChart
• Postman
• Terraform

• Default Value

  GIN_MODE=release
  APP_URL=http://localhost
  APP_PORT=8080
  DB_CONNECTION=dynamo
  DB_HOST=localhost
  DB_PORT=
  DB_DATABASE=dynamodb-golang-adot
  DB_USERNAME=root
  DB_PASSWORD=
  JWT_AUTH_USERNAME=devopscorner
  JWT_AUTH_PASSWORD=DevOpsCorner2023
  JWT_SECRET=s3cr3t
  

• Multi Driver Connection

  # default
  DB_CONNECTION=dynamo
  ---
  Available for:
  - sqlite
  - mysql
  - postgres
  - dynamo
  

• DynamoDB Connection

  DB_CONNECTION=dynamo
  ---
  DB_DATABASE       --> Dynamo Table
  AWS_REGION        --> AWS Region (Dynamo Region)
  AWS_ACCESS_KEY    --> AWS Access Key
  AWS_SECRET_KEY_ID --> AWS Secret Key Id
  

• AWS X-Ray (AWS Distribution Tracing System)

  • XRAY_VERSION: Daemon Version X-Ray

    • latest: Use the latest version of the AWS X-Ray daemon available.
    • 3.x: Use version 3.x of the AWS X-Ray daemon.
    • 2.x: Use version 2.x of the AWS X-Ray daemon.
    • 1.x: Use version 1.x of the AWS X-Ray daemon.

  • `XRAY_DAEMON_ENDPOINT: Daemon Endpoint of X-Ray

    XRAY_DAEMON_ENDPOINT=http://localhost:2000
    

  • `XRAY_DAEMON_PORT: Daemon Endpoint Port of X-Ray

    XRAY_DAEMON_PORT=2000
    

• Prometheus

  PROMETHEUS_ENDPOINT=http://localhost:9090
  PROMETHEUS_PORT=9090
  

• Grafana

  GRAFANA_ENDPOINT=http://localhost:3000
  GRAFANA_API_KEY=YOUR_GRAFANA_API_KEY
  

• OpenTelemetry

  OTEL_INSTRUMENTATION_METRIC_ENABLED=true
  OTEL_INSTRUMENTATION_TRACE_ENABLED=true
  OTEL_INSTRUMENTATION_LOG_ENABLED=true
  
  # Trace Type: xray / jaeger
  OTEL_INSTRUMENTATION_TRACE_NAME=xray
  
  OTEL_SERVICE_NAME=bookstore-adot
  OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
  OTEL_EXPORTER_OTLP_PORT=4317
  OTEL_EXPORTER_OTLP_INSECURE=true
  OTEL_EXPORTER_OTLP_HEADERS=
  OTEL_RESOURCE_ATTRIBUTES=
  

• Jaeger Environment

  • JAEGER_SERVICE_NAME: The name of the service being instrumented (JAEGER_SERVICE_NAME = OTEL_SERVICE_NAME).
  • JAEGER_AGENT_HOST: The host name or IP address of the Jaeger agent to use for sending trace data.
  • JAEGER_AGENT_PORT: The port number of the Jaeger agent to use for sending trace data.
  • JAEGER_SAMPLER_TYPE: The type of sampling to use for tracing (e.g. const, probabilistic, rateLimiting, remote).
  • JAEGER_SAMPLER_PARAM: The parameter value to use for the selected sampler type.
  • JAEGER_SAMPLER_MANAGER_HOST_PORT: The host name and port of the remote sampling manager to use (if using remote sampling).
  • JAEGER_REPORTER_LOG_SPANS: Whether to log spans instead of sending them to a Jaeger agent (e.g. true or false).
  • JAEGER_REPORTER_MAX_QUEUE_SIZE: The maximum number of spans that can be queued before they are sent to a Jaeger agent.
  • JAEGER_REPORTER_FLUSH_INTERVAL: The interval at which to flush the span queue and send spans to a Jaeger agent (e.g. 1 * time.Second).
  • JAEGER_REPORTER_LOCAL_AGENT_HOST_PORT: The host and port of the local Jaeger agent, if using UDP instead of HTTP.
  • JAEGER_REPORTER_COLLECTOR_ENDPOINT: The endpoint URL of the Jaeger collector (e.g. http://localhost:14268/api/traces).
  • JAEGER_REPORTER_COLLECTOR_USER: The username for authenticating with the Jaeger collector, if required.
  • JAEGER_REPORTER_COLLECTOR_PASSWORD: The password for authenticating with the Jaeger collector, if required.
  • JAEGER_TAGS: Additional tags to attach to the trace (e.g. key1=value1,key2=value2).

• RESTful API Testing, go to this link

1. Install Terraform Modules, detail here
2. Provisioning HelmChart Template, detail here
3. Provisioning TFState DB & Bucket, detail here
4. Provisioning Infra Core, detail here
5. Provisioning Infra CI/CD, detail here
6. Provisioning Resources Amazon EKS, detail here
7. Provisioning Resources Managed Services Prometheus (AMP), detail here
8. Provisioning Resources Managed Services Grafana (AMG), detail here
9. Provisioning Resources Managed Services OpenSearch, detail here
10. Provisioning Resources EKS ADOT Collector, detail here

• Grafana Dashboard

• XRay

• Docker version

  docker version
  
  Client:
    Cloud integration: v1.0.22
    Version:           20.10.17-rd
    API version:       1.41
    Go version:        go1.17.11
    Git commit:        c2e4e01
    Built:             Fri Jul 22 18:31:17 2022
    OS/Arch:           darwin/amd64
    Context:           default
    Experimental:      true
  

• Docker-Compose version

  docker-compose -v
  ---
  Docker Compose version v2.11.1
  

• AWS Cli

  aws --version
  ---
  aws-cli/2.8.7 Python/3.9.11 Darwin/21.6.0 exe/x86_64 prompt/off
  

• Terraform Cli

  terraform version
  ---
  Terraform v1.3.5
  on darwin_amd64
  - provider registry.terraform.io/hashicorp/aws v3.74.3
  - provider registry.terraform.io/hashicorp/local v2.1.0
  - provider registry.terraform.io/hashicorp/null v3.1.0
  - provider registry.terraform.io/hashicorp/random v3.1.0
  - provider registry.terraform.io/hashicorp/time v0.7.2
  

• Terraform Environment Cli

  tfenv -v
  ---
  tfenv 2.2.2
  

• Golang Version Manager (GVM)

  gvm version
  ---
  Go Version Manager v1.0.22 installed at /Users/devopscorner/.gvm
  

• Golang Version

  go version
  ---
  go version go1.19.6 darwin/arm64
  

Make sure that you didn't push sensitive information in this repository

• AWS Credentials (AWS_ACCESS_KEY, AWS_SECRET_KEY)
• AWS Account ID
• AWS Resources ARN
• Username & Password
• Private (id_rsa) & Public Key (id_rsa.pub)
• DNS Zone ID
• APP & API Key

• Author: Dwi Fahni Denni (@zeroc0d3)
• Vendor: DevOps Corner Indonesia (devopscorner.id)
• License: Apache v2