Secrets

The secrets package provides access to key management services in a portable way. This guide shows how to work with secrets in the Go CDK.

Cloud applications frequently need to store sensitive information like web API credentials or encryption keys in a medium that is not fully secure. For example, an application that interacts with GitHub needs to store its OAuth2 client secret and use it when obtaining end-user credentials. If this information was compromised, it could allow someone else to impersonate the application. In order to keep such information secret and secure, you can encrypt the data, but then you need to worry about rotating the encryption keys and distributing them securely to all of your application servers. Most Cloud providers include a key management service to perform these tasks, usually with hardware-level security and audit logging.

The secrets package supports encryption and decryption operations.

Subpackages contain driver implementations of secrets for various services, including Cloud and on-prem solutions. You can develop your application locally using localsecrets, then deploy it to multiple Cloud providers with minimal initialization reconfiguration.

Opening a SecretsKeeper🔗

The first step in working with your secrets is to instantiate a portable *secrets.Keeper for your service.

The easiest way to do so is to use secrets.OpenKeeper and a service-specific URL pointing to the keeper, making sure you “blank import” the driver package to link it in.

import (
	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/<driver>"
)
...
keeper, err := secrets.OpenKeeper(context.Background(), "<driver-url>")
if err != nil {
    return fmt.Errorf("could not open keeper: %v", err)
}
defer keeper.Close()
// keeper is a *secrets.Keeper; see usage below
...

See Concepts: URLs for general background and the guide below for URL usage for each supported service.

Alternatively, if you need fine-grained control over the connection settings, you can call the constructor function in the driver package directly (like awskms.OpenKeeper).

import "gocloud.dev/secrets/<driver>"
...
keeper, err := <driver>.OpenKeeper(...)
...

You may find the wire package useful for managing your initialization code when switching between different backing services.

See the guide below for constructor usage for each supported service.

Using a SecretsKeeper🔗

Once you have opened a secrets keeper for the secrets provider you want, you can encrypt and decrypt small messages using the keeper.

Encrypting Data🔗

To encrypt data with a keeper, you call Encrypt with the byte slice you want to encrypt.

plainText := []byte("Secrets secrets...")
cipherText, err := keeper.Encrypt(ctx, plainText)
if err != nil {
	return err
}

Decrypting Data🔗

To decrypt data with a keeper, you call Decrypt with the byte slice you want to decrypt. This should be data that you obtained from a previous call to Encrypt with a keeper that uses the same secret material (e.g. two AWS KMS keepers created with the same customer master key ID). The Decrypt method will return an error if the input data is corrupted.

var cipherText []byte // obtained from elsewhere and random-looking
plainText, err := keeper.Decrypt(ctx, cipherText)
if err != nil {
	return err
}

Large Messages🔗

The secrets keeper API is designed to work with small messages (i.e. <10 KiB in length.) Cloud key management services are high latency; using them for encrypting or decrypting large amounts of data is prohibitively slow (and in some providers not permitted). If you need your application to encrypt or decrypt large amounts of data, you should:

1. Generate a key for the encryption algorithm (16KiB chunks with secretbox is a reasonable approach).
2. Encrypt the key with secret keeper.
3. Store the encrypted key somewhere accessible to the application.

When your application needs to encrypt or decrypt a large message:

1. Decrypt the key from storage using the secret keeper
2. Use the decrypted key to encrypt or decrypt the message inside your application.

Keep Secrets in Configuration🔗

Once you have opened a secrets keeper for the secrets provider you want, you can use a secrets keeper to access sensitive configuration stored in an encrypted runtimevar.

First, you create a *runtimevar.Decoder configured to use your secrets keeper using runtimevar.DecryptDecode. In this example, we assume the data is a plain string, but the configuration could be a more structured type.

decodeFunc := runtimevar.DecryptDecode(keeper, runtimevar.StringDecode)
decoder := runtimevar.NewDecoder("", decodeFunc)

Then you can pass the decoder to the runtime configuration provider of your choice. See the Runtime Configuration How-To Guide for more on how to set up runtime configuration.

Other Usage Samples🔗

• CLI Sample
• Secrets package examples

Supported Services🔗

Google Cloud Key Management Service🔗

The Go CDK can use keys from Google Cloud Platform’s Key Management Service (GCP KMS) to keep information secret. GCP KMS URLs are similar to key resource IDs.

secrets.OpenKeeper will use Application Default Credentials; if you have authenticated via gcloud auth application-default login, it will use those credentials. See Application Default Credentials to learn about authentication alternatives, including using environment variables.

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/gcpkms"
)

keeper, err := secrets.OpenKeeper(ctx,
	"gcpkms://projects/MYPROJECT/" 
		"locations/MYLOCATION/" 
		"keyRings/MYKEYRING/" 
		"cryptoKeys/MYKEY")
if err != nil {
	return err
}
defer keeper.Close()

GCP Constructor🔗

The gcpkms.OpenKeeper constructor opens a GCP KMS key. You must first obtain GCP credentials and then create a gRPC connection to GCP KMS.

import (
	"context"

	"gocloud.dev/secrets/gcpkms"
)

// Get a client to use with the KMS API.
client, done, err := gcpkms.Dial(ctx, nil)
if err != nil {
	return err
}
// Close the connection when done.
defer done()

// You can also use gcpkms.KeyResourceID to construct this string.
const keyID = "projects/MYPROJECT/"  
	"locations/MYLOCATION/"  
	"keyRings/MYKEYRING/"  
	"cryptoKeys/MYKEY"

// Construct a *secrets.Keeper.
keeper := gcpkms.OpenKeeper(client, keyID, nil)
defer keeper.Close()

AWS Key Management Service🔗

The Go CDK can use customer master keys from Amazon Web Service’s Key Management Service (AWS KMS) to keep information secret. AWS KMS URLs can use the key’s ID, alias, or Amazon Resource Name (ARN) to identify the key. You should specify the region query parameter to ensure your application connects to the correct region.

If you set the “awssdk=v1” query parameter, secrets.OpenKeeper will create a default AWS Session with the SharedConfigEnable option enabled; if you have authenticated with the AWS CLI, it will use those credentials. See AWS Session to learn about authentication alternatives, including using environment variables.

If you set the “awssdk=v2” query parameter, it will instead create an AWS Config based on the AWS SDK V2; see AWS V2 Config to learn more.

If no “awssdk” query parameter is set, Go CDK will use a default (currently V1).

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/awskms"
)

// Use one of the following:

// 1. By ID.
keeperByID, err := secrets.OpenKeeper(ctx,
	"awskms://1234abcd-12ab-34cd-56ef-1234567890ab?region=us-east-1")
if err != nil {
	return err
}
defer keeperByID.Close()

// 2. By alias.
keeperByAlias, err := secrets.OpenKeeper(ctx,
	"awskms://alias/ExampleAlias?region=us-east-1")
if err != nil {
	return err
}
defer keeperByAlias.Close()

// 3. By ARN.
const arn = "arn:aws:kms:us-east-1:111122223333:key/"  
	"1234abcd-12ab-34bc-56ef-1234567890ab"
keeperByARN, err := secrets.OpenKeeper(ctx,
	"awskms://" arn "?region=us-east-1")
if err != nil {
	return err
}
defer keeperByARN.Close()

// Use "awssdk=v1" or "v2" to force a specific AWS SDK version.
keeperUsingV2, err := secrets.OpenKeeper(ctx,
	"awskms://1234abcd-12ab-34cd-56ef-1234567890ab?region=us-east-1&awssdk=v2")
if err != nil {
	return err
}
defer keeperUsingV2.Close()

AWS Constructor🔗

The awskms.OpenKeeper constructor opens a customer master key. You must first create an AWS session with the same region as your key and then connect to KMS:

import (
	"github.com/aws/aws-sdk-go/aws/session"
	"gocloud.dev/secrets/awskms"
)

// Establish an AWS session.
// See https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ for more info.
sess, err := session.NewSession(nil)
if err != nil {
	return err
}

// Get a client to use with the KMS API.
client, err := awskms.Dial(sess)
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper := awskms.OpenKeeper(client, "alias/test-secrets", nil)
defer keeper.Close()

awskms.OpenKeeperV2 is similar but uses the AWS SDK V2.

import (
	"context"

	awsv2cfg "github.com/aws/aws-sdk-go-v2/config"
	"gocloud.dev/secrets/awskms"
)

// Establish a AWS V2 Config.
// See https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/ for more info.
ctx := context.Background()
cfg, err := awsv2cfg.LoadDefaultConfig(ctx)
if err != nil {
	return err
}

// Get a client to use with the KMS API.
client, err := awskms.DialV2(cfg)
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper := awskms.OpenKeeperV2(client, "alias/test-secrets", nil)
defer keeper.Close()

Azure KeyVault🔗

The Go CDK can use keys from Azure KeyVault to keep information secret. secrets.OpenKeeper will use default credentials from the environment, unless you set the environment variable AZURE_KEYVAULT_AUTH_VIA_CLI to true, in which case it will use credentials from the az command line.

Azure KeyVault URLs are based on the Azure Key object identifer:

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/azurekeyvault"
)

// The "azurekeyvault" URL scheme is replaced with "https" to construct an Azure
// Key Vault keyID, as described in https://docs.microsoft.com/en-us/azure/key-vault/about-keys-secrets-and-certificates.
// You can add an optional "/{key-version}" to the path to use a specific
// version of the key; it defaults to the latest version.
keeper, err := secrets.OpenKeeper(ctx, "azurekeyvault://mykeyvaultname.vault.azure.net/keys/mykeyname")
if err != nil {
	return err
}
defer keeper.Close()

Azure Constructor🔗

The azurekeyvault.OpenKeeper constructor opens an Azure KeyVault key.

import "gocloud.dev/secrets/azurekeyvault"

// Makes a client to use with the Azure KeyVault API, using default
// authorization from the environment.
clientMaker := azurekeyvault.DefaultClientMaker

// Construct a *secrets.Keeper.
keeper, err := azurekeyvault.OpenKeeper(clientMaker, "https://mykeyvaultname.vault.azure.net/keys/mykeyname", nil)
if err != nil {
	return err
}
defer keeper.Close()

HashiCorp Vault🔗

The Go CDK can use the transit secrets engine in Vault to keep information secret. Vault URLs only specify the key ID. The Vault server endpoint and authentication token are specified using the environment variables VAULT_SERVER_URL and VAULT_SERVER_TOKEN, respectively.

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/hashivault"
)

keeper, err := secrets.OpenKeeper(ctx, "hashivault://mykey")
if err != nil {
	return err
}
defer keeper.Close()

HashiCorp Vault Constructor🔗

The hashivault.OpenKeeper constructor opens a transit secrets engine key. You must first connect to your Vault instance.

import (
	"context"

	"github.com/hashicorp/vault/api"
	"gocloud.dev/secrets/hashivault"
)

// Get a client to use with the Vault API.
client, err := hashivault.Dial(ctx, &hashivault.Config{
	Token: "CLIENT_TOKEN",
	APIConfig: api.Config{
		Address: "http://127.0.0.1:8200",
	},
})
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper := hashivault.OpenKeeper(client, "my-key", nil)
defer keeper.Close()

Local Secrets🔗

The Go CDK can use local encryption for keeping secrets. Internally, it uses the NaCl secret box algorithm to perform encryption and authentication.

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/localsecrets"
)

// Using "base64key://", a new random key will be generated.
randomKeyKeeper, err := secrets.OpenKeeper(ctx, "base64key://")
if err != nil {
	return err
}
defer randomKeyKeeper.Close()

// Otherwise, the URL hostname must be a base64-encoded key, of length 32 bytes when decoded.
// Note that base64.URLEncode should be used, to avoid URL-unsafe characters.
savedKeyKeeper, err := secrets.OpenKeeper(ctx, "base64key://smGbjm71Nxd1Ig5FS0wj9SlbzAIrnolCz9bQQ6uAhl4=")
if err != nil {
	return err
}
defer savedKeyKeeper.Close()

Local Secrets Constructor🔗

The localsecrets.NewKeeper constructor takes in its secret material as a []byte.

import "gocloud.dev/secrets/localsecrets"

secretKey, err := localsecrets.NewRandomKey()
if err != nil {
	return err
}
keeper := localsecrets.NewKeeper(secretKey)
defer keeper.Close()