This project tests and describes the workflow of creating an AWS CDK construct using projen jsii and publishing it to various repositories like npm, Maven Central, PyPi and NuGet.
In order to verify the process is working as expected, this project contains an LambdaConstruct
.
It only creates a very simple Lambda function using inline code that prints out the event it's receiving.
Btw. did you know there are multiple ways to bundle an AWS Lambda function in a CDK construct?
Note: If you are reading this on npm, NuGet, or PyPi as part of the package description, then please head over to https://github.com/seeebiii/projen-test. Otherwise the links might not work properly.
Do you have further questions? Feel free to reach out via Twitter or visit my website - I'm a Freelance Software Engineer focusing on serverless cloud projects.
There's also a CDK developer Slack workspace that you can join to ask questions.
- About projen
- About jsii
- Requirements
- Setup Project
- Verify Your CDK Construct
- Next Steps
- Additional Help
projen is a tool to write your project configuration using code instead of managing it yourself. It was initially created to help you writing CDK constructs but can also be used to create any other project stub like a React app.
The important thing to note is that you only define the project configuration in the .projenrc.js file and it will generate everything for you, like package.json
or other files.
Remember: do not manually change the generated files, otherwise changes will be lost after you run projen
again.
Besides that, it is recommended to create an alias in your command line to avoid typing npx projen
over and over again.
For example: alias pj='npx projen'
Further links:
- A Beginner's Guide to Create AWS CDK Construct Library with projen
- Converting a CDK construct to using projen
jsii is the technology behind the AWS CDK that allows you to write CDK constructs in TypeScript/JavaScript and compile them to other languages like Java or Python. There's an AWS blog post about how it works.
There are a few jsii related projects that support us in the steps below, e.g. for releasing our artifacts.
- ~30 of your time (maybe more if you run into errors or need to read through a few documents)
- Node.js/npm installed on your machine
- AWS CDK installed on your machine
- GitHub Account (free account is enough)
- AWS Account (for verification)
- Java Maven (for local packaging & verification)
- Python (for local packaging & verification)
In case you want to test projen
as well, here are the steps I've performed.
-
Initialize a new project using
projen
:mkdir projen-test cd projen-test npx projen new awscdk-construct
-
Now create a new Git repository using
git init
and connect an existing Git project usinggit remote add origin <REMOTE_URL>
. (Create your Git repository in GitHub first before you callgit remote add ...
)- Note: if your local branch is
master
but your remote branch ismain
, then usegit branch -M main
to rename the local branch.
- Note: if your local branch is
-
Create an alias for your command-line if you haven't done already:
alias pj='npx projen'
-
Adjust the
projen
options in.projenrc.js
a bit. For example:- adjust metadata like
name
,author
,authorName
,authorAddress
,repositoryUrl
. By defaultname
is also used as thename
in the generatedpackage.json
. However, you can define a custom one by usingpackageName
. - add
projectType: ProjectType.LIB
since we'll create a library - add
cdkAssert: true
for being able to test my CDK construct - add
cdkDependencies: ['@aws-cdk/core', '@aws-cdk/aws-lambda']
to letprojen
add these CDK dependencies for me - optional: add
mergify: false
if you don't want to use it at the moment - optional: explicitly add
docgen: true
so it automatically generates API documentation π - optional: explicitly add
eslint: true
to make sure you use common coding standards - optional: add
dependabot: true
anddependabotOptions: {...}
to enable Dependabot if you hate manually managing dependency updates - optional: add
gitignore: ['.idea']
if you love using IntelliJβ₯οΈ but don't want to commit its settings - or if you want to ignore any other files :) - optional: use
packageManager: NodePackageManager.NPM
if you want to usenpm
instead ofyarn
- might be important in case you are migrating an existing CDK Construct toprojen
.
Don't forget to add necessary imports in the config file when applying the
projen
settings, e.g. for usingProjectType
orNodePackageManager
. - adjust metadata like
-
Set a version number in version.json, e.g.
0.0.1
. -
Run
pj
again on your command-line. This will update all project files like package.json based on what you have configured in .projenrc.js. Remember to not manually update these files asprojen
will override your changes otherwise.
π‘ Hint: I can recommend to play around with the options a little bit and run pj
after each change.
Then observe what happens and how the project files differ.
If you commit the changes to Git each time after running pj
, you can easily compare the Git diff π
-
Write a simple CDK construct in
src/index.ts
. There are already great tutorials like cdkworkshop.com available about how to write constructs. Here's a small code snippet for a simple Lambda function using inline code:new Function(this, 'SampleFunction', { runtime: Runtime.NODEJS_12_X, code: Code.fromInline('exports.handler = function (e, ctx, cb) { console.log("Event: ", e); cb(); };'), handler: 'index.handler', timeout: Duration.seconds(10), });
Have a look at LambdaConstruct.ts for an examples construct that declares various Lambda functions. Instead of using a Lambda function, you can also use whatever you like.
-
Write a simple test for this construct in test/index.test.ts. Here's also a small code snippet which ensures that our Lambda function is created in the stack:
test('Simple test', () => { const app = new cdk.App(); const stack = new cdk.Stack(app, 'TestStack'); new LambdaConstruct(stack, 'LambdaConstruct'); expectCDK(stack).to(countResources('AWS::Lambda::Function', 1)); });
The test is creating a stack and verifies that the stack contains exactly one resource of type
AWS::Lambda::Function
. Have a look at index.test.js for further details. -
Run
yarn run build
. This command will compile the code, execute the tests using Jest and also generate API.md π
Add all files to Git, commit and push your changes.
# Maybe adjust this to really add all files from every folder, or use IntelliJ or some Git GUI to do this
git add .
git commit -M "Initial commit"
git push -u origin main
If you want to release your package to one or multiple repositories like npm or Maven Central, you'll have to enable this in .projenrc.js.
After changing the configuration as described below and calling pj
(or npx projen
if you're not using the alias), you'll notice that there are a few files in .github/workflows folder.
They take care of running GitHub Actions to build and release/publish the library to npm and other repositories.
The action steps are using jsii-superchain as the Docker image to run a step.
The release process is also using the npm module jsii-release to help releasing the artifacts. The project's README is explaining all the different parameters that you need to set for publishing to the different repositories that are supported. Currently, npm (Node.js), Maven (Java), NuGet (C#) and PyPi (Python) are supported.
I'm assuming you already have an npm account. If not, register first.
-
Update .projenrc.js and enable npm releases:
releaseBranches: ['main']
releaseToNpm: true
-> Yes, I want to publish a new version to npm every time :)releaseWorkflow: true
- Optional:
releaseEveryCommit: true
-> will run the release GitHub Action on each push to the definedreleaseBranches
-
Run
pj
to update your project files. You'll notice release.yml contains a few steps to build and release your artifact. Therelease_npm
step requires anNPM_TOKEN
so that the GitHub Action can publish your package for you. -
Create an access token for npm. Use type 'Automation' for the token type. The token will be used in a GitHub Action to release your package.
-
Add the access token as a repository secret to your GitHub repository. Use
NPM_TOKEN
as name and insert the access token from the previous step. This is -
Commit and push your changes. If you have configured
releaseBranches: ['main']
in .projenrc.js as discussed above, then a new Action run is triggered that builds your CDK construct and publishes it to npm.
After the last step, you'll notice that the first GitHub Action should be running.
Have a look at the published package: @seeebiii/projen-test
Since we want to make our CDK construct public, I'm describing the steps to publish it to Maven Central, the main Maven repository (like npm fo Node.js packages). However, this process requires a few more steps compared to npm.
Maven Requirements
In order to publish to Maven Central, you need an account and also "authenticate" yourself using GPG. This ensures that others can verify the correctness of your artifacts and no one else distributes them to introduce vulnerable code.
-
Create an account for Maven Central and register your own namespace like
org.example
if you are the owner of the domainexample.org
. This is described in this OSSRH Guide. If you don't have your own domain, you can also usecom.github.<your-github-user>
instead. The Jira system has a bot installed that helps you verifying your domain or your GitHub user, so just register and create an issue as described. The bot will tell you what to do next π -
Upload a PGP key to a well-known key registry, so other people can verify that the artifacts are from you. This is explained well in the jsii-release README in the subsection How to create a GPG key?.
-
Please save the passphrase of your PGP key somewhere safe, e.g. a password manager. You'll need it in a step below!
Setup Maven Release Action
-
Update .projenrc.js and enable Java releases to Maven Central:
publishToMaven: { mavenGroupId: '<your_package_group_id', mavenArtifactId: '<your_package_target_id>', javaPackage: '<your_java_package>', }
You can specify anything you want as long as the namespace you've registered with Maven Central is a prefix of
mavenGroupId
. For example, I have registeredde.sebastianhesse
since I also own the domain www.sebastianhesse.de. Therefore, I can usede.sebastianhesse.examples
as themavenGroupId
. -
Run
pj
to update your project files. You'll notice changes in the release.yml and a new steprelease_maven
. As you can see, it uses a few secrets likeMAVEN_GPG_PRIVATE_KEY
,MAVEN_GPG_PRIVATE_KEY_PASSPHRASE
,MAVEN_USERNAME
,MAVEN_PASSWORD
, andMAVEN_STAGING_PROFILE_ID
. These are the same as described in the jsii-release README. If you need help figuring out theMAVEN_STAGING_PROFILE_ID
, then please see below in the Additional Help section. -
Add all the required secrets as repository secrets to your GitHub repository.
-
Commit and push your changes.
Like with the npm release process above, as soon as you push your changes a GitHub Action is triggered and performs the release. You'll notice that the release process for Maven takes a lot more time than the one for npm. Also take care that it might take some time to find your artifact in Maven Central (usually within a few minutes but it can take longer).
Have a look at the published package: projen-test
In order to publish to PyPi, you need an account there.
-
Update .projenrc.js configuration:
publishToPypi: { distName: '<distribution-name>', module: '<module_name>', },
-
Run
pj
to update your project files. Again, release.yml has been updated and arelease_pypi
step has been added. The step requires two secrets:TWINE_USERNAME
andTWINE_PASSWORD
. -
Use your PyPi
username
forTWINE_USERNAME
andpassword
forTWINE_PASSWORD
. Add the secrets as repository secrets to your GitHub repository.
Have a look at the published package: projen-test
-
Update .projenrc.js configuration:
publishToNuget: { dotNetNamespace: 'Organization.Namespace', packageId: 'Package.Name', },
-
Run
pj
to update your project files. Again, release.yml has been updated and arelease_nuget
step has been added. The step requires the secretNUGET_API_KEY
. -
Generate an API Key for your account and use it for
NUGET_API_KEY
. Add the secret as a repository secret to your GitHub repository.
Have a look at published package: Projen.Test
Congratulations π
You have successfully published your first CDK construct using projen
to multiple repositories.
(At least you hope so at the moment π)
Let's verify that our construct is working as expected in different languages. Since my main programming languages are Java and Node.js/JavaScript, I'll only present those two here. If I have time, I'll add the others as well - or if you want to contribute something? π
Note: when deploying a CDK app, you need to provide the AWS Account ID and Region.
You can do this either using environment variables in the CLI or by explicitly providing environment
details in the file/class where the CDK app is defined.
See this page for further help.
-
Initialize a new CDK app in TypeScript:
mkdir cdk-npm-test cd cdk-npm-test cdk init app --language=typescript
This will initialize a new CDK app project. You can also use
--language=javascript
which initializes the project using JavaScript instead of TypeScript. Inbin/cdk-npm-test.ts
you'll find the CDK app definition and inlib/cdk-npm-test-stack.ts
you can find the stack definition. -
Add your new CDK construct as a dev dependency:
npm i -D <your-package-name>
, e.g.npm i -D projen-test
.Take care that the versions for all AWS CDK dependencies in
package.json
(e.g. foraws-cdk
or@aws-cdk/core
) are matching the version that you have specified in .projenrc.js undercdkVersion
. Otherwise you'll see some funny compilation errors. If you need to update the dependencies, you can usenpm i -S <dependency>@latest
(for dependencies) ornpm i -D <dependency>@latest
(for dev dependencies). -
Add your new CDK construct in
bin/cdk-npm-test-stack.ts
like:new LambdaConstruct(this, 'NpmSample');
-
Build your code:
npm run build
-
Now you can deploy your app using
cdk deploy
. Wait for the stack to be created and test that everything looks like expected.
-
Initialize new CDK app in Java:
mkdir cdk-java-test cd cdk-java-test cdk init app --language=java
This will initialize a new CDK app project. In
src/main/java/com/myorg
you'll see the filesCdkJavaTestApp
andCdkJavaTestStack
that contain CDK samples. -
Add the CDK construct you've just published to the project's
pom.xml
(adjust the details):<dependency> <groupId>de.sebastianhesse.examples</groupId> <artifactId>projen-test</artifactId> <version>0.1.16</version> </dependency>
Replace the Maven coordinates based on your settings that you've configured with
publishToMaven
in .projenrc.js and the correct version. -
Then go to
CdkJavaTestStack
and make use of your CDK construct. In my case, I have added the following line in the constructor:new LambdaConstruct(this, "JavaSample");
-
Package your code (including running the tests):
mvn package
-
Now you can deploy your app using
cdk deploy
. Wait for the stack to be created and test that everything looks like expected.
TODO
Note: If you are eager to go through this tutorial and explore the steps for Python, feel free to contribute to this README :)
TODO
Note: If you are eager to go through this tutorial and explore the steps for NuGet / C#, feel free to contribute to this README :)
You are amazing! π If you went until this point, you have successfully published your first multi-language CDK construct!
Check out more resources:
- cdkworkshop.com
- CDK Constructs
- awesome-cdk
- cdkpatterns.com
- CDK Construct Catalog
- More CDK Constructs 1
- More CDK Constructs 2
Below are a few topics that I've discovered along the way and would like to explain further.
In order to figure out the MAVEN_STAGING_PROFILE_ID
, follow these steps:
- Login to oss.sonatype.org using your Maven Central credentials.
- On the left, click on
Staging Profiles
- In the main window, make sure you select the tab
Staging Profiles
and select the entry matching your namespaceorg.example
- You'll notice the URL has changed to
https://oss.sonatype.org/#stagingProfiles;12abc3456789
- Use the id after the
;
semicolon as theMAVEN_STAGING_PROFILE_ID