Skip to content

Commit

Permalink
Merge pull request #160 from marcospereira/build/use-docker-for-kafka…
Browse files Browse the repository at this point in the history 
…-server

Use external Kafka server for shopping cart samples
  • Loading branch information
@mergify
mergify[bot] authored Feb 27, 2020
2 parents 4601ef1 ca128e6 commit 627939b
Show file tree
Hide file tree
Showing 9 changed files with 140 additions and 73 deletions.
22 changes: 12 additions & 10 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 22,6 @@ before_install:

install: jabba install $(jabba ls-remote "adopt@~1.$TRAVIS_JDK.0-0" --latest=patch) && jabba use "$_" && java -Xmx32m -version

services:
# shopping cart is using postgres for testing
- postgresql

cache:
directories:
- "$HOME/.cache/coursier"
Expand Down Expand Up @@ -56,15 52,18 @@ jobs:
# Shopping-Cart Samples
- name: "Run tests Shopping Cart example (java/sbt)"
script:
- shopping-cart/travis-scripts/create-database.sh
- docker-compose -f shopping-cart/shopping-cart-java/docker-compose.yml up -d
- sleep 30s
- cd shopping-cart/shopping-cart-java && sbt test
- name: "Run tests Shopping Cart example (java/mvn)"
script:
- shopping-cart/travis-scripts/create-database.sh
- docker-compose -f shopping-cart/shopping-cart-java/docker-compose.yml up -d
- sleep 30s
- cd shopping-cart/shopping-cart-java && mvn test
- name: "Run tests Shopping Cart example (scala/sbt)"
script:
- shopping-cart/travis-scripts/create-database.sh
- docker-compose -f shopping-cart/shopping-cart-scala/docker-compose.yml up -d
- sleep 30s
- cd shopping-cart/shopping-cart-scala && sbt test

# Couchbase Persistence Samples
Expand Down Expand Up @@ -103,17 102,20 @@ jobs:
- name: "Run tests Shopping Cart example (java/sbt)"
env: TRAVIS_JDK=8
script:
- shopping-cart/travis-scripts/create-database.sh
- docker-compose -f shopping-cart/shopping-cart-java/docker-compose.yml up -d
- sleep 30s
- cd shopping-cart/shopping-cart-java && sbt test
- name: "Run tests Shopping Cart example (java/mvn)"
env: TRAVIS_JDK=8
script:
- shopping-cart/travis-scripts/create-database.sh
- docker-compose -f shopping-cart/shopping-cart-java/docker-compose.yml up -d
- sleep 30s
- cd shopping-cart/shopping-cart-java && mvn test
- name: "Run tests Shopping Cart example (scala/sbt)"
env: TRAVIS_JDK=8
script:
- shopping-cart/travis-scripts/create-database.sh
- docker-compose -f shopping-cart/shopping-cart-scala/docker-compose.yml up -d
- sleep 30s
- cd shopping-cart/shopping-cart-scala && sbt test

# Couchbase Persistence Samples
Expand Down
61 changes: 38 additions & 23 deletions shopping-cart/shopping-cart-java/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,46 1,61 @@
# Shopping Cart

This sample application demonstrates a simple shopping cart built with Lagom. It contains two services, a shopping cart service, for managing shopping carts, and an inventory service, for tracking inventory.
This sample application demonstrates a simple shopping cart built with Lagom. It contains two services, a shopping cart service, for managing shopping carts, and a inventory service, for tracking inventory.

The shopping cart service persists its data to a relational database using [Akka Persistence Typed](https://doc.akka.io/docs/akka/2.6/typed/persistence.html) API, and is intended to demonstrate how to persist state using Lagom.
The **shopping cart** service persists its data to a relational database using Lagom's persistence API and demonstrates how to persist state using Lagom.

The inventory service consumes a stream of events published to Kafka by the shopping cart service, and is intended to demonstrate how Kafka event streams can be consumed in Lagom. However, it doesn't persist its state to a database, it just stores it in memory, and this memory is not shared across nodes. Hence, it should not be used as an example of how to store state in Lagom.
The **inventory service** consumes a stream of events published to Kafka by the shopping cart service, and demonstrates how to consume Kafka event streams in Lagom. However, it doesn't persist its state to a database, it just stores it in memory, and this memory is not shared across nodes. Hence, it should not be used as an example of how to persist state in Lagom.

## Setup
## Requirements

To run this application locally you will need access to a Postgres database. We suggest you run it on a docker container but a local or remote native instance will also work.
This sample requires Kafka and Postgres as external services. They are pre-configured in `docker-compose.yml` file. Meaning that before running the service, you first need to start the services using the following command:

We provide a `docker-compose.yml` file that you can use to run a Postgres database already configured for this application. The docker container will be exposed on port `5432`.

To create the image and start the container, run the command below at the root of this project.

```bash
```bash
cd shopping-cart/shopping-cart-java
docker-compose up -d
```

If you prefer to run Postgres natively on your machine, you need to create the database, the user and password yourself. The application expects it to be running on `localhost` on the default port (`5432`), and it expects there to be a database called `shopping_cart`, with a user called `shopping_cart` with password `shopping_cart` that has full access to it. This can be created using the following SQL:
Postgres will be available on port `5432` and Kafka on port `9092`.

Of course, local or remote instances for both services will also work. See more details below.

### Postgres configuration

This configuration is only necessary if you are not using the instance provided by `docker-compose.yml`.

First, you need to create the database, the user, and the password yourself. The application expects it to be running on `localhost` on the default port (`5432`), and it expects there to be a database called `shopping_cart`, with a user called `shopping_cart` with password `shopping_cart` that has full access to it. This can be created using the following SQL:

```sql
CREATE DATABASE shopping_cart;
CREATE USER shopping_cart WITH PASSWORD 'shopping_cart';
GRANT ALL PRIVILEGES ON DATABASE shopping_cart TO shopping_cart;
```

Once Postgres is setup, you can start the system by running:
### Kafka Server

This configuration is only necessary if you are not using the instance provided by `docker-compose.yml`.

You need to configure the application to connect to an external Kafka server. Follow [Lagom documentation to see how to do that](https://www.lagomframework.com/documentation/latest/java/KafkaServer.html#Connecting-to-an-external-Kafka-server).

> **Note**: Some of those configurations are already there, so you only need to update them to your correct values.
## Running in dev mode

After setting up all the requirements, to run the application in dev mode, execute the following command:

```bash
sbt runAll
mvn lagom:runAll
```

or, if you prefer to use Maven:
Or, if you are using sbt:

```bash
mvn lagom:runAll
sbt runAll
```

## Shopping cart service

The shopping cart service offers the following REST endpoints:
The shopping cart service offers four REST endpoints:

* Get the current contents of the shopping cart:

Expand Down Expand Up @@ -72,15 87,15 @@ curl -X DELETE http://localhost:9500/shoppingcart/123/item/456
curl -H "Content-Type: application/json" -X PATCH -d '{"quantity": 2}' http://localhost:9500/shoppingcart/123/item/456
```

* Checkout the shopping cart (ie, complete the transaction)
* Check out the shopping cart (i.e., complete the transaction)

```bash
curl -X POST http://localhost:9500/shoppingcart/123/checkout
```

For simplicity, no authentication is implemented, shopping cart IDs are arbitrary and whoever makes the request can use whatever ID they want, and shopping cart item IDs are also arbitrary and trusted. An a real world application, the shopping cart IDs would likely be random UUIDs to ensure uniqueness, and item IDs would be validated against a product database.
For simplicity, no authentication is implemented, shopping cart IDs are arbitrary and whoever makes the request can use whatever ID they want, and item IDs are also arbitrary and trusted. In a real world application, the shopping cart IDs would likely be random UUIDs to ensure uniqueness, and item IDs would be validated against an item database.

When the shopping cart is checked out, an event is published to the Kafka called `shopping-cart` by the shopping cart service, such events look like this:
When the shopping cart is checked out, an event is published to the Kafka topic called `shopping-cart` by the shopping cart service. Such events look like this:

```json
{
Expand All @@ -95,18 110,18 @@ When the shopping cart is checked out, an event is published to the Kafka called

## Inventory service

The inventory service offers the following REST endpoints:
The inventory service offers two REST endpoints:

* Get the inventory of an item:

```
```bash
curl http://localhost:9500/inventory/456
```

* Add to the inventory of an item:

```
```bash
curl -H "Content-Type: application/json" -d 4 -X POST http://localhost:9500/inventory/456
```

The inventory service consumes the `shopping-cart` topic from Kafka, and decrements the inventory according to the events.
The inventory service consumes the `shopping-cart` topic from Kafka and decrements the inventory according to the events.
5 changes: 5 additions & 0 deletions shopping-cart/shopping-cart-java/build.sbt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -104,4 104,9 @@ def getDockerBaseImage(): String = sys.props.get("java.version") match {
case _ => "adoptopenjdk/openjdk8"
}

// The project uses PostgreSQL
lagomCassandraEnabled in ThisBuild := false

// Use Kafka server running in a docker container
lagomKafkaEnabled in ThisBuild := false
lagomKafkaPort in ThisBuild := 9092
33 changes: 24 additions & 9 deletions shopping-cart/shopping-cart-java/docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,9 1,24 @@
postgres:
image: postgres:latest
container_name: shopping_cart_postgres
environment:
- "TZ=Europe/Amsterdam"
- "POSTGRES_USER=shopping_cart"
- "POSTGRES_PASSWORD=shopping_cart"
ports:
- "5432:5432" # credentials (shopping_cart:shopping_cart)
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
container_name: shopping_cart_zookeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:2.12-2.1.1
container_name: shopping_cart_kafka
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: localhost
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
postgres:
image: postgres:latest
container_name: shopping_cart_postgres
environment:
- "TZ=Europe/Amsterdam"
- "POSTGRES_USER=shopping_cart"
- "POSTGRES_PASSWORD=shopping_cart"
ports:
- "5432:5432" # credentials (shopping_cart:shopping_cart)
3 changes: 2 additions & 1 deletion shopping-cart/shopping-cart-java/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 37,6 @@
</profile>
</profiles>


<build>
<pluginManagement>
<plugins>
Expand Down Expand Up @@ -65,6 64,8 @@
<version>${lagom.version}</version>
<configuration>
<cassandraEnabled>false</cassandraEnabled>
<kafkaPort>10000</kafkaPort>
<kafkaEnabled>false</kafkaEnabled>
</configuration>
</plugin>
<plugin>
Expand Down
45 changes: 30 additions & 15 deletions shopping-cart/shopping-cart-scala/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,32 1,47 @@
# Shopping Cart

This sample application demonstrates a simple shopping cart built with Lagom. It contains two services, a shopping cart service, for managing shopping carts, and an inventory service, for tracking inventory.
This sample application demonstrates a simple shopping cart built with Lagom. It contains two services, a shopping cart service, for managing shopping carts, and a inventory service, for tracking inventory.

The shopping cart service persists its data to a relational database using Lagom's persistence API, and is intended to demonstrate how to persist state using Lagom.
The **shopping cart** service persists its data to a relational database using Lagom's persistence API and demonstrates how to persist state using Lagom.

The inventory service consumes a stream of events published to Kafka by the shopping cart service, and is intended to demonstrate how Kafka event streams can be consumed in Lagom. However, it doesn't persist its state to a database, it just stores it in memory, and this memory is not shared across nodes. Hence, it should not be used as an example of how to store state in Lagom.
The **inventory service** consumes a stream of events published to Kafka by the shopping cart service, and demonstrates how to consume Kafka event streams in Lagom. However, it doesn't persist its state to a database, it just stores it in memory, and this memory is not shared across nodes. Hence, it should not be used as an example of how to persist state in Lagom.

## Setup
## Requirements

To run this application locally you will need access to a Postgres database. We suggest you run it on a docker container but a local or remote native instance will also work.

We provide a `docker-compose.yml` file that you can use to run a Postgres database already configured for this application. The docker container will be exposed on port 5432.

To create the image and start the container, run the command below at the root of this project.
This sample requires Kafka and Postgres as external services. They are pre-configured in `docker-compose.yml` file. Meaning that before running the service, you first need to start the services using the following command:

```bash
cd shopping-cart/shopping-cart-scala
docker-compose up -d
```

If you prefer to run Postgres natively on your machine, you need to create the database, the user and password yourself. The application expects it to be running on localhost on the default port (5432), and it expects there to be a database called `shopping_cart`, with a user called `shopping_cart` with password `shopping_cart` that has full access to it. This can be created using the following SQL:
Postgres will be available on port `5432` and Kafka on port `9092`.

Of course, local or remote instances for both services will also work. See more details below.

### Postgres configuration

This configuration is only necessary if you are not using the instance provided by `docker-compose.yml`.

First, you need to create the database, the user, and the password yourself. The application expects it to be running on `localhost` on the default port (`5432`), and it expects there to be a database called `shopping_cart`, with a user called `shopping_cart` with password `shopping_cart` that has full access to it. This can be created using the following SQL:

```sql
CREATE DATABASE shopping_cart;
CREATE USER shopping_cart WITH PASSWORD 'shopping_cart';
GRANT ALL PRIVILEGES ON DATABASE shopping_cart TO shopping_cart;
```

Once Postgres is setup, you can start the system by running:
### Kafka Server

This configuration is only necessary if you are not using the instance provided by `docker-compose.yml`.

You need to configure the application to connect to an external Kafka server. Follow [Lagom documentation to see how to do that](https://www.lagomframework.com/documentation/latest/scala/KafkaServer.html#Connecting-to-an-external-Kafka-server).

> **Note**: Some of those configurations are already there, so you only need to update them to your correct values.
## Running in dev mode

After setting up all the requirements, to run the application in dev mode, execute the following command:

```bash
sbt runAll
Expand Down Expand Up @@ -66,15 81,15 @@ curl -X DELETE http://localhost:9500/shoppingcart/123/item/456
curl -H "Content-Type: application/json" -X PATCH -d '{"quantity": 2}' http://localhost:9500/shoppingcart/123/item/456
```

* Checkout the shopping cart (ie, complete the transaction)
* Check out the shopping cart (i.e., complete the transaction)

```bash
curl -X POST http://localhost:9500/shoppingcart/123/checkout
```

For simplicity, no authentication is implemented, shopping cart IDs are arbitrary and whoever makes the request can use whatever ID they want, and item IDs are also arbitrary and trusted. An a real world application, the shopping cart IDs would likely be random UUIDs to ensure uniqueness, and item IDs would be validated against a item database.
For simplicity, no authentication is implemented, shopping cart IDs are arbitrary and whoever makes the request can use whatever ID they want, and item IDs are also arbitrary and trusted. In a real world application, the shopping cart IDs would likely be random UUIDs to ensure uniqueness, and item IDs would be validated against an item database.

When the shopping cart is checked out, an event is published to the Kafka topic called `shopping-cart` by the shopping cart service, such events look like this:
When the shopping cart is checked out, an event is published to the Kafka topic called `shopping-cart` by the shopping cart service. Such events look like this:

```json
{
Expand Down Expand Up @@ -103,4 118,4 @@ curl http://localhost:9500/inventory/456
curl -H "Content-Type: application/json" -d 4 -X POST http://localhost:9500/inventory/456
```

The inventory service consumes the `shopping-cart` topic from Kafka, and decrements the inventory according to the events.
The inventory service consumes the `shopping-cart` topic from Kafka and decrements the inventory according to the events.
5 changes: 5 additions & 0 deletions shopping-cart/shopping-cart-scala/build.sbt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 73,9 @@ lazy val inventory = (project in file("inventory"))
.settings(dockerSettings)
.dependsOn(`inventory-api`, `shopping-cart-api`)

// The project uses PostgreSQL
lagomCassandraEnabled in ThisBuild := false

// Use Kafka server running in a docker container
lagomKafkaEnabled in ThisBuild := false
lagomKafkaPort in ThisBuild := 9092
33 changes: 24 additions & 9 deletions shopping-cart/shopping-cart-scala/docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,9 1,24 @@
postgres:
image: postgres:latest
container_name: shopping_cart_postgres
environment:
- "TZ=Europe/Amsterdam"
- "POSTGRES_USER=shopping_cart"
- "POSTGRES_PASSWORD=shopping_cart"
ports:
- "5432:5432" # credentials (shopping_cart:shopping_cart)
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
container_name: shopping_cart_zookeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:2.12-2.1.1
container_name: shopping_cart_kafka
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: localhost
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
postgres:
image: postgres:latest
container_name: shopping_cart_postgres
environment:
- "TZ=Europe/Amsterdam"
- "POSTGRES_USER=shopping_cart"
- "POSTGRES_PASSWORD=shopping_cart"
ports:
- "5432:5432" # credentials (shopping_cart:shopping_cart)
6 changes: 0 additions & 6 deletions shopping-cart/travis-scripts/create-database.sh
View file

This file was deleted.

0 comments on commit 627939b

Please sign in to comment.