The merging of two companies (Company A and B) gives the raise to the need for the merge of their Sales systems. After a meeting between the IT departments' manager, they decided to continue the IT operations using the Sales system of Company B and discontinue the Sales system of Company A. Because the complete merge will take some time to be finalized, the managers decide that in the meantime the daily sales data generated by the Company A’s Sales system and its summary should be transferred to Company B. Hence, every day the Company A’s Sales system should export a comma separated values (CSV) file containing all sales carried out in that day. This file would then be summarized and both the raw data and summarized data should be sent to Company B for further action.

Because this is a temporary operation, the managers decided to use the Amazon AWS infrastructure to perform the data processing and transferring from company A to B.

Your task is to develop three applications: Client A, Worker, and Client B.

The Client A is responsible for

1. reading the CSV file

2. upload it into the cloud

3. add a message in the queue with the name of the file

4. notify the Worker signaling that there is a file ready to be processed

The Worker is responsible for

1. wait for a notification message from the Client A

2. read the message in the queue and retrieve the name of the file to process

3. read the file

4. calculate (a) the Total Number of Sales, (b) the Total Amount Sold and (c) the Average Sold per product

5. write the summary into a file in the cloud

6. add a message with the name of the original file and the summary file in another queue

7. notify Client B there are files ready to be processed

The Client B is responsible for

1. wait for a notification message from the Worker

2. read the message in the queue and retrieve the name of the files

3. download the files

All 4 Apps run on Java 11 and Maven.

• repeat-project contains the EC2 worker that is uploaded to AWS
• repeat-project-client-A contains the code the send the file and notify our AWS structure
• repeat-project-client-B contains the code to receive the file and download it
• repeat-project-lambda contains the code for the lambda function. The message with which you trigger the lambda function does not matter, it always executes the same instructions.

Chronological functioning of the app: (before using : upload lambda function and ec2 worker to aws) the EC2 worker will then build the Queues and topics and S3 bucket that we will use.

Then you can run the app :

• Client A uploads the file to a S3, sends a message to the SQS entry queue with the informations of the file and bucket and notifies the SNS topic Entry
• The SNS topic Entry triggers the lambda function which retrieves the csv file in the bucket, processes it and upload a new json file with the processed data to the S3 bucket. The lambda function then sends a message to the SQS Exit queue with the informations of the original and the processed file.
• Client B is waiting for a message in the SQS Exit queue ( request every 5 seconds ). When the message is available, Client B downloads both files locally.

Presentation video: https://www.youtube.com/watch?v=Q8ZHVOLAfsM