Pump.co’s Post

Keeping in line with the last post, this is how we reduced our AWS costs (2021-2024). Points are color-coded as per the attached image. 1. Large to small 🟦🟥 Everyone believes they are going to get 50 million users the moment their app hits prod. But I implore you to memorize the following code. If company.isFunded('Series A') { instance.set('Large') } else { instance.set('Small') } Cost me a few lakhs to learn this. Totally worth it. 2. On-demand to reserved 🟦 I have heard serverless scales infinitely. Well, it does scale your costs infinitely. Reserve your instance. Then ask AWS for a discount. 3. Filestore to CloudWatch 🟧 Our system logs were stored for years. Our server's file store was bloated. Now we send them to Cloudwatch, which retains them for three days. Then it's asta-la-vista, baby. 4. ElasticSearch to Postgres 🟪 Why did I even spin up these instances? I thought our discussion forums would need "text search capabilities". Well, since I don't need to do DNA matching on the comments, the cost of ElasticSearch was prohibitively high. Instead, we moved our text search to the browser. Using javascript :D BLASPHEMY! -------------------------------------------- These experiences have helped us appreciate the practical aspect of designing systems (which often comes up during live classes at InterviewReady). If you are a startup founder, I wish you all the best! Cheers! #AWS #CostManagement

𝑰𝒏𝒇𝒓𝒂𝒔𝒕𝒓𝒖𝒄𝒕𝒖𝒓𝒆-𝒂𝒔-𝑪𝒐𝒅𝒆 What exactly does this mean? 🤔 Suppose your team is working on AWS. Developer A created a bucket on S3, and Developer B created an instance on EC2, along with a few other services. Now, you both don’t know which region each of you used. Also, If Developer B created different resources (assuming he has access), how could you know? It’s hard to go through all the resources used to check them or to inform the other developer every time. So, isn’t it great if we write code and then the infrastructure will generate? This is where Infrastructure-as-Code (IaC) comes into play. -------{ 𝑬𝒙𝒕𝒓𝒂: 𝘛𝘩𝘪𝘴 𝘱𝘰𝘴𝘵 𝘪𝘴 𝘢 𝘱𝘢𝘳𝘵 𝘰𝘧 𝘢 𝘴𝘦𝘳𝘪𝘦𝘴 ---> 𝘛𝘦𝘳𝘳𝘢𝘧𝘰𝘳𝘮 }------ But wait, what exactly it is Infrastructure-as-Code (IaC) is just a method Method? For what? Ahmmmmmm, Seems interesting to read, right? Continue reading the full article here, 😊 ------------------------------------------------------------------------------- 𝐋𝐢𝐧𝐤 ---> https://lnkd.in/dZZuP-TA -------------------------------------------------------------------------------

Hi Everyone, Let's start discussing how my 1st day of #100daysofcode went. Well on the first day, I hovered around a few topics like System Design, AWS Cloud, and DSA(How can I neglect that). Let me travel you around all of them. 1. System Design: I went across the fundamentals of System Design and came to know about a few major topics: CAP Theorem: This theorem says that in the case of distributed datastore systems in case of a network failure, it is only possible for a system to guarantee either data consistency or availability. It’s a tradeoff. At all other times, all three can be provided. But, in the event of a network failure, a choice must be made. Latency VS Throughput: Both of them measure the performance of a computer network. Latency refers to the time data takes to transfer across a network whereas Throughput is the average volume of data that can actually pass through the network over a specific time. It indicates the number of data packets that arrive at their destinations successfully and the data packet loss. ACID and BASE Transactions: ACID(Atomicity, Consistency, Isolation, and Durability) is basically a set of properties that are designed to ensure the reliability and consistency of database transactions. This comes in handy especially when the data integrity is a top priority for us but also its a bit harder to scale them because they have additional overhead. On the other hand BASE(Basically Available, Soft state, Eventually consistent) trades off near-term consistency for long-term stability. Although it assumes that all data will eventually become consistent, it cannot guarantee this. This approach is appropriate for high-volume distributed systems because it provides greater scalability and availability. It is often used in conjunction with NoSQL databases. 2. AWS Cloud: This was my grand highlight of the day. After digging around a variety of resources and articles and solving a lot of errors I finally integrated a CI/CD pipeline for one of my experiment projects through #githubactions. It might seem a bit funny to integrate a pipeline just as an experiment but I like doing things this way. Maybe you should try it sometime. 3. DSA: To your surprise, I solved only a few pattern questions which we used to mug up in school time, and tried to emphasize more on logic building rather than doing more and more questions. Yeah, So that’s it for Day 1. Meet you on the 2nd Day Again.  Cheers! 🚀

Check out my new blog post at https://lnkd.in/gEEeh6Zb. In today's world, latency and performance are critical for organisations, and optimising images becomes even more essential for an improved user experience and reduced data usage. My blog post takes a deep dive into how you can create a serverless image optimization pipeline on AWS. You can also access the full code for this repository at https://lnkd.in/gnTSQZWF. #aws #serverless #nodejs #typescript

Here is my latest blog about a #serverless solution to dataset/batch processing on AWS. In the example, transaction data from various locations of a business get uploaded to AWS every day and then need to be processed. One approach to handle this is to use a Lambda functions to pre process the data and then trigger a Step Function to orchestrate the detailed processing of the data. One of the requirements in this case is that the processing of the data has highly variable durations which in many cases will exceed the Lambda execution timeout of 15 minutes. Also we need to be able to scale to very large size when needed without provisioning lots of servers. One more requirement is to update the status and report outputs of the process to people via slack messages. The solution i detail is a serverless approach involving the use of #Fargate containers on the Elastic Compute Service (ECS). I have been learning #Rustlang for a while now so I decided to tackle this project by coding with it! 🦀🦀 All of the source code is included in the repo mentioned in the article.

Get help at any level. Junior developers gain confidence knowing the suggestions from Amazon CodeWhisperer represent MongoDB best practices and syntax while senior developers see performance gains and time savings, freeing them up from boilerplate and mundane code to concentrate on refinement and performance.

This is my first blog post as one of the AWS News Blog authors. I love writing about technology, so here it goes. Check out the last week's exciting announcements! #awsweeklyroundup #amazonbedrock #claude3opus #llama3 https://lnkd.in/dFmw3MGd

Today I read the article "How to Scale an App to 10 Million Users on AWS" at https://lnkd.in/eU4AvQX4 by Neo Kim and I was disappointed by the quality of it compared to the rest of his writing. I wrote a long list of suggestions. I hope they can help someone. Comments here are welcome. As I have many years of experience in small teams either in startups or big companies, I can totally relate to providing a service that has to scale to millions of users when you only have 2 Engineers writing the code and managing the infrastructure. So my biggest point in my notes is: Don't try to install and manage a database on EC2 even with 10s of users if you can instead use a managed solution like RDS or DynamoDB. It's not even worth it from a cost perspective at that scale. Use the saved time to implement nice observability (think OpenTelemetry) and automation (think IaaC with Pulumi) from the very beginning. Scaling from a small scale to millions of users will be much easier. My notes are available at https://lnkd.in/eYgueKAN

Running code after returning a response from an AWS Lambda function https://flip.it/kPvBXZ