Shoreline.io (Acquired by NVIDIA)

"Shoreline.io is helping us proactively eliminate issues before they lead to downtime. Self-healing repairs are also freeing my team so they can work on new features instead of fixing the same things over and over." - Knowde

We are pleased to confirm that Shoreline.io will be Bronze sponsors at the AI and Big Data Expo. Shoreline.io is a cloud reliability company focused on real-time automation and control for cloud operations teams. Shoreline enables real-time debugging and automated remediation across your fleet, significantly improving availability by reducing mean time to resolution. Shoreline.io’s CEO & Founder Anurag Gupta will present ‘DevOps is Broken’ at 3.20pm on Day 2 of the Expo. In this talk, he will describe how generative AI can effectively help teams that are struggling under the weight of over-reliance on operations consoles, manual interventions that take too long and phishing attack vulnerabilities. Register to join the expo now: https://lnkd.in/ducmhs6h

Dive into the complexities of on-call operations. We've gathered expert insights to shed light on the challenges and costs that teams face, and offer transformative strategies for improvement. How do you currently navigate the challenges of on-call operations in your organization?

The state-of-the-art way to manage access to your production databases nowadays is to give people real-time credentials. But that doesn't go far enough for database security. With real-time credentials, an operator can only log in with the credential when they need to make a change to a database, either to query it, change it, figure out what's going wrong, etc. But it makes no sense for database security. Let me tell you why. If you're giving somebody SYSDBA (if you're on Oracle or equivalent), they can do whatever they want. They can make mistakes, or perhaps they are a bad actor. What you need to do instead is: Control the capabilities that you allow people to do when they access that production database. There are only 10-15 things they need to do: - Check to see what queries are running long. - Check to see whether there's a deadlock through locking. - Check to see whether the query plan has changed for a long-running query, etc. These are not the biggest things in the world. So figure them out and give people access to just those things. Have you ever seen someone in your production ops team accidentally type “<” instead of “>” when doing the DELETE statement? I have. It doesn't make sense to allow these things to be done. You wouldn't let people do that for your applications; you have parameterized queries rather than freeform queries. Similarly, you shouldn't allow it for ad hoc access from your operators. It's the same problem. And real-time credentials do not fix that problem. Real-time credentials just mean that people have some other tool that gets in the way when they have to access production. You can do a lot better. That's my view anyway. Let me know in the comments if you agree or disagree! #cloud #cybersecurity #devops

Our most recent blog takes a look at how to manage zombie processes in containers. Learn how poor container management can lead to orphan processes, draining resources and risking operational integrity.

The biggest security vulnerability for an enterprise is its people. It doesn't matter how good an alarm system might be at my house or how many locks I have on the door if my wife loses her keys at the coffee shop. The challenge is that the Blackhat security guys innovate faster than anyone else. Gone are the days when phishing attacks were easily spotted due to poor grammar or broad targeting. Now, these attacks are hyper-personalized to the individual, using perfect grammar and relevant details that can easily fool even the most vigilant among us. Imagine I am at AWS Reinvent. And an attacker sends a text to one of my employees saying, "Hey, this is Anurag. I need you to buy 25 Starbucks cards because I'm sitting at AWS Reinvent and ran out of gifts. Please just get them and send me a link at this address." Natural language processing and GenAI can be used to mimic my language style and send this request  to the right person in the company, making it look far more legitimate. It’s difficult to address this because it’s not being fronted by AI – a human is reading the messages. That's one of the security risks enterprises may face from AI. What are you doing to prevent such attacks at your company? #cybersecurity #phishingattacks #ai

Let's talk about service graph debugging. Many people have moved to Kubernetes and microservices, which makes sense. It allows them to innovate faster, rapidly deploying smaller chunks of independent functionality with stable interfaces to other components. However, it makes production operations stupidly harder. You get spikes in latency at the top of your service. Examples: At one of my customers, a large gaming company, they've got 50 services underneath. This means 50-100 people on a call could be pointing fingers at each other, with valid reasons to state: “It’s not me. My thing is fine.”  A half-hour in wasted time. It's even harder for Razorpay, a large payment provider we work with. The time taken to diagnose leads to  a backlog of payments that are not getting processed or accepted. This is a situation that has to be manually rectified for every minute of outage. So what do you do? This is where service graph debugging comes in. A lot of observability tools have something in this area. They look at elements of the service graph and figure out what's going on. But the challenge is that it isn't deep. They look at metrics or logs but don't diagnose deeply to figure out where or why something's wrong. This is where Shoreline comes into play. You can build a runbook that checks each component for issues and integrate it into systems like PagerDuty or Opsgenie before paging anyone. Then, you only page the people involved with the specific microservice having the issue. Generally speaking, 90% of the time in an incident is spent on diagnostics, not repair. The repairs tend to be straightforward. Razorpay, for instance, went from hour-long calls with 50 people to maybe a five-minute call for diagnosis. Of course, there's no magic to it. If one of their banks is down, that still needs to be addressed. But at least they're not spending all the time just finding out what’s wrong. That’s why you need to automate the deep diagnostics that your developers or SREs would have done. Because they're doing that work again, and again, and again. And it's completely safe to have a machine do it. #SRE #devops #cloud

Let's talk about service graph debugging. Many people have moved to Kubernetes and microservices, which makes sense. It allows them to innovate faster, rapidly deploying smaller chunks of independent functionality with stable interfaces to other components. However, it makes production operations stupidly harder. You get spikes in latency at the top of your service. Examples: At one of my customers, a large gaming company, they've got 50 services underneath. This means 50-100 people on a call could be pointing fingers at each other, with valid reasons to state: “It’s not me. My thing is fine.”  A half-hour in wasted time. It's even harder for Razorpay, a large payment provider we work with. The time taken to diagnose leads to  a backlog of payments that are not getting processed or accepted. This is a situation that has to be manually rectified for every minute of outage. So what do you do? This is where service graph debugging comes in. A lot of observability tools have something in this area. They look at elements of the service graph and figure out what's going on. But the challenge is that it isn't deep. They look at metrics or logs but don't diagnose deeply to figure out where or why something's wrong. This is where @Shoreline comes into play. You can build a runbook that checks each component for issues and integrate it into systems like PagerDuty or Opsgenie before paging anyone. Then, you only page the people involved with the specific microservice having the issue. Generally speaking, 90% of the time in an incident is spent on diagnostics, not repair. The repairs tend to be straightforward. Razorpay, for instance, went from hour-long calls with 50 people to maybe a five-minute call for diagnosis. Of course, there's no magic to it. If one of their banks is down, that still needs to be addressed. But at least they're not spending all the time just finding out what’s wrong. That’s why you need to automate the deep diagnostics that your developers or SREs would have done. Because they're doing that work again, and again, and again. And it's completely safe to have a machine do it. #SRE #devops #cloud

Anurag Gupta (and his company Shoreline.io ) are likely some of the industry’s foremost experts on keeping your fleet up and running - have a good listen!

Observability tools rank as the 3rd-highest cost for engineering teams after their people and cloud infrastructure. That’s insane! What's even more insane? You hardly ever use the collected data. Learn more on our blog: