When we run Elasticsearch in production, one of the common issues is imbalance in "shards". There may be one node in the cluster that is out of disk space, while a few nodes with no shards on them. For example, here is a node with all the shards: Node Shards Disk Used Disk % Free Space PESD222 957 329.1 GB 32% 694.2...
Introduction What if upgrading your Kubernetes cluster required no downtime at all? Imagine if you could upgrade your Kubernetes cluster and keep everything running smoothly, with zero downtime. Sounds pretty great, right? A lot of teams worry that upgrading will mean their apps go offline, but with solid planning, it's actually...
It is painfully inefficient to check metrics across a large collection of AWS accounts (development, staging, uat, production, etc.). This is a major time waster, not just a small irritation. In addition to wasting valuable engineering time, you run a much higher risk of missing an alert that could result in a full-blown outage every time...
Introduction Modern applications have distributed systems consisting of multiple services, containers, and infrastructure components. While it improves scalability, security and reliability, it also increases the chances of unexpected failures and downtime. Application testing methods majorly focus on application functionality, but...
Introduction For years, Site Reliability Engineering (SRE) has been built around a simple mission: keep systems reliable at scale. We measure SLOs, manage error budgets, write runbooks, respond to incidents, and automate toil wherever possible. But even with automation, most SRE work remains fundamentally reactive: Alerts wake us...
Introduction For years, Site Reliability Engineering (SRE) has been built around a simple mission: keep systems reliable at scale. We measure SLOs, manage error budgets, write runbooks, respond to incidents, and automate toil wherever possible. But even with automation, most SRE work remains fundamentally reactive: Alerts wake us...
Introduction When you work with AWS infrastructure for some time, you realise that not all problems announce themselves with alerts or outages. Some problems stay quiet, blend into the background, and only reveal themselves later-usually when someone asks a question you can’t answer clearly. This is one such experience from my early...
Introduction When teams start on their DevOps journey, the excitement is real. CI/CD pipelines, faster deployments, cloud-native tools, automation everywhere - it feels like everything is finally going to be smooth. But in reality, the first year of DevOps is rarely smooth. It’s messy, experimental, and full of learning. [caption...
Introduction We used to rely on Pingdom for uptime monitoring. It worked well with simple checks, nice UI, and reliable and clean alerts. But one day, someone on our DevOps team casually said: "Hey, why are we paying for something that only pings URLs?" And that kicked off a big conversation. The Cost Wake-Up Pingdom wasn’t...