Introduction Apache Kafka has become the backbone of modern event-driven architectures, powering everything from micro services communication to real-time analytics and stream processing. As Kafka deployments grow, developers and operators frequently need answers to questions such as: Are messages reaching Kafka successfully? Which partition received a message? Are consumers keeping up with producers? How much […]
Introduction Managing IAM (Identity and Access Management) compliance manually is one of those tasks that sounds simple but quietly consumes hours every week. Someone has to read the daily report, identify non-compliant users, send individual emails, track who responded, follow up again, and eventually rotate keys manually for users who never got around to it. […]
You’d think moving data from one AWS account to another would be straightforward. Export it, import it, done. That’s what the documentation implies, and honestly, that’s what I assumed going in. It wasn’t. What started as a data migration quickly turned into a system reconstruction problem. In DynamoDB, data alone is only half the story. […]
Introduction In early microservices architectures on AWS, communication between services relied on infrastructure components like load balancers. There was no native, built-in way for services to discover each other dynamically. A common architecture looked like this: Public DNS → Amazon Application Load Balancer → Frontend Service → Private ALB → Backend Service While this approach […]
If you’ve ever had to jump between six different AWS accounts just to figure out why one Lambda function is behaving oddly – you already know the pain. Multi-account AWS environments are great for security and governance, but they can turn basic monitoring into a logistical nightmare. The good news? AWS gives you everything you […]
As a developer, I thought that once I pushed the code to the repository, I was done. I thought that things such as deployment, scalability, networking, and monitoring were all handled by the DevOps team. However, this is only true for a short period of time. At some point, when something go wrong in production […]
Patching Azure VMs from AWS Systems Manager using Hybrid Activation Each cloud platform provides its own native tools, which can lead to fragmented processes and increased administrative overhead. To address this challenge, AWS Systems Manager (SSM) offers a powerful solution through its Hybrid Activation feature. This capability allows non-AWS machines, such as Azure Virtual Machines […]
Introduction In a multi-account AWS environment, log management for services such as Cloud Trail, VPC Flow Logs, and WAF is a complex and fragmented process. This is due to the fact that each account has its own log data, which is not easy for security and operations teams to manage centrally. This issue, however, can […]
Introduction We know that many applications generate large amounts of event data such as alerts, application events, logs, and notifications. This data is usually unstructured and arrives in a continuous manner. The initial step in creating a data engineering pipeline is to store this event data into a reliable and long term storage system in […]
