{"id":78238,"date":"2026-03-10T11:20:28","date_gmt":"2026-03-10T05:50:28","guid":{"rendered":"https:\/\/www.tothenew.com\/blog\/?p=78238"},"modified":"2026-03-16T15:42:58","modified_gmt":"2026-03-16T10:12:58","slug":"chaos-engineering-simulating-network-latency-using-aws-fis","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/chaos-engineering-simulating-network-latency-using-aws-fis\/","title":{"rendered":"Chaos Engineering: Simulating Network Latency using AWS FIS"},"content":{"rendered":"

Introduction<\/h2>\n

Modern applications have distributed systems consisting of multiple services, containers, and infrastructure components. While it improves scalability, security and reliability, it also increases the chances of unexpected failures and downtime.<\/p>\n

Application testing methods majorly focus on application functionality, but they rarely test how systems behave in real-world failures such as instance crashes, network latency, or service outages in a live production environment on the cloud.<\/p>\n

Chaos Engineering solves this problem by intentionally introducing failures into systems to test their resilience and recovery capabilities before such failures occur in production.<\/p>\n

What is Chaos Engineering?<\/h2>\n

Chaos engineering is the intentional and controlled causing of failures in the production or pre-production environment to understand their impact and plan a better defense posture and incident maintenance strategy.<\/p>\n

Instead of relying on failures to happen by chance, engineers intentionally attempt scenarios such as server crashes, network failures, and other stress tests to see how the system behaves under stress or failure.<\/p>\n

Netflix learned this concept firsthand when it switched from on-premises to the cloud1 -they experienced an outage that led to a three-day interruption to service delivery in 2008.<\/p>\n

Netflix created chaos monkey, an open source tool\u202fthat creates random incidents in IT services and infrastructure meant to identify weaknesses that can be fixed or addressed through automatic recovery procedures. They implemented chaos monkey when it moved from a private data center to Amazon Web Services (AWS) in response to unreliability from the cloud.<\/p>\n

\"chaos1\"

Chaos Engineering<\/p><\/div>\n

<\/h2>\n

Types of chaos engineering experiments<\/h2>\n
    \n
  1. Latency injection<\/strong>:\u202fDevOps teams intentionally create scenarios that emulate a slow or failing network connection. This includes the introduction of network delays or slower response times.<\/li>\n
  2. Load generation:<\/strong>\u202fThis relates to intentionally stressing the system by sending significant traffic levels well beyond normal operations. This helps the site reliability engineers (SREs) or DevOps to understand any bottlenecks in the system.<\/li>\n
  3. Fault injection:\u202f<\/strong>This involves\u00a0 introducing errors into the system to determine how it affects the application and\u00a0 other dependent systems and whether it interrupts services. Examples of fault injections include inducing disk failures, terminating processes, shutting down a host or introducing power or temperature increases.<\/li>\n<\/ol>\n

    Why Chaos Engineering Matters<\/h2>\n
    \"chaosimage1\"

    Chaos Engineering Cycle<\/p><\/div>\n

    Increased System Resilience<\/strong>:\u00a0 Resilience refers to a system\u2019s ability to recover from disruptions and continue functioning at an optimal level despite having failures and ability to recover from failures.Improved Incident Response: By intentionally introducing disruptions in a controlled environment, teams can gain valuable insights into how their systems break under pressure and act on the incidents proactively without delays and get feedback on failures. These failures help engineering teams to identify potential bottlenecks, inefficiencies, or issues in the System.<\/p>\n

    Cultural Shift towards Proactive Reliability<\/strong> : Chaos Engineering flips this script by encouraging proactive reliability. Through the practice of chaos experiments, organizations are not merely responding to failures\u2014they are actively seeking them out, understanding them, and learning from them.This fosters the members to understand and experiment with the systems.<\/p>\n

    Validation of Redundancy and Failover Mechanisms<\/strong>: Through chaos experiments, teams can validate the effectiveness of their redundancy and failover strategies in real-world conditions.<\/p>\n

    For instance, by simulating the failure of a database server, engineers can test whether traffic is properly rerouted to a backup system without degrading performance. Another real-world example can be introducing latency in the application to validate network lags and performance issues.<\/p>\n

    In this blog, we will explore Chaos Engineering on AWS and walk through a practical experiment using AWS Fault Injection Simulator<\/strong><\/p>\n

    What is AWS Fault Injection Simulator (FIS)?<\/strong><\/h2>\n

    AWS Fault Injection Service (AWS FIS) is a managed service that enables you to perform fault injection experiments on your AWS workloads. Fault injection is based on the principles of chaos engineering. These experiments stress an application by creating disruptive events so that you can observe how your application responds. You can then use this information to improve the performance and resiliency of your applications so that they behave as expected.<\/p>\n

    With AWS FIS, you can simulate scenarios such as:<\/p>\n

      \n
    • EC2 instance termination<\/li>\n
    • CPU stress on instances<\/li>\n
    • Network latency injection<\/li>\n
    • EBS I\/O disruption<\/li>\n
    • API throttling<\/li>\n
    • Database load testing<\/li>\n<\/ul>\n

      Simulating Network Latency Using AWS Fault Injection Simulator (FIS).<\/h2>\n

      In this experiment, we use AWS Fault Injection Service to inject artificial network latency into an EC2 instance.<\/p>\n

      \"FIS\"

      FIS<\/p><\/div>\n

      Step 1: Launch an EC2 Instance<\/h2>\n

      Allow ports in Security Group: 22 (SSH), 80 (HTTP)<\/strong><\/p>\n

      \"EC2

      EC2 Instance<\/p><\/div>\n

      Step 2: Install a Web Server<\/h3>\n

      Install Apache to simulate an application service.<\/p>\n

      $ sudo yum update -y<\/strong><\/p>\n

      $ sudo yum install httpd -y<\/strong><\/p>\n

      Start the server:<\/strong><\/p>\n

      $ sudo systemctl start httpd<\/strong><\/p>\n

      $ sudo systemctl enable httpd<\/strong><\/p>\n

      \"starthttpd\"

      starthttpd<\/p><\/div>\n

      Create a simple test page:<\/h3>\n

      $ echo “Chaos Engineering Demo Application” | sudo tee \/var\/www\/html\/index.html<\/strong><\/p>\n

      \"DemoApp\"

      DemoApp<\/p><\/div>\n

      Step 3: Measure Normal Application Response Time<\/h3>\n

      Before introducing failures, measure the application’s normal latency.<\/p>\n

      Command
      \n$ curl -o \/dev\/null -s -w “Response Time: %{time_total}s\\n” http:\/\/54.92.212.143<\/strong><\/p>\n

      \"Initial

      Initial Response<\/p><\/div>\n

      The application responds in approximately 0.83 ms, indicating normal network performance.<\/p>\n

      Step 4: Create an IAM Role for Fault Injection<\/h3>\n

      -> Go to AWS Identity and Access Management<\/p>\n

      -> Click Roles \u2192 Create Role<\/p>\n

      -> Select Fault Injection Service<\/p>\n

      -> Attach policy:
      \nAWSFaultInjectionSimulatorNetworkAccess
      \nAWSFaultInjectionSimulatorEC2Access<\/p>\n

      AmazonSSMManagedInstanceCore<\/p>\n

      Step 5: Create a Fault Injection Experiment Template<\/h3>\n

      Now configure the chaos experiment.<\/p>\n

      -> Open AWS Fault Injection Service<\/p>\n

      -> Click Experiment Templates<\/p>\n

      ->Click Create experiment template<\/p>\n

      Configure the Template<\/strong>
      \nTarget<\/p>\n

      Resource type: EC2 instance<\/p>\n

      -> Select the instance created earlier<\/p>\n

      -> Action<\/p>\n

      Choose:<\/strong><\/p>\n

      aws: AWSFIS-Run-Network-Latency<\/p>\n

      Select the IAM role created earlier.<\/p>\n

      Save the template.<\/p>\n

      \"setupFIS\"

      setupFIS<\/p><\/div>\n

      Step 6: Start the Chaos Experiment<\/h2>\n

      Run the experiment from the template.Experiment Status: Running<\/p>\n

      Target: EC2 Instance<\/p>\n

      Fault Type: Network Latency Injection<\/p>\n

      Latency: 200 ms<\/p>\n

      Duration: 1 minute<\/p>\n

      During this time, AWS injects a 200 ms network delay into the EC2 instance<\/p>\n

      \"runningFIS\"

      runningFIS<\/p><\/div>\n

      Step 7: Test Application During the Experiment<\/h2>\n

      While the experiment is running, test the application again.<\/p>\n

      Command
      \n$ curl -o \/dev\/null -s -w “Response Time: %{time_total}s\\n” http:\/\/54.92.212.143<\/strong><\/p>\n

      \"response\"

      response<\/p><\/div>\n

      Observation<\/strong>
      \nResponse time increased from: 0.80s \u2192 1.78s<\/strong><\/p>\n

      This confirms that the injected network latency is affecting the application.<\/p>\n

      Step 8: Observe Monitoring Metrics<\/h2>\n

      While the experiment is running, open Amazon CloudWatch and observe the instance metrics.<\/p>\n

      \"CloudWatch\"

      CloudWatch<\/p><\/div>\n

      Final Result of the Experiment<\/h2>\n

      This chaos engineering experiment demonstrated that:<\/strong><\/p>\n

        \n
      • AWS Fault Injection Service can simulate network failures safely.<\/li>\n
      • Application response time increases when network latency is injected.<\/li>\n
      • Monitoring tools like CloudWatch detect performance degradation.<\/li>\n
      • The system recovers automatically after the experiment ends.<\/li>\n
      • Such experiments help engineers validate the resilience and observability of cloud applications.<\/li>\n<\/ul>\n

        Conclusion<\/h2>\n

        Chaos engineering plays an important role in making the system more reliable and resilient. The goal of modern infrastructure engineering is not to eliminate failures but to design systems that recover quickly and automatically.<\/p>\n

        Chaos Engineering helps teams proactively identify weaknesses before they impact production environments.<\/p>\n

        With AWS Fault Injection Simulator, organizations can safely run controlled experiments to validate the resilience of their architecture.<\/p>\n

        By regularly testing failure scenarios, teams can build systems that are not only scalable but also truly fault-tolerant and resilient distributed systems.<\/p>\n

        In the world of distributed systems, resilience isn\u2019t proven through design diagrams \u2014 it\u2019s proven through experiments.<\/p>\n","protected":false},"excerpt":{"rendered":"

        Introduction Modern applications have distributed systems consisting of multiple services, containers, and infrastructure components. While it improves scalability, security and reliability, it also increases the chances of unexpected failures and downtime. Application testing methods majorly focus on application functionality, but they rarely test how systems behave in real-world failures such as instance crashes, network latency, […]<\/p>\n","protected":false},"author":1863,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":52},"categories":[5877],"tags":[248,1916,7723],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/78238"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1863"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=78238"}],"version-history":[{"count":5,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/78238\/revisions"}],"predecessor-version":[{"id":78255,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/78238\/revisions\/78255"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=78238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=78238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=78238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}