When we run Elasticsearch in production, one of the common issues is imbalance in “shards”. There may be one node in the cluster that is out of disk space, while a few nodes with no shards on them. For example, here is a node with all the shards: Node Shards Disk Used Disk % Free […]
Introduction Elasticsearch has become a go-to choice for building fast, intelligent search experiences. But what if your source of truth is a relational database like MySQL? In this blog, we’ll walk through how we migrated structured relational data into Elasticsearch using Python — with company_registry as a working example. Why Migrate from MySQL to Elasticsearch? […]
Introduction We recently needed visibility into what our Java services were actually doing in production — response times, slow queries, errors, that sort of thing. We landed on Elastic APM. Here’s exactly how we set it up, including some decisions we made around multi-environment support that saved us a lot of headache later. The setup […]
Introduction Elasticsearch is a powerful search engine that’s commonly used for log and data analytics. Setting a multi-node cluster enhances the availability, fault tolerance, and performance of Elasticsearch, making it a preferred choice for production environments. In this blog post, I’ll walk you through the steps to create a multi-node Elasticsearch cluster using Terraform ensuring […]
Introduction In today’s competitive market, delivering a seamless and responsive user experience is crucial for retaining customers and driving business growth. Elastic APM (Application Performance Monitoring) has emerged as a vital tool for achieving these objectives. By providing deep insights into application performance and pinpointing bottlenecks, Elastic APM empowers businesses to optimize their applications, enhance […]
Elasticsearch offers highly useful plugin mechanism as a standard way for extending its core functionality such as custom analyzer, native scripts and more. While some plugins may contain static content which is served through its HTTP server, some others offer a graphical front-end for selected parts of the Elasticsearch REST API such as monitoring, managing cluster and […]
One of the most challenging tasks in any microservices ecosystem is the centralized log management, and there are many open source and paid solutions available in the market. In our ecosystem, we are using ELK stack as it provides scalability and the multitenant-capable full-text search engine that easily integrates with Logstash and Kibana for centralized […]
In our previous blog, we have covered the basics of fluentd, the lifecycle of fluentd events and the primary directives involved. In this blog, we’ll configure fluentd to dump tomcat logs to Elasticsearch. We’ll also talk about filter directive/plugin and how to configure it to add hostname field in the event stream. Fluentd is a log collector that works on […]
With the recently launched AWS Mumbai region, many of us might need to migrate our infrastructure there. In one such scenario, we have to setup Elasticsearch cluster on AWS EC2 instances and test it in Mumbai region. While migrating Elasticsearch cluster from AWS Singapore region to Aws Mumbai region, we faced few challenges, one […]
