![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-15-23-35-38.png\")
<\/p>\n<\/p>\n
Imagine your coffee shop is thriving, but one location can’t keep up with the growing demand. To fix this, you open more shops across the city, easing the load.<\/p>\n
Sharding<\/strong><\/span> works similarly for databases. When one MongoDB<\/a> server can’t handle the volume, data<\/a> is distributed across multiple machines (shards), making your system more efficient.<\/p>\n In this blog, we\u2019ll walk through what and how to implement Sharding in MongoDB and access it using a Spring Boot application. The goal is to provide a beginner-friendly, hands-on experience with sharding in MongoDB, all while creating a scalable system. We’ll also walk through the steps of setting up sharding with MongoDB Community Edition on Linux.<\/p>\n MongoDB uses sharding to distribute large datasets across multiple servers, called shards<\/strong>. This ensures that the database can grow as your data grows, keeping performance high without hitting bottlenecks. Each shard holds a portion of the entire data collection, so queries and operations can be split across different shards for faster results.<\/p>\n When your database grows beyond the capacity of a single server, things get slow. That\u2019s where sharding<\/strong> comes in to:<\/p>\n Sharded Cluster Architecture<\/p><\/div>\n Now, let\u2019s talk about the types of sharding and how MongoDB decides where to put your data.<\/p>\n In range-based sharding, MongoDB divides data based on a shard key value, splitting it into ranges (like dividing numbers from 1-100 into smaller chunks). Each shard is responsible for a particular range of data. In hash-based sharding, MongoDB<\/strong> applies a hash function to the shard key to ensure an even distribution of data. The data is then randomly spread across all shards based on this hash value.<\/p>\n How It Works:<\/strong> Zone-based sharding is a combination of the two methods mentioned above but adds the flexibility of defining zones for specific ranges of data. You can \u201ctag\u201d certain shards to handle specific data ranges based on the shard key, which is useful for geographical distribution or regulatory compliance.<\/p>\n How It Works:<\/strong> Choosing a sharding method depends on how you plan to query the data:<\/p>\n Prerequisites:<\/strong> Now, let’s walk through the step-by-step process of configuring MongoDB sharding on your local machine.<\/p>\n Step 1: Set Up a Config Server<\/strong><\/p>\n In a MongoDB sharded cluster, the config server stores metadata and routing information. To start a config server, we need to run :<\/p>\n Once the config server is running, connect to it:<\/p>\n Each shard is an individual MongoDB instance or replica set. For simplicity, let\u2019s create two shards:<\/p>\n Now, initiate the replica sets for both shards:<\/p>\n The mongos process acts as the query router for the sharded cluster, directing operations to the correct shard.<\/p>\n Step 4: Add Shards to the Cluster<\/strong><\/p>\n Now, connect to the mongos instance and add both shards to the cluster:<\/p>\n Now, How Sharding is Applied to Collections?<\/strong> Shard the collection:<\/strong> \u00a0We\u2019ll be using range sharding<\/span>, which divides the data into continuous value ranges based on the shard key. This allows efficient querying of specific data ranges.<\/p>\n The collection coffeeShop.orders is now sharded based on the customerId field.<\/p><\/div>\n Important<\/span>: Once the collection is sharded, MongoDB will distribute the data into chunks across different shards based on the specified shard key.<\/strong><\/p>\n Split the Data at customerId: 100<\/em><\/span><\/strong><\/span> Move Chunks to the Appropriate Shards<\/strong><\/p>\n Now, move the chunks based on customerId: whereas customerId values greater than 100, move to shard2ReplSet.<\/strong><\/p>\n Split at customerId<\/em><\/span>: 100 creates two logical ranges in the orders collection. It is not directly supported. Once a collection has been sharded using a specific key, it cannot be changed. If you need to change the shard key, you have a few options:<\/p>\n Resharding (since MongoDB 4.4): Manual Approach<\/em>:<\/span> For a comprehensive list of all commands and steps involved in setting up sharding, please refer to the README<\/a> file.<\/p>\n meme<\/p><\/div>\n Now that we have MongoDB sharding set up, let’s build a Spring Boot application to interact with this sharded setup.<\/p>\n Step 1: Create a Spring Boot Project<\/strong><\/p>\n First, create a new Spring Boot project with the necessary dependencies.<\/p>\n Step 2: MongoDB Configuration in application.properties<\/strong><\/p>\n Now, configure your Spring Boot application to connect to the Mongos router:<\/p>\n Step 3: Create MongoDB Entity<\/strong><\/p>\n Create an entity class to represent orders in your database:<\/p>\n Step 4: Create a Repository<\/strong><\/p>\n Define a repository to handle CRUD operations for Order:<\/p>\n Step 5: Service Layer<\/strong><\/p>\n Now, let\u2019s create a service that interacts with the repository:<\/p>\n Step 6: REST Controller<\/strong><\/p>\n Lastly, expose the service through a REST API:<\/p>\n Testing the Sharded Application<\/strong><\/p>\n Start your Spring Boot application and test it using hitting post API on Postman<\/span>.<\/p>\n postman<\/p><\/div>\n MongoDB<\/span> will automatically route your data to the appropriate shard based on the customerId. data distribution<\/p><\/div>\n With sharding<\/strong>, you\u2019re essentially splitting your coffee shop into multiple locations. Each MongoDB<\/span> shard handles part of the data, and your application seamlessly interacts with the entire dataset. Whether you’re dealing with a small application or planning for large-scale growth, sharding ensures MongoDB<\/span> can scale with your needs.<\/p>\n By setting up a sharded MongoDB cluster and connecting it to a Spring Boot application, you’ve taken the first steps toward building scalable, distributed systems.<\/p>\n To access the project repository, visit here<\/a>. TO THE NEW offers end-to-end Data & Analytics services<\/a> to help our customers leverage the power of Data and unlock actionable business insights. Click here<\/a> to start your project with us.<\/p>\n Happy Coding! \ud83d\ude42<\/span><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":" Imagine your coffee shop is thriving, but one location can’t keep up with the growing demand. To fix this, you open more shops across the city, easing the load. Sharding works similarly for databases. When one MongoDB server can’t handle the volume, data is distributed across multiple machines (shards), making your system more efficient. In […]<\/p>\n","protected":false},"author":1889,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":115},"categories":[446],"tags":[4843,4846,6451,2072],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/65419"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1889"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=65419"}],"version-history":[{"count":30,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/65419\/revisions"}],"predecessor-version":[{"id":67898,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/65419\/revisions\/67898"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=65419"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=65419"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=65419"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Read More: Should You Choose AWS DynamoDB for Your Next Project?<\/a><\/h4>\n
What is Sharding in MongoDB?<\/h3>\n
Why Sharding Matters?<\/h4>\n
\n
Key Terminologies Before Diving In<\/h4>\n
\n
![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-15-23-19-23-1.png\")
Types of Sharding in MongoDB<\/h3>\n
\n<\/strong><\/span>1. Range-Based Sharding<\/h4>\n
\n<\/em>
\nHow It Works:<\/strong>
\nIf your shard key is a date field, for example, MongoDB could split data into shards like:
\nShard 1: 2010\u20132015
\nShard 2: 2016\u2013present<\/p>\n2. Hash-Based Sharding<\/h4>\n
\nLet\u2019s say your shard key is a customer ID. MongoDB would hash these IDs, ensuring they are distributed evenly across all shards, regardless of the ID value.<\/p>\n3. Zone-Based Sharding (or Tag-Aware Sharding)<\/h4>\n
\nYou might set up zones like:
\nShard 1: Data for customers from the India.
\nShard 2: Data for customers from U.S.<\/p>\nHow to Choose the Right Sharding Strategy?<\/span><\/h4>\n
\n
Setting Up MongoDB Sharding Locally<\/h4>\n
\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 MongoDB Community Edition installed on Linux
\nSpring Boot and Java are set up locally<\/span><\/p>\n![\"directory](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-08-19-14-55.png\")
<\/p>\n![\"terminal\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-08-19-17-42-1.png\")
<\/p>\n
\nStep 2: Set Up Shards<\/strong><\/p>\n![\"cmd\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-08-19-38-24.png\")
<\/p>\n![\"cmd\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-08-19-40-15.png\")
<\/p>\n
\nStep 3: Start the Mongos Router<\/strong><\/p>\n![\"cmd\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-08-19-41-31.png\")
<\/p>\n![\"cmd\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-08-19-42-41.png\")
<\/p>\n
\nIn MongoDB, sharding is applied at the collection level<\/strong>. To shard a collection, follow these steps:
\n
\nEnable sharding for the database:<\/strong> You first need to enable sharding on the database that contains the collection.
\nNote :- use coffeeShop, before enabling sharding.<\/em><\/p>\n![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-18-19-32-40.png\")
<\/p>\n![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-01-13-53.png\")
\nTo distribute data across shards, split the data at customerId: 100. This will create two ranges:
\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 customerId \u2264 100 (will go to shard1ReplSet)<\/em>
\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 customerId > 100 (will go to shard2ReplSet)
\n<\/em>
\nExecute the following command to split the data:<\/p>\n![\"img\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-19-00-41-09.png\")
<\/p>\n
\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0Move customerId \u2264 100 to shard1ReplSet<\/em>
\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0Move customerId > 100 to shard2ReplSet
\n<\/em>
\nMove data for customerId values up to 100 to shard1ReplSet:<\/strong><\/p>\n![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-19-00-43-08.png\")
<\/p>\nSummary of the Approach<\/span><\/h4>\n
\nMoving chunks ensures that<\/em><\/span>: Orders with customerId values less than or equal to 100 are stored in shard1ReplSet<\/span>. Orders with customerId values greater than 100 are stored in shard2ReplSet<\/span>.<\/p>\nModifying the shard key in MongoDB<\/h4>\n
\n<\/em><\/span>MongoDB has introduced resharding in newer versions, where you can define a new shard key and MongoDB will redistribute the data according to the new key.
\nExample:<\/p>\n![\"img\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-18-19-49-20.png\")
<\/p>\n
\nIf you’re using an older version, the alternative is to create a new collection with the desired shard key and migrate the data manually. This is cumbersome and should be avoided in production environments.<\/p>\n![\"meme\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-14-20-29-13-1-300x237.png\")
\n<\/strong>Spring Boot Application for Sharded MongoDB<\/h3>\n![\"img\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-12-32-04.png\")
<\/p>\n![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-12-29-29.png\")
<\/p>\n![\"repo\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-12-50-16.png\")
<\/p>\n![\"img\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-12-53-30.png\")
<\/p>\n![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-12-58-49.png\")
<\/p>\n![\"image\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-13-01-23-1024x277.png\")
\nfor e.g. Added 11 data entries, with 7 moving to the shard2ReplSet and 4 to the shard1ReplSet, as shown in the snapshot below.<\/p>\n![\"img\"](\"https:\/\/www.tothenew.com\/blog\/wp-ttn-blog\/uploads\/2024\/09\/Screenshot-from-2024-09-20-14-18-13-1024x551.png\")
Conclusion<\/strong><\/h3>\n