Big Data, Data & Analytics

Efficient Data Migration from MongoDB to S3 using PySpark

Data migration is a crucial process for modern organizations looking to harness the power of cloud-based storage and processing. The blog will examine the procedure for transferring information from MongoDB, a well-known NoSQL database, to Amazon S3, an elastic cloud storage solution leveraging PySpark. Moreover, we will focus on handling...

by Bishal Kumar Singh
Tag: dataengineering
18-Sep-2023

Big Data, Data & Analytics

Spark Structured Streaming

In this blog, I will discuss how Spark structured streaming works and how we can process data as a continuous stream of data. Before we discuss this in detail, let’s try to understand stream processing. In layman’s terms, stream processing is the processing of data in motion or computing data directly as it is produced or...

by Ravindra Jain
Tag: dataengineering
31-Aug-2023