AWS, Cloud, Data & Analytics

Snapshot Migration to S3 and Extract Data from Snapshot using Athena

Introduction Sometimes We have to keep our database backup for longer retention for Security and DR Compliances in RDS. But as we know, if we have a longer backup period in RDS, it will also create more burden on the pocket, as RDS charges for storage in Normal S3 bucket format. So we can export […]

November 11, 2023

Big Data, Data & Analytics

Simplifying Data Engineering: An Introduction to DBT

Introduction Data is a key asset in today’s business environment, holding great potential for making wise decisions and preserving a competitive edge. However, the road to efficient data management is frequently difficult and time-consuming, especially when dealing with big and varied datasets. In this first blog post of the DBT series, we will introduce dbt, […]

October 30, 2023

Big Data, Data & Analytics, DevOps

Enhancing Workflows with Apache Airflow and Docker

In today’s world, handling complex tasks and automating them is crucial. Apache Airflow is a powerful tool that helps with this. It’s like a conductor for tasks, making everything work smoothly. When we use Airflow with Docker, it becomes even better because it’s flexible and can be easily moved around. In this blog, we’ll explain […]

October 17, 2023

Big Data, Data & Analytics, Software development

How to Setup Astro CLI and deploy to Astro (Windows)

Setup: Download the optimal version of Astro for your Windows system from link. Rename the downloaded file to “astro.exe” and save it. Add the file path to environment variables. To check if Astro has been configured correctly, run “astro” command On cmd. After the successful configuration of Astro CLI, you should get a response like […]

October 10, 2023

Android, Data & Analytics, Manual Testing

ANR for TV

Description When the UI thread of an Android app is blocked for too long, an “Application Not Responding” (ANR) error is triggered. If the app is in the foreground, the system displays a dialog to the user, as shown in the figure below. ANRs are a problem because the app’s main thread, which is responsible […]

October 8, 2023

Anaytics, Data & Analytics

Microsoft Fabric: An Umbrella for Data Analytics Services

Microsoft Fabric is an end-to-end, cloud-based SaaS solution for data and analytics. It is built on top of an open lakehouse (OneLake) and weaves together multiple Microsoft tools to streamline all data and analytics workflows, from data integration to data engineering to data science. What is Microsoft Fabric, and why is it needed? Microsoft Fabric […]

September 30, 2023

Big Data, Data & Analytics, Testing

Spark with Pytest : Shaping the Future of Data Testing

PySpark is an open-source, distributed computing framework that provides an interface for programming Apache Spark with the Python programming language, enabling the processing of large-scale data sets across clusters of computers. PySpark is often used to process and learn from voluminous event data. Apache Spark exposes DataFrames and Datasets API that enables writing very concise […]

September 29, 2023

Data & Analytics, DevOps

Enabling Client-Side Encryption for MongoDB

In today’s data-driven world, the security of your data is paramount. MongoDB, a popular NoSQL database, offers robust security features to protect your sensitive information. One of the most powerful security mechanisms MongoDB provides is client-side encryption. This approach allows you to encrypt data on the client side, ensuring that even if unauthorized users gain […]

September 27, 2023

Anaytics, AWS, Data & Analytics

Automated PDF Filing with AI and NLP

Automating PDF Filing with AI and NLP In the ever-evolving world of data science and automation, innovative solutions have continually emerged, simplifying intricate tasks and enhancing efficiency across various industries. One such transformative application is the automation of PDF document filing, a process that has witnessed significant enhancements due to advances in artificial intelligence (AI) […]

September 25, 2023