Let me tell you about the moment I realized I’d been overcomplicating things for years. I was working on a pipeline in Snowflake. You know the type — a multi-stage transformation process where a few base tables feed into intermediate tables, some reconciliation happens, and eventually it all lands in a final reporting layer. I’d...
Fun fact! Around 80%-90% of the world's data is unstructured. I was shocked when I read this fact. The unstructured data contains images, emails, PDF files social media posts, and other formats. Even though it is widely present 70% of data is not being used to drive insights and get analytics. As a Data Engineer, you must have...
In the previous blog, we briefly introduced DBT (Data Build Tool) and the fundamental ways it could change how you analyze and transform your data. We discussed the basics, explored its main components, and established the basis for comprehending its capabilities. DBT (Data Build Tool) is a remarkable data analytics tool that is...
Introduction Data is a key asset in today's business environment, holding great potential for making wise decisions and preserving a competitive edge. However, the road to efficient data management is frequently difficult and time-consuming, especially when dealing with big and varied datasets. In this first blog post of the DBT series,...
I had the opportunity to experience the Data Cloud World Tour, and it was all about collaborating with data in unimaginable ways. I joined the event with Suprakash Maity, Prashant Singhal, Sushant, and Vikramjeet along with leaders, to learn about the latest capabilities of the Data Cloud and to hear directly from our customers about...
In the previous blog, we understood how to Optimally use Snowflake Warehouse and Tables. Now, we continue the series by diving into Snowflake performance tuning, focusing on how to enhance query performance and manage associated costs in Snowflake cloud services. So let's continue the blog series, where we will now focus on improving the...
This is the first blog in a series that will focus on Snowflake, where we'll cover best practices for using Snowflake, explore various Snowflake functionalities, discuss how to maximize the benefits of Snowflake, and address the challenges that come with its implementation or migration. In this blog, we'll start by discussing setting...