In the previous blog, we understood how to Optimally use Snowflake Warehouse and Tables. Now, we continue the series by diving into Snowflake performance tuning, focusing on how to enhance query performance and manage associated costs in Snowflake cloud services. So let’s continue the blog series, where we will now focus on improving the performance […]
Big DataData & AnalyticsIndustry Buzz
AI is everywhere, but there’s less attention to the engine running behind it. Data Engineering is what makes the right data available at the right time for all these AI use cases. Happy that someone thought about it and organized this 2-day event DES2023, focused on Data Engineering in Bengaluru, where we got to hear […]
In the previous blog, we discussed the Best practices to be followed while Data loading into Snowflake from Stages. Continuing the snowflake blog series lets us understand how to use Snowflake Warehouse and Tables optimally. Snowflakes Virtual Warehouses Virtual Warehouses is one of the critical components in Snowflake architecture and deciding the correct configurations for […]
Data Ingestion: Data ingestion is the transportation of data from assorted sources to a storage medium where it can be accessed, used, and analyzed by an organization. The destination is typically a data warehouse, data mart, database, or a document store.Sources can be from RDBMS like MySql, Oracle, Postgres, File based like FTP,SFTP,Rest api’s,Streaming .The […]
Continuing our Snowflake blog series, after learning about setting up a Snowflake account using System-defined Roles, we will explore the best practices for loading data from a file into Snowflake. Let’s begin. Snowflake supports file-based data ingestion through Internal and External stages. However, there are various factors to consider when performing data ingestion, including the […]
This is the first blog in a series that will focus on Snowflake, where we’ll cover best practices for using Snowflake, explore various Snowflake functionalities, discuss how to maximize the benefits of Snowflake, and address the challenges that come with its implementation or migration. In this blog, we’ll start by discussing setting up a Snowflake […]
For one of our Global Advertising Management Platform clients, we did one migration project with zero downtime for components like Platform DB, Ceph, Aerospike, Kafka (Zookeeper +data nodes), MapR (hive, oozie, hue), Druid (Zookeeper +data nodes), Flink (Zookeeper +data nodes), Monitoring (Icinga,collectd, cloudwatch), Logging (logstash & Opensearch) & Other Components ( Nexus, SFTP, Jenkins ). […]
Introduction Amazon Elastic Map Reduce is a managed platform. We can run big data frameworks like Apache Hadoop and Apache Spark on AWS to process and analyze large volumes of data. We can process huge amounts of data for analytics purposes and business intelligence workloads with help of this framework. Amazon Elastic Map Reduce also […]
What’s up with this Google Analytics 4? Is it worth implementing? How exactly will it enhance our existing processes? How to get started with it? Announcing GA4, Google had this to say: “To help you get better ROI from your marketing for the long term, we’re creating a new, more intelligent Google Analytics that builds […]
