Previously, data engineers used Kinesis Firehose to transfer data into blob storage (S3) and then load it into Snowflake using either Snowpipe or batch processing. This introduced latency in the data pipeline for near real-time data processing. Now, Amazon Kinesis Data Firehose (Firehose) offers direct integration with Snowflake Snowpipe Streaming, […]
Read More
This blog post showcases a real-time data pipeline built in Snowflake that leverages Slowly Changing Dimensions (SCD 2) and Finalizer Tasks to ensure your customer data is always fresh, accurate, and reflects historical changes. Imagine you have a system that continuously generates customer data, including customer number, status, balance, invoice […]
Read More
Efficient management of Snowflake resources is crucial for optimizing performance and cost. One of the key aspects is monitoring warehouse usage. In this blog post, we’ll explore a use case where we automate the reporting of warehouse usage and send detailed weekly summaries via email. Our Organization is a data-driven organization […]
Read More
Imagine a sales team at a fast-growing software company. They rely on a robust understanding of sales performance across different regions and product lines to make informed decisions and optimize sales strategies. The primary objective of this use case is to develop an interactive sales data dashboard that allows business […]
Read More
In today’s dynamic environment, automating business day-specific tasks is crucial for aligning with operational workflows and regulatory requirements. The Snowflake Data Cloud offers powerful tools and capabilities to streamline such processes, ensuring efficiency and accuracy. Imagine you manage a data pipeline in Snowflake that involves various tasks, including data ingestion, […]
Read More
In modern data-driven environments, it’s common to receive daily data feeds from various sources. These files need to be ingested into a data warehouse like Snowflake for further processing and analysis. Automating this process ensures data is consistently and reliably loaded without manual intervention. Suppose you are a data engineer […]
Read More
