top of page

Snowflake Sales ELT

Problem:

  • Sales data from various sources needs to be efficiently ingested, transformed, and analyzed in Snowflake to provide business insights

​​

Solution:

  • Data Ingestion - Ingested data in various formats (CSV, JSON, Parquet) using Snowpark.

  • Data Processing - Transformed and analyzed data through different stages (bronze, silver, gold).

  • Data Visualization - Visualized results using SnowSight Dashboard.

logo-snowflake-l-min.jpg

System Architecture

Data:

  • Only Paid and Delivered Order were processed. We only want completed orders.

    • Payment Status - Failed & Pending were ignored.

    • Shipping Status - Pending, In Transit & Shipped were ignored.

  • Duplicate Orders are ignored by Order_ID.

system_architecture.png

Video Walkthrough

In this video, I walk through the implementation of an ELT process using Snowflake and Snowpark. The project involves ingesting sales data in CSV, JSON, and Parquet formats, loading it into Snowflake's internal stage, and performing data transformations across the bronze, silver, and gold layers. The final dataset is then visualized using SnowSight Dashboard, showcasing the capabilities of Snowflake for robust data warehousing and analytics.

Conclusion

Working on the Snowflake ELT Project significantly enhanced my expertise in data warehousing and analytics using Snowflake. Through the process of ingesting, transforming, and analyzing sales data in various formats, I gained hands-on experience with Snowpark and SnowSight. This project reinforced my understanding of efficient ELT processes and the importance of data organization across different layers. It also solidified my skills in leveraging Snowflake's capabilities to deliver actionable business insights, preparing me for future data engineering challenges.

bottom of page