Skip to content

Latest commit

 

History

History
38 lines (23 loc) · 1.8 KB

README.md

File metadata and controls

38 lines (23 loc) · 1.8 KB

Azure Projects Repository

This collection showcases end-to-end data engineering solutions and advanced analytics implementations. Below, you’ll find highlights of my work with Azure tools.

Featured Projects

1. Azure End-to-End Data Engineering Project

An end-to-end solution leveraging the Medallion Architecture to ingest, process, and analyze data using Azure Data Factory (ADF) and Azure Databricks.

Key Features:

  • Staged data transformations across Bronze, Silver, and Gold layers.
  • Automated pipelines for data ingestion, transformation, and load.
  • Integration with Power BI for seamless analytics.

2. Azure Serverless Logical Data Warehouse

A demonstration of serverless analytics with Azure Synapse Analytics, showcasing advanced SQL features and seamless integration with Power BI for insights.

Key Features:

  • Implemented CETAS (Create External Tables As Select) and Incremental Load Design.
  • Utilized Change Data Capture (CDC) for real-time updates.
  • Demonstrated SQL performance monitoring and query optimization.

3. Azure Data Factory Pipelines

Multiple pipelines developed to demonstrate below functions

  1. Data Ingestion Pipeline: Automated ingestion of structured and unstructured data into Azure Data Lake.
  2. Transformations Pipeline: ETL workflows built for scalable data processing.
  3. Orchestration Pipeline: Dependencies managed using pipeline chaining, conditional execution, and alerts for monitoring.