End-to-end Azure data engineering pipeline ingesting real-time earthquake data from the USGS API. Implements a Bronze–Silver–Gold lakehouse using Azure Data Factory, Databricks, ADLS Gen2, and Synapse Analytics, with both manual execution and fully automated daily-triggered workflows.

Kafka Connect Sink for ADLS Gen2 with SAS auth, Avro support, GZIP compression, and partition-based file batching.

End-to-end Azure Data Engineering project using Medallion Architecture, Databricks, Synapse, and Power BI.

End-to-end Azure Data Engineering pipeline using ADF, Databricks (PySpark), ADLS Gen2, Azure SQL, and Power BI for COVID-19 analytics

Master Azure Data Engineering with this Basic to Advance guide! Covers SQL, PySpark, Kafka, Databricks, Snowflake & Airflow. Build 15+ industrial projects using Azure (ADF, Synapse, Event Hubs), GCP & modern table formats (Delta Lake, Iceberg, Hudi). Learn real-time streaming, Medallion architecture, and cloud data warehousing with hands-on labs.

In this project we will orchestrate the movement of raw nyc taxi data from staging layer to presentation layer and to semantic model for creating visualizations.

Real-time streaming data pipeline using Apache Kafka, Spark Structured Streaming, and Delta Lake on Azure. Secure SSL Kafka integration, ADLS storage with OAuth2, and ML-driven anomaly detection with automated email alerts. Modular, scalable, and configurable for IoT and log analytics pipelines.

GitOps-driven Azure Data Factory pipeline that ingests multi-source data (GitHub + ADLS) into ADLS Bronze using dynamic, parameterized ETL workflows.

End-to-end data engineering pipeline implementing Medallion Architecture (Bronze-Silver-Gold) for trip transaction analytics. Automated ETL using Azure Data Factory, Databricks, and Delta Lake with real-time monitoring and email notifications via Logic Apps.

Azure pipeline for USGS earthquake API data. ADF-orchestrated ingest → Databricks bronze/silver/gold transforms → ADLS storage → Synapse-ready outputs (plus optional Power BI). Built for scalable daily runs.

End-to-end e-commerce data platform built on Microsoft Azure using Databricks Medallion Architecture.

End-to-end Azure Data Engineering project implementing Medallion Architecture using ADF, ADLS Gen2, Databricks (Spark), Synapse Analytics, and Power BI for Adventure Works analytics.

Predictive maintenance alert pipeline (C-MAPSS): ingest → preprocess to Parquet/Delta → train & score failure risk in Databricks → write alerts.json → Logic App notifies Teams/Email → Power Automate creates Planner tasks.

🚀📊 This project demonstrates a complete ETL pipeline for retail sales data using ☁️ Azure Databricks, 📦 ADLS Gen2, and PySpark, following the 🟫 Bronze → 🟦 Silver → 🟨 Gold architecture.

Automated Medallion Architecture (Raw ➔ Bronze ➔ Silver) in Azure Data Factory featuring dynamic schema mapping and incremental delta upserts.

ETL/ELT of Grocery Sales data by Azure Data Factory (ADF)

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

Designed a production-grade Azure Data Engineering project centered on Azure Data Factory. Built dynamic, metadata-driven pipelines to ingest data from on-prem systems, REST APIs, and Azure SQL into ADLS Gen2 using Medallion Architecture, incremental loading, and enterprise-scale orchestration patterns.

End-to-end Metadata-Driven Data Engineering framework built on Azure. Features dynamic SQL/REST API ingestion with range pagination, automated schema mapping, and event-driven orchestration. Implements robust CI/CD via GitHub Actions/YAML and automated failure alerting with Logic Apps. Optimized for scalability and DE best practices.

🚀 Build an automated data pipeline with Azure Data Factory and Databricks to efficiently process and analyze trip transaction data using Medallion Architecture.