PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.

Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes synthetic data, step-by-step guide, and certification prep.

End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)

Repository for Microsoft Databricks Training Events - Hosted by BlueGranite

[archived] A Python SDK for the Azure Databricks REST API 2.0

Reusable Python classes that extend open source PySpark capabilities. Examples of implementation is available under notebooks of repo https://github.com/bennyaustin/synapse-dataplatform

Automated pipeline for energy consumption forecasting across Europe using Azure cloud and Databricks.

Free High-Quality Financial Data in Azure

A wrapper for the Azure Databricks REST API

Applying data engineering techniques to create data pipeline with Azure Cloud Computing

A demand forecasting pipeline deployed on Azure and AWS

ETL motor racing data project using Azure Databricks, Pyspark and Azure Date Lakes

A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.

This branch focuses on building Data Engineering Interview Question and Answer

F1 Data Engineering Project on Azure Databricks!

A powerful Model Context Protocol (MCP) server for executing Databricks SQL queries and comparing table data.

Production-grade RAG Agent system for TD SYNNEX on Azure Databricks. Features Vector Search, MLflow pipelines, and GenAI product recommendations.

Article Repository for: Ensemble Machine Learning Modeling for the Prediction of Artemisinin Resistance in Malaria

In this project, I've created an end-to-end ETL pipeline and subsequently developed a machine learning model to predict the price of Amazon products based on several product-related features.