the work

Pipelines that don’t wake me up at 3 a.m.

Most of these are reference builds — complete, runnable implementations on GitHub, not client work I can’t show you. Each one says what it does, why it’s built that way, what’s actually proven, and what’s still demo-only. One of them runs on real data.

HMIS Realtime Data Platform (opens in new tab)Reference platform
A multi-cloud streaming platform that coordinates shelter beds, outreach, meals, and case management across homeless-service agencies in real time.
Built on the real HUD HMIS data standard every federally-funded agency uses, and designed privacy-first — client identities are hashed and masked before they ever reach a dashboard.
Proof
End-to-end Kafka and Spark Structured Streaming (exactly-once) with Debezium CDC, plus separate Azure and AWS deployment branches, all readable on GitHub.
Still limited
Runs on simulated agency event streams, not live agency data — a working reference architecture, not a deployed system.
Kafka · Spark · Delta · Snowflake · Airflow · dbt
CDC: Postgres → BigQuery (opens in new tab)Reference build
A change-data-capture pipeline: a Postgres table streams row-level changes through Debezium and Kafka into BigQuery, in both append-only and SCD Type 2 modes.
Handles the parts of CDC most demos skip — dead-letter routing for malformed events and daily SCD2 compaction — with the full Debezium → Beam → BigQuery path wired end to end.
Proof
Runs two ways: a zero-infra Python demo and a full Docker stack (Postgres + Kafka + Connect + Beam), with unit tests for envelope parsing and SCD2 logic.
Still limited
A demonstration on synthetic data; not pointed at a production database.
Debezium · Kafka · Beam · BigQuery · SCD2
Cloud Cost Anomaly Monitor (opens in new tab)Real-data tool
Pulls daily AWS / Azure / GCP billing into a local DuckDB warehouse and flags per-service cost spikes with statistical anomaly detection, alerting to Slack.
The one project here with no synthetic data — it runs against real billing APIs on a daily GitHub Actions cron, so the time series and anomalies are genuine.
Proof
A real recurring pipeline on a scheduled cron, an EWMA + IQR detector with unit tests, and a live Streamlit dashboard — all in the repo.
Still limited
Built and run on my own free-tier cloud accounts; the scale is personal, not enterprise.
Multi-cloud · DuckDB · FinOps · Slack
SAP → Lakehouse → Snowflake (opens in new tab)Reference build
A medallion (bronze → silver → gold) lakehouse that moves simulated SAP ECC finance extracts through Delta Lake into a Snowflake-style serving layer.
Models a real enterprise pattern — Auto Loader-style incremental ingestion with schema evolution and SCD2 dimensions — but runs on open-source stand-ins so anyone can execute the whole thing with one command.
Proof
Self-contained: runs locally with MinIO for ADLS, Spark for Databricks, Airflow for ADF; Great Expectations validation in CI; Azure ADF and Snowpipe artifacts included for reference.
Still limited
A reference/demo on synthetic SAP data — the cloud services are stood in by local equivalents, not deployed.
Azure · Delta · Snowflake · PySpark
Realtime Inventory on Delta (opens in new tab)Reference build
A streaming inventory pipeline: simulated warehouse events flow through Kafka and Spark Structured Streaming into a Delta Lake table, with a live Streamlit stock dashboard.
Gets the streaming fundamentals right — exactly-once semantics, watermarking and late-arrival handling, MERGE upserts, and backpressure tuning — the things that actually break in production streaming.
Proof
The whole stack runs on `docker compose up` with no cloud account; CI builds and integration-tests on every push.
Still limited
Inventory events are produced by a simulator, not a real warehouse feed.
Kafka · Spark Streaming · Delta · Streamlit
Claims Modernization on AWS (opens in new tab)Reference build
A Hadoop-to-AWS modernization pattern: synthetic insurance claims land in S3, are processed by Glue/PySpark into partitioned Parquet, and load into Redshift for actuarial reporting.
Treats governance as a first-class concern — column-level PII tags in the Glue Catalog and masking enforced at the Redshift view layer, with Terraform IaC for the whole stack.
Proof
Runs locally on LocalStack or against real AWS, with the full Terraform and an EventBridge → Lambda trigger included.
Still limited
Explicitly a reference pipeline on synthetic, Faker-generated claims.
AWS · Glue · Redshift · Governance
Data Quality Framework (opens in new tab)Reusable framework
A reusable framework that wires schema-drift, null-rate, uniqueness, freshness, and row-count checks into CI as blocking quality gates.
Built from a clear opinion: most teams write one-off validation scripts that rot, so this gives one declarative place to register checks and surfaces failures where the team already lives — Slack, Teams, email.
Proof
An installable CLI that exits non-zero in CI, seven check types out of the box, and an end-to-end example suite against DuckDB.
Still limited
A starter framework, not a hosted service — you point it at your own warehouse.
Python · Great Expectations · YAML · CI
FaceMood Music (opens in new tab)Student project
A Django web app that detects your emotion live from the webcam and builds a “mood-journey” playlist that meets your mood and gradually lifts it.
An honest final-year computer-science project — kept here because it shows end-to-end product thinking (auth, history, admin, graceful ML fallback), not the systems work the rest of this page is about.
Proof
A full working Django app with DeepFace integration and a stub-predictor fallback so the UI still runs without the ML model.
Still limited
A college project, labeled as such — not professional or production work.
Django · DeepFace · Computer Vision
Martha — Voice Assistant (opens in new tab)Early project
A Windows desktop voice assistant (C# / WinForms) that takes spoken commands and responds with speech.
An early build from before the data work — kept for honesty about where the path started.
Proof
Source on GitHub.
Still limited
An old personal project; not representative of the current work.
C# · WinForms · Speech