Data Engineering Projects
Work samples using popular data engineering tools, ranging from SQL and Python to AWS.
Built an end-to-end Medallion data pipeline on a real Walmart weekly-sales dataset, ingesting 421,570 department-level records into Snowflake via AWS S3, modeling a star schema with SCD1/SCD2 dimensional tables using dbt Cloud, and delivering 10 business reports as Python visualizations querying the warehouse directly. No BI tool required.
Designed a Customer Lifetime Value analytics platform for a multi-location restaurant business under real enterprise constraints: SQL Server as the source system, AWS-only tooling (no Snowflake, no dbt), and PySpark mandated for every transformation. The core deliverable is a daily CLV snapshot model built with Spark window functions — tracking how each customer's value evolves over time, not just a static total — alongside RFM segmentation, churn indicators, and discount effectiveness analysis.
Processed the full official CMS Nursing Home Provider Data Catalog, 22 files, ~780MB, 1.3 million daily staffing records across 14,564 facilities — on an AWS data lake using PySpark on Glue. Joined staffing data to provider quality ratings, ownership records, and compliance citations to surface metrics on nurse-to-patient ratios, facility occupancy, and regulatory risk, served through a 6-page Streamlit dashboard.
Built a live, event-driven AWS platform unifying Close CRM, Calendly, and Wistia into a single operational and analytics system. Webhook receivers process CRM and scheduling events in real time, a 10-minute SQS delay queue coordinates asynchronous lead-owner enrichment, and a paginated batch pipeline handles incremental Wistia video analytics, all feeding Slack notifications and 6 Streamlit dashboards. The entire system ran unattended for 7 consecutive days, with documented fault-tolerance and idempotency tests.