50PB Cost Optimization Program

Deep archival strategy and compute portability for 50PB of data with substantial monthly savings.

## Context Cloud spend was rising with scale. We needed a long-term data lifecycle strategy without degrading access or reliability. ## Constraints - 50PB of data across multiple workloads. - Business expectations for access, latency, and compliance. - Cost cuts must not reduce quality or resiliency. ## What I built - Deep archival strategy for cold data. - Workload right-sizing and cluster optimization. - Migration of select workloads off Databricks to EKS-based clusters. ## Impact - Identified and realized ~20% cost reduction, about 150k USD per month. - Saved ~2M USD in Databricks costs. ## Stack Cloud storage tiers, EKS-based compute, and cost governance workflows.

Mission Brief

The Problem

Cloud costs were scaling faster than usage, threatening margin and velocity.

The Solution

A cost-physics program that cut 50PB storage and compute spend without reliability regressions.