Project Lumos

## Context Metadata was fragmented and stale across data platforms, leaving ownership, lineage, and documentation inconsistent. ## Constraints - Built as a side project with a small team and limited calendar time. - Needed to integrate across multiple stacks without rewrites. - Adoption had to be obvious in the first week, not month three. ## What I built - A modular, catalog-backed metadata automation toolkit (DataHub today) to reduce drift in ownership, lineage, and documentation. - Adapter-driven ingestion and enrichment for repeatable, low-friction onboarding. - Versioning + bulk update workflows to keep metadata current without manual upkeep. - A roadmap toward a control plane (Spring Boot) + workers (Python) for scheduling, retries, and audit trails. ## Impact - Saved 400+ effort hours annually by reducing manual discovery and bug triage. - Unblocked 10 FTE across teams and became a priority platform initiative. ## Stack Python (CLI + modular controllers/services), DataHub integration, and adapter-based ingestion; roadmap includes Spring Boot control plane + Redis Streams + Python workers.