A unified data platform that broke departmental silos and centralised the organisation’s single source of analytical truth.
Maintenance and evolution of the corporate Deltalake, a medallion-structured platform (bronze/silver/gold) on replicated buckets with partitioned parquet files, using Apache Iceberg as the catalog layer and OpenMetadata for governance, lineage and data discovery.
Batch pipelines on PySpark over Kubernetes. Real-time ingestion combined Debezium and Kafka to capture changes from heterogeneous operational systems. Bronze ingested raw data, silver handled aggregation, cleaning and normalisation, and gold was generated on demand to feed internal analytics, downstream AI processes and leadership reporting.