RaccoonLaptopGif

Modern Data Stack Data Warehouse

As the lead data engineer and manager of the data & analytics department at OHB I was responsible for the end to end ideation, design and implementation of this enterprise data warehouse project. Keeping aligned with cutting edge industry trends I architected a solution inspired by the ‘modern data stack’ opting into Snowflake as the primary data store, Fivetran as the primary ETL service, and the Dbt framework for accurate and reproducible CI/CD data modelling. This stack was chosen to be highly scalable, cost effective and risk-averse considering the limited in-house developer resources at the time as the team was actively growing. The wider system included a data audit system complete with monitoring dashboards, a self-hosted and maintained orchestration platform (Apache Airflow), and ancillary data stores (AWS RDS) and data pipelines handling legacy data and fringe data sources that couldn’t be handled by Fivetran alone.

Architecture Diagram Fivetran Overview