I am pleased to present the dbt-duckdb-tickit-pipeline, a data engineering workflow integrating dbt with DuckDB tailored around the TickitDB schema.

https://github.com/senthilsweb/dbt-duckdb-tickit-pipeline

This project is a tailored workflow designed to empower data engineers with a seamless and robust data engineering pipeline using dbt alongside DuckDB, focusing on the well-known TickitDB schema.

As a complement to this work, my open-source projects, DuckDB Data API and DuckDB Studio, serve as utility development tools for building and managing data engineering pipelines.

The Final Lineage Graph in DBT.

What’s Inside?

::list{type=“success”}

• An advanced data pipeline crafted for local data lake development.
• A process stretching from raw data ingestion to the final business intelligence layer, including raw, staging, intermediate, and mart creation.
• A strong foundation in data observability with data quality and lineage tracking. ::

This work builds upon Gary A. Stafford’s dbt-redshift-demo, adapting the TickitDB data model from AWS RedShift for local development, which enables not just sophisticated data transformations but also facilitates data governance and metadata management.

For those eager to dive deeper into the concepts behind this implementation, I highly recommend Gary A. Stafford’s insightful article on “Lakehouse Data Modeling using dbt, Amazon Redshift, Redshift Spectrum, and AWS Glue.”