DataHub
DataHub is an open-source data catalogue that helps teams discover datasets, understand their origins, manage ownership and organise metadata across connected systems.
Why it caught my attention
DataHub addresses a problem that grows quietly as organisations add more databases, pipelines and reporting tools: nobody has a complete picture of their data. Bringing technical metadata and business context together makes it easier to investigate dependencies, find responsible teams and understand unfamiliar datasets.
What problem does this actually solve?
DataHub could be worth considering if you recognise any of these challenges.
Finding data takes too long
Information is scattered across databases and analytics tools. DataHub provides a searchable catalogue of connected data assets.
Changes have unexpected downstream effects
Teams cannot easily see what depends on a dataset. Lineage views help identify potentially affected reports and pipelines.
Nobody knows who owns what
Unclear ownership slows questions and incident investigations. DataHub records responsible people or teams alongside data assets.
Data quality problems lack context
Unreliable information can undermine reporting. DataHub brings supported quality checks and their results closer to relevant datasets.
Is this a good fit for you?
A quick way to see if DataHub is relevant for your situation.
- Your organisation uses several databases, warehouses or analytics platforms.
- You need to understand dependencies before changing important data pipelines.
- Different teams struggle to find reliable datasets or their owners.
- You want shared business definitions and clearer data governance.
- You need a catalogue that integrates with existing data engineering workflows.
- Your data is small, centralised and already easy to understand.
- You primarily need to move or transform data rather than catalogue it.
How does it work?
In practice, the workflow looks something like this:
Deploy the open-source platform using supported infrastructure or choose DataHub Cloud, depending on your operational requirements and preferred management approach.
Configure metadata ingestion for supported sources such as Snowflake, BigQuery, PostgreSQL or dbt, supplying appropriate connection details and permissions.
Run ingestion jobs to collect available schemas, descriptions, ownership information and lineage, then schedule updates to reflect changes.
Assign owners, organise assets into domains and add descriptions or glossary terms so colleagues can interpret what they find.
Search the catalogue, inspect dataset details and follow available lineage connections to understand upstream sources and downstream dependencies.
What else should I know?
A few important things to be aware of before you investigate further.
Open source and managed options
DataHub Core is open source, while DataHub Cloud offers managed services. Compare current features, support and commercial terms.
Integration capabilities vary by source
Not every connector extracts the same metadata. Check lineage, profiling and other supported capabilities for your particular systems.
Metadata requires ongoing maintenance
Automated ingestion collects technical information, but useful descriptions, ownership assignments and business definitions still need attention.
Self-hosting involves infrastructure management
Production deployments require suitable infrastructure and operational expertise. The Docker quickstart is intended for initial exploration.
Think DataHub could fit your environment?
Visit the DataHub website to find out more, request a demo or speak to their team.
Something you'd like to know about DataHub?
Is there something I haven't covered in my research? Send me your question and I'll look into it and reply personally by email.