DataHub

Since many developers were developing and creating data models, and there were no DDLs left in the company, we had to recreate all the descriptions of tables and clarify which columns meant what. We made it a place where all stakeholders in our company could log in and see which data were used for which data marts, which column values meant for which definitions, and how they were measured. We primarily used Data Hub for sharing information and increasing the data literacy of our company.

The data injection feature was valuable to me. If comments were inserted in the tables, the data would automatically gather and enter all necessary data into Data Hub. Additionally, the data lineage graphs were really helpful in showing how data flowed to data marts and which columns were used for creating other columns. Those were helpful but somewhat hard to manage features.

I am not familiar with how people use Data Hub as a knowledge system for developing LLM models or RAG, so I am uncertain what improvements could be made in that sense. However, the way we used it, the data was quite heavy because it consisted of multiple components such as graph DBs, Elasticsearch, Kafka for data injection, and MySQL  for metadata storage. This made it somewhat bulky on our server when we deployed Data Hub, and we had difficulty managing the memory constraints and disk usage.

When we used Data Hub, we attempted to provide different servers for different components, but we could not find good manuals on how to use it in a more productive server manner. Having guidelines or manuals on this could be helpful.

I used Data Hub for about a year.

Other than out-of-memory issues, Data Hub was stable because we did not have to restart services much except for memory issues. The overall service itself was stable, but it was very bulky.

For growing data graphs or data lineage, scalability depended on our manpower. From that understanding, Data Hub's scalability was not as great as we expected because we could not enter or obtain better data for all our tables because it was too much for our data teams. We had to be selective in that matter. From our understanding, we could not really enjoy the scalability of the data.

We were fortunate not to need customer service, but we did not know that technical support for Data Hub was possible. If we had been aware of that, we could have asked for support or guidance.

The initial deployment of Data Hub itself was easy because we could get the Docker  Compose YAML file and run the Docker  command to get the service up. However, since Data Hub uses various types of components, we had trouble assigning each server for each component and connecting them via network. We overcame that problem.

We searched for other use cases and how other companies were developing their own solutions for using data, but we have not been able to use other products similar to Data Hub itself.

I am not familiar with how the data is priced, so I cannot answer that question. Since we were running all the components on one server, there were issues when data injection was occurring too frequently, and sometimes the server could run out of memory or the disk storage could become full. We had maintenance issues that we had to handle in terms of memory and disk storage. We had to alter some injection strategies and timelines so that the data would not grow from too many tables at the same time. These maintenance jobs took place regularly. I would rate this product a 7.

Our main use case for Data Hub is to build our enterprise data catalog within Visa using the open-source version.

We use Data Hub to build pipelines and to construct our enterprise data catalog to see where the data is coming from, how the lineage flows, where the lineage of the data originates from, and how the metadata propagation occurs. With this metadata information and the description of all the fields, we have built a layer on top of this that performs natural language querying for people to find where and how the data comes from.

The best features that Data Hub offers include metadata propagation and lineage propagation.

These features have specifically helped my team and our workflows by enabling people to find the right data. We have different sets of data that include business data and application data. People who are new, including data analysts, machine learning scientists, or data scientists, can easily find the specific data they are looking for because it is all centralized in one place.

Data Hub has positively impacted our organization by centralizing and co-locating all data through metadata, and we have made this our enterprise metadata catalog rather than having disorganized information across different teams. It has saved time for many data analysts and data scientists to find the right data.

I have no comments on how Data Hub can be improved at this time.

I have been using Data Hub for the past four years.

On a scale of one to ten, I rate Data Hub a nine.

I chose nine out of ten because Data Hub is a single solution that we could adopt easily and build our platform on top of it. It provided all the features that we needed, which is why I gave that rating.

Regarding Data Hub's AI capabilities, I have not explored its governance and security features, but I would like to explore them. I have not gone through the AI features of Data Hub concerning the accuracy and reliability of output.

My advice to others looking into using Data Hub is that it is a fantastic tool for people who want to centralize and keep all the data discoverable in one single place. I would highly recommend using it. I give this review an overall rating of nine out of ten.

Within the engineering teams of Renault, there was a lot of data without sufficient metadata, such as descriptions of tables and columns. The objective was to complete the definitions and descriptions of business data objects within the glossary and map these descriptions to the tables and columns that comprise the data sets of this engineering department to ensure a comprehensive experience when searching for data, providing adequate definitions and descriptions of the data used in this department.

I use Data Hub within two of my clients. With Renault, the car constructor, they changed their data catalog from Zeenea to Data Hub, and I have a mission to contribute to the enrichment of this data catalog by conducting workshops with data providers, data stewards, and all the stakeholders involved in this data catalog. The aim of this mission is to map real data to the definitions and descriptions of business data objects available in the company's glossary. My second mission was with Hitachi Rail, a company that provides rail services, where the mission involved benchmarking several data catalogs including OpenMetadata and Data Galaxy . Data Hub was chosen for its available functionalities, with the task of implementing this data catalog with a specific scope and then completing the usage of this data if everything works well.

The best feature that Data Hub offers in my experience is the ability to map between real data and data sets.

The mapping feature helps my team and clients significantly because it addresses the lack of metadata information about the tables and columns used in the company's data lake, enriching the data catalog considerably through this mapping.

Data Hub positively impacts my organization and clients by making it easier to search for data. It facilitates easier collaboration and helps save time. However, concerning data quality, it is not sufficiently equipped as it lacks components to evaluate the data quality level, which is a feature available in other data catalogs, indicating an area for improvement.

Data Hub can be improved in several ways, primarily by enhancing the data quality evaluation capabilities. Additionally, I would suggest improving the hierarchy of business glossary terms, as understanding the characteristics of each business data object can be challenging within the current structure of business glossary terms in Data Hub.

I have conducted benchmarks with OpenMetadata and Data Galaxy , but I have never used them for a mission with my clients. Before choosing Data Hub, I evaluated all the principal tools on the market, including Castor, Data Galaxy, and OpenMetadata.

I have no experience with pricing as I used the free license. My advice for others looking into using Data Hub is to consider the paid version for enhanced options related to data quality and the availability of KPIs regarding the completeness and accuracy of metadata, which results in a superior experience with this tool. I would rate this product an eight out of ten.