Collibra Data Intelligence Platform

I am a user who has worked at companies that use Collibra Platform  as their data catalog and data intelligence platform tool. My first company, MetLife, used Collibra Platform , but I wouldn't know where they purchased it from because I joined after the implementation. Then I worked at HCL Tech, and one of our clients, Genmab, a pharmaceutical company, also used Collibra Platform. I was onboarded onto that project after the licensing and purchase were completed.

I worked at MetLife, which is an insurance company with different lines of businesses including US business, EMEA business, and LATAM. Based on the different geographies and lines of businesses, we needed to ingest the metadata of insurance products. Insurance, as part of the financial services industry, is highly regulated and must comply with regulations such as GDPR, HIPAA, and BCBS 239. Many insurance companies have faced heavy fines when they failed to comply with regulations, experienced customer data leaks, or had privacy breaches. Having data governed became critically important.

The main purpose of data governance in any industry is to have a single source of truth. For example, at PeerSpot , if you ask what a customer means, one person may have a specific definition while someone else may have a different definition. However, as an enterprise, you would want to define what a customer is, establish which attributes a customer should have such as customer ID, the date the customer was onboarded, and the revenue generated from that customer. Data governance ensures that every organization has a single source of truth and users have a common vocabulary.

The main challenge organizations face today is that business and IT are often at odds with each other. Business uses the data while IT generates it, leading to constant debates about ownership, control, and governance. My role as a data governance consultant was to build a bridge between the business and technical folks, and between business stakeholders and IT staff. We achieved this by leveraging Collibra Platform.

We started by creating the community structure. Community structure means organizing the metadata based on lines of business or geography. We created communities based on the geography and line of business. For the different communities we built, I worked closely with the US business data governance council, and most of our work was for the US business. Within the US business, there were many sub-lines of businesses, each of which had a business glossary. A business glossary is a container that contains all the business terms used in an organization. Every business term would have a definition, indicate which column it is stored in, and describe what business rule governs the business term.

Next, we ingested the technical metadata, which is called a physical data dictionary. Technical metadata includes schemas, tables, and columns. Collibra Platform has a unique functionality called Edge. Edge extracts metadata and registers any source such as databases stored in SQL, Oracle, Snowflake , and Fabric , which are all different sources we worked with. Collibra Platform has a tool using Edge, and the benefit is that the Linux servers are stored on your organization's server. For example, at MetLife, Collibra Platform's Edge servers would be stored on our MetLife cloud only, not externally. The organization is assured that their data is safe. Using Edge, we extracted the technical metadata of schemas, tables, and columns.

We created the business glossary and the physical data dictionary, then ingested the business rules and the data quality rules. Finally, we created a mapping specification using field mapping to create a lineage. A lineage is something which every stakeholder looks for and represents the flow of data or metadata from different sources to targets. In a typical scenario, metadata starts from a system of record or SOR, flows to a raw data zone or RDZ, then to a curated data zone or CDZ, and finally  to a distribution data zone or TDZ. These are four different layers, and some organizations use a Medallion structure with gold, silver, and bronze levels. A lineage gives users a visual sight of where the metadata is coming from and where it is going.

Lineage helps with impact analysis. If an organization experiences a security breach and does not have Collibra Platform or data governance in place, they will be wondering where the data can be impacted and what customer data could be leaked, requiring reactive analysis. However, if lineage is already established, when a bug or ransomware hits systems, we already have lineage in place and can mitigate the downstream systems so that before data reaches them, we can pause dashboards or disconnect connections. We can understand the impact as soon as an incident occurs, allowing us to be proactive rather than reactive. This is why lineage is so critical to have in place beforehand.

For each of my different customers within MetLife, including those working on different insurance products such as long-term disability, short-term disability, and accident and health insurance, my role was to create the business glossary, the physical data dictionary, the business rules, the data quality rules, and ultimately the lineage.

The main purpose of using any data governance tool depends on why a business wants to do data governance. Some businesses may pursue data governance due to regulatory reporting requirements such as GDPR, HIPAA, and BCBS 239, while others may want to achieve operational efficiency. However, for the stakeholders I have worked with, they were looking for improved data trust and consistency.

Before Collibra Platform, insurance had different departments and teams such as claims, underwriting, actuarial, and finance departments. Since they did not have Collibra Platform in place, for the same terms such as loss ratio or premium, every department had a different definition. However, by implementing Collibra Platform, we had a single governed source of business definitions. That eliminated any conflicting numbers in the reports. The tensions between business and IT existed partly because of inconsistent definitions. By having Collibra Platform and business glossaries in place, it reduced the back and forth between the business and IT teams and increased confidence in the reporting for regulatory purposes.

We are audited every year at MetLife by the big four companies such as Deloitte and KPMG. Insurance companies have to go through many different kinds of audits each year. There is significant regulatory scrutiny, but since we had Collibra Platform beforehand, we had clearly documented data ownership and stewardship even before the audits were in place. For example, if there is a source system for long-term disability using a Snowflake  source, sometimes the business does not know who owns it. At the time of reporting, everybody wants to put the blame on somebody else, which indicates a lack of ownership. An important part of data governance is to define roles and responsibilities beforehand. If Irina is responsible for a particular source system, then it is her responsibility to ensure that all business terms are consistent. By doing that beforehand, we had clearly defined who were the data owners and who were the stewards. We also had end-to-end lineage in place, from the source to the targets, which helped in regulatory reporting.

Auditors often ask for evidence in audits and want to see definitions, controls, and data flows. Since we had that beforehand, our teams spent almost 30 to 40% less time responding to audits and regulatory queries. Earlier, they would spend days and weeks preparing these reports and presentations, but almost 30 to 40% of time was saved by having Collibra Platform in place.

Another benefit is having a central single source of truth. The goal of any tool such as Collibra Platform is that any user across the organization can search for a term. MetLife is in many countries across the world, and any employee should be able to search for what a party identifier is, what accident and health insurance means, or what long-term disability is. If we did not have a central repository such as Collibra Platform, they would have to ask the most senior person they knew, who would then have to ask someone in the states. It would be difficult to find the right person who knows where the data is located, who owns it, and what its meaning and definition are. By having Collibra Platform, we have a central repository which allows users to discover and find curated best quality data beforehand.

Before Collibra Platform and after Collibra Platform, users saved at least two to three hours per week on average. Over a week, this may seem minimal, but translated across 50 weeks, that is almost 100 hours saved per employee per annum. When multiplied across an organization with tens of thousands of employees, this represents significant savings in hours per week. These benefits include saving time, being ready for audits, and having a single trustworthy central repository.

Collibra Platform's Edge capability is a very strong capability that allows us to extract metadata. The benefit of Edge is that it even does classification for us. For example, if there is a column which contains social security numbers, Collibra Platform will identify that this column appears to have social security numbers and will mask those columns automatically. It will suggest this to me, and then I have the opportunity to either approve or reject the suggestion. Ultimately, that is in the hand of us users. The extraction and classification features are very, very strong.

Collibra Platform also has very good mapping specification capabilities. As I mentioned, lineage is important, and Collibra Platform also has automated lineage. For example, if you have a source system in Oracle and your target is in SQL, if you scan these systems, Collibra Platform is capable of automatically identifying the relationship between the source and target and fetches the automated lineage as well. That is a very good feature.

The visual user interface is very helpful. I have worked with other data governance tools as well, and I found Collibra Platform to be one of the best tools because end users should have an easy time navigating through the metadata. When you search any term in the search bar at the top, such as risk or loss ratio, it gives you a list of search results. At the same time, on the left hand side, it gives you a filter panel. You can filter whether you are looking for a column or a table, a business term, or a data quality rule. You can filter by the kind of asset you are looking for, or even filter by line of business to see if you are looking for something specific to the US business or the Australia business. The search interface is very helpful.

The workflows are also very strong capabilities in Collibra Platform.

A separate team did the setup for this tool and was involved in setting it up. When they were doing the setup, it was a significant upfront investment in terms of the configuration, modeling, and governance design because the team first had to define the domains, the assets, and the roles. That takes time, and in the insurance environment where we had many legacy systems, this increased the setup effort. It would be helpful if Collibra Platform could have industry specific starter templates that could accelerate adoption.

As we ingested more and more metadata, sometimes the catalog searches became slower by a fraction of a second because there was more metadata in the tool. After inserting millions of rows into the physical data dictionaries, the catalog searches could become a little bit slower. Having some better performance optimization might help with this minor issue.

I have used Collibra Platform for around four years.

Collibra Platform is pretty stable and is a reliable tool, and I would use it again.

Collibra Platform is a scalable tool and is very scalable. It is very critical for a data governance tool to be stable and scalable because we start with critical data sets in one line of business, but ultimately, we want to cover the enterprise and all lines of businesses. Unless the tool is strong in its capabilities in terms of scalability, no organization would be able to implement data governance at large. Scalability is critical, and I think Collibra Platform is a suitable tool for scaling.

Collibra Platform has a Collibra University, and the Collibra Platform team helped us gain access to it. They also guided me to complete two learning paths, the Collibra Data Steward and the Collibra Solution Architect. The team has been very helpful in guiding us in terms of training and documentation. On the Collibra Learning Center, if you go on help, there is extensive documentation available. In terms of customer support, we would connect with them once a month to discuss any kinds of queries. Whenever we did have challenges, they were available, and we were satisfied with their support.

Collibra Platform is a premium tool and an enterprise grade premium tool, but I think it is justified because the only other tool more expensive than this is the Informatica suite, which includes Informatica Cloud Data Governance , Informatica EDC, and Axon. I think it is worth it because the features present in Collibra Platform are extensive. Sometimes companies consider Microsoft Purview  because they get Purview  for free with the Microsoft suite if they have an E5 license or something similar. However, they do not realize that with the free tool, they do not have as many capabilities as Collibra Platform has. I think it justifies the upfront investment, and we have seen good ROI in terms of the search time saved by our business users and the time saved in preparing for audits. We have seen good ROI in our business.