Monte Carlo Data + AI Observability Platform

I work as a business analyst and I usually see data anomalies in our company's data set, and I also work a lot on Power BI reports to see our performance on the supplier side.

When we receive data from our suppliers to view their performance, sometimes the data is not complete or they are doing backfills of previous data, so we have established some rules in Monte Carlo  to monitor these anomalies, and whenever we see something passing a preset limit, we receive an alert.

When I open Monte Carlo , I usually look at my dashboard to see how many alerts we have received in the last few days, but I usually check the alerts in the last month, and I see which rule has received the maximum amount of alerts; then I try to solve it first because the pattern is similar, and then I try to solve other alerts based on other rules.

In my team, it's me who handles those alerts, but we have another team who works on these alerts as well, although they are working on another kind of data set, but in the company, it's used by many people.

The best features of Monte Carlo for my work are the ability to see alerts clearly, how many alerts we have received on which rule and for which country, and there is a feature called investigation query inside Monte Carlo which shows a pre-done analysis, so you don't have to run an SQL query by yourself to do manual checks.

It gives a clear analysis in a large data set, which is very time-saving, so I don't have to run manual codes to verify the data, and it has helped me a lot in saving time and improving efficiency while doing data checks or verifying data. It's a really great tool to explain the anomalies when we see one in Monte Carlo, as we have actual proofs to show to people or to the managers that we are having this anomaly or that data is missing.

There's also an AI feature that is inbuilt in Monte Carlo, but you have to pay separately for that feature, and I used it for quite a while in the beginning, but now my company has disabled it.

The biggest pain point with Monte Carlo is that we have created some rules, but those rules cannot judge everything, and I think the platform is a bit complex for someone new, so it can be more intuitive; a display adoption platform could guide the user on how to use this, like a DAP  system. It took a lot of time for me to learn it, and without a guide, a new user would be clueless.

If I could change one thing about Monte Carlo, it would be for the platform to suggest some data quality rules by itself or some algorithms based on the anomalies and the patterns of our anomalies, which would be helpful, and also changes in our rules according to past anomaly patterns. I think that would be good, and they should also improve their support system, as I find it a bit weaker at the moment.

It's been six months since I've been working on Monte Carlo, and it's a really great tool for analyzing data quality anomalies.

Previously, the data checks were performed manually; they extracted data on Snowflake  and then did manual verifications on Excel using formulas.

It was not me who implemented Monte Carlo; it was another senior data analyst who implemented it a year ago, but I think it took a few months to get everything up and running.

It needed formal training because the tool is not that easy; if someone doesn't have a data analyst or business analyst background, you have to explain every rule which you have set by yourself, because the rules are created by us, not Monte Carlo; Monte Carlo is just a tool. We put our own rules to govern the data sets, and we literally had to make a guide to help users get to know that platform.

It was not me who implemented Monte Carlo; it was another senior data analyst who implemented it a year ago, but I think it took a few months to get everything up and running.

Monte Carlo saves me roughly 30% to 40% of my time in doing verifications or data quality checks, and related to my team, I'm the only one who uses it in my team, but we collaborate with another team that uses it.

I didn't ask that question in my company; it was a good choice, so it's a very popular tool, which is why my company picked Monte Carlo.

I would like to leave a review for the tool called Monte Carlo, which is a data quality analysis tool.

I think they just changed the layout a little bit, and during my work, I didn't see any changes in the platform except for the AI feature, which I don't use anymore.

Our final goal is to automate every data quality manual check into Monte Carlo so we don't have to do a lot of checks by ourselves.

If I find something bizarre, I go to Monte Carlo and see if it's happening there as well, so it gives me confirmation that the issue is occurring and we can contact our supplier or the country to verify what it is.

Monte Carlo is a really good tool; whenever I do data verification checks for the dashboards, and if I find something bizarre, I go to Monte Carlo and see if it's happening there as well, so it gives me confirmation that the issue is occurring and we can contact our supplier or the country to verify what it is. I would rate this product a 7 overall.

My main use case for Monte Carlo  starts with working on data quality related problems, but now it is a very necessary element for our whole data pipeline because it lets us know about the freshness and other data quality related metrics as well, covering all the data quality related metrics, including completeness, correctness, accuracy, and majorly, the freshness part of it.

A specific example of how Monte Carlo  helped me with a data quality issue is in our architecture, where we are getting all the data, storing all the historical snapshots in a raw layer, deduplicating it in the second layer, and curating it in the third layer, with alerts and monitors on all layers whenever we move from raw to dedupe or dedupe to curated, checking whether the number of IDs or the number of data is as expected or not, leading to alerts being triggered in the past for the orders table that helped us investigate pipeline related failures.

I often rely on Monte Carlo for our tables that get refreshed in three hours, where it sets up alarms or alerts if the tables do not get updated on their usual trend, such as triggering an alert if a table has not been updated for four or five hours.

Monte Carlo has positively impacted my organization by removing the need for manual monitoring of pipelines, automatically triggering alarms whenever something goes wrong, thus allowing the data engineering team to focus on more positive work, and helping us in manual tasks where after every logic change, we can track data and updates, along with proper catalog maintenance to find out specific information in our data warehouse.

Since using Monte Carlo, the freshness of our data has improved a lot from less than eighty percent to above ninety percent and there has been significant time saved, noting that while we do not keep a precise record of this, there is a steep decrease in time consumed on monitoring and related activities.

The best features Monte Carlo offers include an AI related trend analysis tool that checks the number of records of a certain table or the kinds of records affected by delete, insert, or update operations, triggering alerts if those numbers become unusual and providing a triage solution to investigate specific base tables or parent tables behind specific issues.

The AI trend analysis and triage solution have helped my team by alerting us during manual deletions or update activities and if there is a logic change in the main curated layer; if deletion rates deviate from expected numbers, we receive alerts that we may have messed up with the code, allowing us to check the code logics that we have implemented.

Regarding Monte Carlo's AI capabilities, it offers a tool that provides a mechanism to select or exclude specific parts of the data from the training cycle, allowing companies to adjust incorrect or ambiguous trends in the data, thus showing consideration for governance.

There are some improvements needed for Monte Carlo's code used for migration, which has not been set up well; improving documentation and migration features from other services, along with enhancing historical maintenance and version control on Monte Carlo's code, would greatly help.

In some cases, with multiple tables, the UI sometimes crashes, but it is still the best I have seen so far, making it a great tool overall.

I have been using Monte Carlo in my previous organization for about one year, and here in my current organization, I have been using it for one and a half years.

Monte Carlo is stable, with ongoing feature improvements; while there were initial breaking issues, they are fixed quickly when reported.

In terms of scalability, Monte Carlo handles our organization's multitude of tables and connections well, although there could be improvements in its implementation scalability, particularly with monitors as code.

Customer support was great, with dedicated resources from the Monte Carlo team who assisted with issues during our weekly calls, ensuring we understood specific features.

We previously used the Great Expectations library, which did not offer a solution like AI trend analysis and only provided basic data-based monitoring, lacking features that led us to switch to Monte Carlo.

In the beginning, I found that Monte Carlo took time to learn and understand the metrics and trends we have, but after six or seven months, it has shown great and accurate responses.

We have been tracking our return on investment, which has not been long, but we have saved significantly in time utilization of our resources and in capturing criticalities through this solution.

In terms of pricing, setup cost, and licensing, I rate it a bit high on the pricing side; it is pricey, but given the features and flexibility it offers during implementation, it stands out against specific libraries that are less handy to use, requiring extensive documentation.

Before choosing Monte Carlo, we evaluated Evidently AI , and our existing organizational ties to Monte Carlo influenced our decision-making.

My advice for others considering Monte Carlo is to assess whether their data platform is large enough to benefit from AI capabilities, as smaller scale industries with basic rule-based monitoring might find it a bit pricey. I would rate this review a nine out of ten.