Monte Carlo Data + AI Observability Platform

My organization relies on Monte Carlo  for data observability, such as whether tables were loaded on time and whether the load met expectations regarding volume. We also use it for observability into our data transformation pipelines. We use it to trigger alerts to our respective product and engineering teams if loads are delayed or the volume of loads does not meet expectations. In essence, we use Monte Carlo to gain observability into our data.

We have encountered scenarios where a particular data load would not have generated any alerts if Monte Carlo had not been in place, since the data load went through successfully. However, the volume of the data loads in that case was below the threshold volume that triggered a volume anomaly alert from Monte Carlo. We were able to go back, fix the data, and report it to the upstream source.

We also use Monte Carlo to catch long-running queries. We have monitors set up for that purpose as well. We have an in-house solution developed to catch long-running queries on Snowflake  in real time, something that is not currently available in any other SaaS providers for Snowflake .

The volume monitors and the anomaly monitors regarding volume, freshness, and data consistency are the best features Monte Carlo offers in my experience. These are the best features because they help flag issues that are more abstract and difficult to measure. While a data load that did not happen is an easier thing to track, a data load that happened but the volume was not in a particular range is a very tricky metric to monitor. That is a great feature.

It is mostly a combination of volume, freshness, and consistency monitors that I find myself relying on the most. The specific monitors I use depend on the business and use case we are catering to, the tables, and the data involved. It is difficult to point out one monitor as the most useful, but we use all three of them in different combinations very extensively.

Monte Carlo has had a major impact on my organization in terms of data health for the downstreams and for all the engineering teams that depend on our data. We are now getting timely alerts around the quality of the data, the volume of the data, and the health of the data. We are able to get that visibility more granularly into every single table. We are able to draw the data lineage to understand failures faster. Overall, Monte Carlo has had a very positive impact in terms of having healthier data and being able to trace through the data lineage to understand where exactly in the data life cycle things are going wrong.

Monte Carlo needs to stop their reliance on AI, as it is not going well and is degrading the entire product. They need to find their way back, establish a product roadmap, and have real engineers work on improvements rather than heavily push AI down users' throats. They need to stop relying on AI as heavily as they have been doing, as this has really degraded the user experience. The overall direction they are taking with AI needs to be examined, as at some point it seems they have simply stopped making any improvements.

We have not used Monte Carlo's AI capabilities significantly. We primarily use it for investigating alerts from time to time. However, we do not use it extensively, so I do not think it is fair to comment comprehensively on it.

Their incident tracking and incident debugging bot is useful for new analysts who are starting onboard. It helps them debug incidents, get a clearer picture, and achieve a clear head start to reach the root of the problem faster. Regarding accuracy and reliability, I would rate it at eighty to eighty-five percent. Given the current inherent non-reliability of AI models, every single thing that Monte Carlo says needs to be validated.

I have been using Monte Carlo  for the last three years.

Monte Carlo is a fairly stable product.

Monte Carlo is robust and scalable for our data needs. We have not encountered any issues or challenges with the scalable platform.

We did not previously use any other solutions.

Time has been saved in reporting errors, SLAs, and performing reloads because we have been able to catch data errors faster. We estimate approximately eight to ten percent time savings, but regarding money savings and fewer employees needed, I do not think we can achieve that.

We have seen a reduction of incidents in approximately seven to eight percent in production scenarios, which has definitely been positive. I recommend checking out Monte Carlo to see if it fits your data-related needs. Conduct a thorough proof of concept, review the licensing and contract agreements, and if it meets your requirements, proceed with it. I would rate this review at seven out of ten.

My main use case for Monte Carlo  is to create monitors to monitor data quality. I created different types of monitors, metrics, SQL, and custom SQL. I also utilize the Monte Carlo  API to ingest data and create a datamart for analytical purposes.

Recently, I used the Monte Carlo API to ingest data to create a datamart, where I created several tables, such as the monitor snapshot, alerts, the run monitor history, and the monitor credit consumption.

Other use cases I have for Monte Carlo include creating a monitor, where I can write custom SQL to monitor data quality. If the data volume fluctuates dramatically, either increasing or decreasing too much, it will fire an alert.

Monte Carlo offers several great features that stand out to me. Monte Carlo's use of machine learning and AI to determine the threshold for the monitors that trigger alerts is particularly impressive.

Recently, Monte Carlo added features based on AI that respond to alerts. When you see an alert, the AI can help you analyze it and find the root cause.

Monte Carlo has positively impacted my organization by ensuring the data quality of the data pipeline, which is essential for data-driven decision-making. The company's business depends on data-driven decision-making, so data quality is very important. Monte Carlo monitors data quality issues and helps identify and fix those issues efficiently.

I estimate that I resolve data issues roughly 30% more efficiently since using Monte Carlo.

Monte Carlo adopted AI just recently, so there is room for improvement in the accuracy of the AI.

I have been using Monte Carlo for two years.

Monte Carlo is stable.

Monte Carlo's scalability is good. As our company's business grows and the data volume increases, Monte Carlo scales very well.

I find Monte Carlo's customer support to be good, and their customer support team responds very fast. We can submit a support ticket, and they also provide a platform using AI to answer questions. If the AI cannot resolve the issue, a ticket is created, and a team member will help.

I did not use a different solution before Monte Carlo.

My experience with pricing, setup costs, and licensing is limited as that falls under the management team's responsibility.

I am not responsible for how Monte Carlo is deployed in my organization. I believe the platform team is responsible for that.

I do not have a specific number regarding the return on investment with Monte Carlo, but I believe it saves a lot of workload for employees if we do not use it.

My experience with pricing, setup costs, and licensing is limited as that falls under the management team's responsibility.

I did not evaluate other options before choosing Monte Carlo. When I joined the company, we were already using it.

Monte Carlo does a good job overall. I rate Monte Carlo an 8 out of 10.

I chose eight because Monte Carlo does a good job, even though there are places for improvement. Overall, it is good.

Regarding Monte Carlo's AI capabilities, I find its accuracy and reliability of output to be good. Even though it is not 100% accurate, it is still good. A human review is necessary, but it does a good job.

My advice to others looking into using Monte Carlo is that it is a good product.

Overall, I think Monte Carlo is good, and the questions cover almost all the aspects. My overall review rating for Monte Carlo is 8.

My use case for Monte Carlo  is both data quality and observability. We are using it as part of robustly monitoring the jobs as well as finding out any anomalies with respect to data quality issues.

In Monte Carlo , as part of observability, we have dynamic alert systems that learn the previous patterns of data anomalies and customize the monitoring system. It does not only have static rules because it has machine learning based models that learn the patterns. For example, during Thanksgiving, more purchases are happening, so you can expect more issues. It learns those patterns and sends the alerts based on that. The system does not send false alerts.

We use anomaly detection as part of a monitoring system. For instance, I was working for an airline where daily check-ins, checkouts, and transactions happen in real time. We wanted a very robust monitoring system that could monitor the data in real time. Whenever there is an anomaly, such as some columns which are not supposed to have nulls or which are not supposed to have certain data, you can train your machine learning model to have that threshold. You cannot just keep that threshold at 10% or something. You can train that machine learning model so that whenever a null detection happens or some kind of data mismatch happens, or when there is a schema change, it detects so many anomalies. We had many anomaly detection alerts.

The customizable alerts and dashboards in Monte Carlo were very customizable because it not only gives you the option to select alert features using drop downs, it also opens up a window where you can write your own customizable queries in SQL.

Certain features of Monte Carlo have contributed to maintaining our data trust by having multiple steps where you can define the model and also specify the probability distribution for your input. You can simulate that model over the past pattern. Additionally, it will give nice dashboards which are very handy and easy to understand to check how the anomaly patterns are progressing. If there is any sudden spike on a particular day, you can easily spot that and dig deep into it.

Regarding Monte Carlo, I would say that currently we can have machine learning options. We might have to integrate MCP servers so that it can connect to multiple systems at once and we should have some kind of a placeholder for artificial intelligence integration. Artificial intelligence can access multiple systems underneath Monte Carlo, such as any kind of database or any kind of real-time source systems. Currently, I think it is lacking that capability.

I have been using the solution since 2024, which is around more than two years.

The stability was very stable. I did not see any issues with respect to stability, and I would rate it a ten.

The scalability was good because when we enrolled it, it was already scaled up. We did not require it to be scaled up again, so I cannot fully comment on that.

I rate the technical support around nine out of ten because they are pretty responsive.

Data quality monitoring throughout the data lifecycle is very important, especially in this artificial intelligence era. If you feed garbage into artificial intelligence, it will hallucinate more and will not give you accurate results. It might divert into deploying many more agents and utilizing many more tokens rather than confining to a particular set of tokens. It is not only important from your data perspective, but also very important from your revenue perspective. The lost tokens are directly impacting an increase in costs or a decrease in revenue.

Regarding the pricing, it is a bit expensive compared to traditional monitoring systems provided by other vendors. However, the extra features and the trust come with some cost, so I think it should be fine. I have worked with many customers who do not have any complaints. In fact, they migrated many other systems from traditional monitoring systems to Monte Carlo. The customers are accepting of this pricing model.

Monte Carlo has many advantages compared to other solutions. As I mentioned, it has a lot of machine learning functionality and excellent user friendliness. The interface is quite crisp and the appearance is quite good. Traditional tools require some prior knowledge, but with Monte Carlo, you can onboard any user at any time. They can easily understand how to use that tool.

The solution requires maintenance because new features get rolled out and you need to upgrade those features. During that time there is a little bit of a pain point, but that is acceptable because you will experience new functionality.

If others are looking to implement this product, my advice is to robustly monitor their system with very little human intervention. Monte Carlo has an option where it will directly allow you to dig deep into the root cause and you just need to do a few clicks and it will get you to that data issue where it is happening. Very little human intervention is required for this. I give this solution an overall rating of eight out of ten.