Grafana Cloud observability: Grafana, Prometheus metrics, logs, traces

For my personal usage, I do use Grafana , and it does make life easier for solving stuff.

The main benefits I have seen from using Grafana  in my day-to-day activities is the visualization of the metrics, specifically Dora Metrics. Instead of saying 'I feel the team is developing bad or slow,' you can actually have metrics to support that. You can actually show that this team could work better here, in this case, or needs to focus more on peer reviews or on deployments. The visualization of the data is key.

For my personal usage of Grafana, I think there could be improvements, particularly that I use the Grafana Cloud SAS solution. The integration guides could be simpler and easier.

Regarding the clarity of the official documentation for installation, I think the official documentation, which has something called Alloy, the Alloy integration, is not that clear. It was mostly for personal usage that I'm talking about right now.

I have been using Grafana for around a year, maybe more.

Grafana is stable; however, in the previous company, we also had Grafana for a long period of time, and its stability depends on the load. Once you get to a higher load, you need to re-evaluate your architecture and put that into account.

In assessing Grafana's scalability, we started noticing logs missing or metrics not syncing in time, which indicated a bottleneck that we had to address by updating the infrastructure.

I'm not familiar with Grafana's feature in terms of community and documentation since I've been here only a few months. The community provides extensive knowledge about Grafana. You will get faster answers about Grafana than about Datadog  from the community.

My advice for people who are new to Grafana or considering it is to reach out to the community mainly, as that's the primary benefit of Grafana.

Positive

The reasons for moving from Grafana to Datadog  include the fact that Datadog is a SAS solution, and we don't need to manage it. I was told, and as I mentioned I'm kind of new, there was already Datadog running, so it's not creating a new one but just moving the data. The main benefit of Datadog versus Grafana is the ease of creating a dashboard, which is just simple for the user.

In the previous company, about three or four people worked on Grafana implementation. It was an ongoing effort that took several months to complete.

Grafana is easier to sell to management compared to Datadog and other solutions because it has an open-source tier, which makes it easier.

If I'm coming as a personal user, then Grafana is the best solution out there right now. But if we're looking at it as an enterprise, then when money is less of an issue, Datadog is much easier to use and implement than Grafana.

Managing and maintaining Grafana is kind of managing another application; as far as Grafana itself, you don't have much to do. It's mostly about the infrastructure, resources, and if you're deploying it in high availability, maybe more instances. So it's mostly infrastructure work and less about Grafana itself.

I rate Grafana an eight out of ten.

Public Cloud

Other

We manage log processing with Grafana  because we found that it is much easier for us to manage it on our infrastructure on AWS . We can maintain all the things we do not need. DataDog told us we have to wait and that they need to provide features we need to develop, so it does not suit our needs.

We are still using DataDog, but for important assets that we need to analyze the logs, we send it to Grafana .

The challenges we face with DataDog compared to Grafana include the need to analyze very important brands, network trafficking, and maintaining many websites, most of which are very important domains that cost a lot of money, so we are getting attacked each day and we need to analyze all of the logs. Sometimes we have false positives and things similar to that, so we have to make sure that we are doing the correct decision of blocking or trying to mitigate attacks. Using the logs with Grafana it is much easier for us to analyze rather than DataDog. DataDog has their own language and they want you to plot things with their own vocabulary. We do not have time to memorize things. We especially wanted to use something that was open source at the beginning, and then other people started using it, took that product and modified it for extra cost, but it is a better solution for us.

We switched from DataDog to Grafana because we wanted to reduce the logs costs, as we are streaming approximately five million logs or even less.

We can find information with Grafana much more quickly compared to DataDog because it was open source and there was extensive documentation about it, enabling us to fetch data or information much more quickly using AI tools. With DataDog, they always wanted us to have a meeting or talk with us on a call. It was redundant. We just wanted to get to the solution without making a big deal out of it.

Grafana saves us hours compared to DataDog. It takes about two weeks to figure out what is going on with DataDog, but with Grafana, we just started to initialize the service, had a few issues, fixed them, and that was it. I did not have any major problems that forced me to halt everything in my work. It cost me hundreds of hours with DataDog because I needed to see all the documentation and all the special caveats they have there.

I would rate Grafana overall as an eight out of ten. It is pretty good, and I would recommend it. I would give it a ten if it were much simpler for users who just want to get a simple objective in Grafana and are not experienced with technical configuration. It would be better if users could simply state they want to see the amount of requests on a graph through an AI implementation. For example, when getting attacked, users should be able to easily filter all requests to a specific site or resource, or identify IPs that were recently attempting access.

I would describe Grafana's ease of use as a necessity to just get things fixed. We are working in a marketing company, and if something does not work great, or people are looking for the solution, I am looking to get a temporary solution until I can fix it properly, and then I can reach support if I cannot fix it for the long-term.

I have been using Grafana for quite a long time, but we have only recently started using all of its features.

I would rate Grafana's overall stability as much more stable because when it is not working, users are given a much broader oversight of what is not working, rather than talking to DataDog and asking them to check it out. When something in their dashboard does not work, because it is open source, I am able to find all the relative combinations that people are having, making it much easier for me to fix.

The size of our infrastructure affects using Grafana as it really depends on that. We are considered quite small or medium at least, so it is quite easy for us. In terms of our company, the infrastructure is using two availability zones in AWS , one is US East, so we are looking to expand. Currently, we have only two availability zones to deploy, so it is quite easy for us right now.

I do not use Grafana's support for technical issues because I have found solutions on Stack Overflow and ChatGPT  helps me as well.

On a day-to-day basis with Grafana, I tend to refrain from using their support, not because of their level of professionalism.

Positive

We switched from DataDog to Grafana because we wanted to reduce the logs costs, as we are streaming approximately five million logs or even less.

I initially learned to use Grafana by working locally and seeing how it works and presenting a proof of concept. We started to make a small plan of how we could make the change, how to deploy the switch, making the changes and checking it in other environments that we have. We are working with Agile, so I have to work with development first and then we are staging it. It is a long process but eventually we made the switch quite quickly, it was just a simple day when we deployed it.

Three people manage Grafana in our organization: one is a sysadmin, another one is a DevOps full-time, and I am managing the entire operation.

The costs associated with using Grafana are somewhere in the ten thousands because we are able to control the logs in a more efficient way to reduce it. That is pretty much great for us.

My recommendation for future users of Grafana is that it is all great. I am hoping that all the companies, at least the open-source projects, do not go closed source because then users will have to find something else. That is what we love to do - we love to use open source projects and improve them for our usage, not something that follows an agenda of another company's product.

My recommendation to other users of Grafana is to not be afraid and always look online. I started using it a long time ago for different projects. Users need to learn about the basics before they can really understand what is going on. There are people who start the self-hosted web server and see all the metrics going to their server, but they do not really understand what is happening, they just see a small count of the graph. Users need to understand that configuration is necessary. I experienced this when I started and was clueless, thinking of going to DataDog. Then I realized I just did not understand it correctly - there is much more once you see a bigger horizon of things that you could do.

Overall rating: 8/10

Public Cloud

Amazon Web Services (AWS)

My main use case is monitoring. We use a lot of different products for monitoring, but Grafana  is specifically for monitoring Kubernetes . We use Grafana  mainly for Prometheus.

The features I appreciate most are the dashboards and the integrations with multiple data sources. The feature that sets Grafana apart from its competitors is how easy it is to set up data sources. The integration helps our organization in centralizing and analyzing data from diverse sources.

Regarding joining between queries, merging between two queries that give the same information could be simple, and there are some ways to do that, but if there was something even easier, it would be great.

The deployment process was completed before I joined the organization. We have been using it for a couple of years, but it was deployed previously.

We never had any issues with Grafana at all.

Grafana's customer support is mainly for developers. We didn't need to reach out to them for troubleshooting or any issues.

Positive

We centralize all the metrics from Prometheus and also from Graphite  and all other data sources. We have dashboards to integrate Grafana's real-time metrics with visualization capabilities. We're not using Grafana's role-based access control and multi-tenancy features.

Seeing the metrics helps in finding issues, such as memory leaks or spikes and some optimization. We use Grafana on a day-to-day basis to get a better look at our environments and the usage of our resources.

We don't use Grafana for alerting, just for visualization. For alerting, we have different tools. On a scale of 1-10, I would rate Grafana as 8.5.

Public Cloud

Amazon Web Services  (AWS )

We actually work with BMC and similar solutions, and while I wouldn't say that they are the same as a full, fully-fledged APM , we're working our way up using open source and everything, so I don't know how far we'll get with that. We're a government agency, and although these costly tools are very efficient and useful, it's hard for me to convince the organization about that, especially since we're not allowed to use public cloud solutions, so we have an on-prem solution, which makes us a demanding customer.We use BMC Helix with the AIOps  solution, and while all of these are public domain knowledge because we have to be open with how we procure and purchase our services, I am not the one who would purchase such a system. I would be the customer to it and someone else would host it, while I work more with observability, such as the Grafana  stack and the cloud-native solutions.We're working with Grafana  more actively now for maybe a couple of years, and we are looking at Tempo and the tracing solutions to see if it will scale. We have metrics back-ends and everything, and we're basically late-stage discovery, early-stage implementation at large scale, with 1,400 people in our tech organization and a 14,000 people organization, where the technology side is just 10%.We get alerts from the alerting tools, and we act on them. I'm looking now at logs, Grafana, trace profiles, and while we're not yet into AI/ML and root-cause analysis, we do focus on alerting and, to some extent, plugins and app observability, mainly utilizing the basics for now.

We use Tempo for tracing and, of course, alerting and monitoring metrics. We've tested some features but we're only using a few of them actually, so I can't provide the full range of what different tools Grafana offers as an open-source project, but it's definitely useful for monitoring, alerting, logs, and analysis.Grafana integrates with some of the tools we've chosen beforehand, and while I'm not the right person to elaborate on this since I'm more focused on managing big teams, the teams working with it seem happy. We used to have performance issues, which could be due to running it in an OpenShift environment, but functionality-wise, it seems to be useful, albeit not offering the same power tools as APM  such as AppDynamics.

The whole AI capability would be useful for Grafana in the future, and while I don't think we're mature yet, I would want to see improvements, especially in the tracing part, where following different requests between different services could be more powerful.

Depending on how you define use, whether it's in production or just dabbling with it, I would say that using it actively with a subset of the team has been for maybe two years.

You have to scale horizontally to deal with issues.

We might have gotten help from technical support, but I'm not sure how much since I haven't seen an invoice for purchasing support, so it's probably community open-source style.

Neutral

I would probably recommend Grafana for monitoring and analytics tasks, but since I haven't really pushed it to the limits from a non-functional perspective, I would hesitate if someone from a high-scale Telco environment asked for my recommendation.I rate Grafana a six out of ten.

On-premises

Other

I use Grafana  mostly for monitoring purposes, such as tracking errors or alerts. When certain thresholds of warnings or errors are breached, Grafana  sends me an alert because I am supplying it with data. Another use case I have is monitoring CPU usage and bandwidth.

I find Grafana beneficial due to its numerous plugins. Its alerting feature is effective because it allows me to set thresholds to send an email if a certain threshold is met. User management is good, and the administration features enable me to control user access.

The interface is not well-liked by my customers. Grafana cannot be easily embedded into certain applications and offers limited customization options for graphs. I think predictive capabilities in alerting could also be improved.

Scalability is uncertain for me since I installed Grafana on only one server and did not scale it further.

The setup process was easy for me to follow. I just needed to adhere to a step-by-step guide which was quite straightforward.

I end up using my own customized open-source visualization tool instead of Grafana. Most of the time, I deal with technical aspects. 

I rate the overall product seven out of ten.

On-premises