Redpanda Cloud - Annual Commits (BYOC & Dedicated/Serverless)

I have worked with Redpanda  for the past two to three months. Mainly in the tech industry or software industry, there's a huge rise of streaming data. 

Redpanda  serves as a very reliable and fast message broker, which lets you build applications asynchronously. The major use case is for my project specifically, we're using it for a monitoring system that we're building.

I haven't worked on enterprise-grade applications. It's a project for academic research for the college.

The industry standard for this kind of platform is Kafka. Confluent  Kafka has acquired it. Kafka is an open-source platform built by Apache. Confluent  is the commercial version of it. 

The major improvement of Redpanda over Kafka is firstly, good documentation. Redpanda's documentation is very easily understandable, and they have a lot of examples. In addition to that, most of the setups include using another technology called Docker , which I am very familiar with. 

Setting up technologies using Docker  is very convenient to me, and Redpanda has provided many templates for that. Redpanda has its own built-in metrics exporter, making it easier to monitor and check performance. 

What makes Redpanda superior is its performance since it's written in C++. C++ is pretty much the standard for high-performance applications.

Recently, for the documentation, they've built their own AI chatbot, which is focused on giving you answers based on their documentation. While using that, I did not find it to be very good. Maybe due to the fact that it's a first prototype and was very recently released. However, from a product perspective, I do not have any problems.

I have used Redpanda for the past two to three months.

I am definitely satisfied with the stability provided by Redpanda.

In the free version, it's still working with containers itself. They've provided the template for one and three nodes for horizontal scaling. 

For our project, we've set up five containers. The template is ready in the Dockerfile they provided. You just add the existing nodes and give them the respective configurations.

I have not escalated any questions to technical support.

Positive

The industry standard for this kind of platform is Kafka. Confluent Kafka has acquired it. Kafka is an open-source platform built by Apache. Confluent is the commercial version of it. I have worked with Kafka, so I pretty much know how it works. I found Redpanda's documentation and setup to be more straightforward.

Once I learned about this platform, I read about it a bit and then I looked at how they set it up on their systems. It's a simple straightforward Dockerfile. If you have Docker installed on your system, you can just spin it up quickly without any issues.

Redpanda is actually a commercial platform, but they do provide free versions as well. I've been working only with the free versions.

Anyone finishing their bachelor's degree in computer science engineering would have had some hands-on experience with Kafka and understand how it works. Shifting from Kafka to Redpanda would be very simple for them.

I'd rate the solution ten out of ten.

On-premises

We handle high volumes of telemetry data and operate under stringent latency requirements, with our data pipeline demanding sub-second response times. Redpanda seamlessly integrates into our data plane, particularly as a message broker system, where performance is absolutely critical. Its low-latency capabilities and robust performance have been essential to meeting our operational demands

The cost savings have been significant. We were able to eliminate one MSK cluster, and still, the end-to-end lag hasn't been worse. The performance is superb, and the value we are getting for the money we pay is great. Additionally, peace of mind is also a significant benefit due to the high performance and fault tolerance.

The whole system itself is valuable. The cost and performance are the primary benefits. Redpanda  is extremely fast, which I describe as Kafka on steroids.

When it comes to self-hosting, their documentation could be improved. At the time we were onboarded, we had to apply our logic. Updating the documentation and managing the automation file for customer users to self-host would be beneficial.

I have been using Redpanda  for two years now.

Redpanda has been highly stable for our workloads. It utilizes the RAFT consensus algorithm, which ensures reliability by maintaining a quorum for data consistency and fault tolerance in cluster formation.

I would rate the scalability of Redpanda as very high, around nine out of ten. They support a feature called tiered storage to offload data to keep the systems running efficiently, which provides us with ample capacity.

They are really helpful. During our onboarding, they were very resourceful and knew what they were doing.

Positive

We previously used AWS  MSK. We switched because MSK was very costly.

The initial setup of Redpanda was straightforward and rated an eight out of ten. The setup involved using an Ansible  script to bootstrap the cluster, which took about 10-15 minutes per cluster.

The deployment was handled internally by one person with assistance from the provided Ansible  script.

Apart from cost savings, the high performance and low latency provide peace of mind and operational efficiency.

Redpanda is very cost-effective and offers competitive pricing compared to other options in the market. While not the lowest, the pricing is reasonable considering the high performance and value it delivers.

We considered Confluent  Cloud, but after reviewing our needs, Redpanda seemed like the perfect fit due to its superior performance and cost benefits. Given that, we decided to move forward without a formal evaluation of other options.

I would advise users to consider their in-house technical expertise when deciding the deployment model for Redpanda. If there is a lack of expertise, it is better to opt for the managed service offered by Redpanda.

Public Cloud

Amazon Web Services  (AWS)

The main use case is a message stream. Just to process the data with window processing.

Operationally, it is good.

Two projects use artificial intelligence in combination with Redpanda  right now.

The best thing is that it takes less resources compared to Kafka. We have a bad experience with Kafka; we need to restart the services because it's built on Java and requires a huge space on servers. 

Redpanda  is based on C++ and is native to the operating system. Basically, it is memory efficient. We like it.

It's easy to learn to use Redpanda for the first time because it supports the Kafka client as a developer. It supports native Kafka clients. 

And from the operations perspective, it's a pretty standard installation. The online community support is also available.

The command-line tools need to be improved. To quickly check the status of the topics and all. 

For example, if I wanted to check how many messages are in a topic, there is a command, but we need to use different tools to get that information. It's not really built into the native tool.

I have been using it for three years. I use version 23.3. 

I never experienced any bugs. So far, so good.

I tested it with ten-plus nodes, and it's highly scalable.

We are about 55 people, 12 people actively use it, and we use it for six products.

I never needed to contact the customer service and support because it's working as expected.

We opted for Kafka for two reasons. 

• One is we used to use Kafka, and we had to restart Kafka every quarter or so due to memory issues. 
• Compared to Kafka, Redpanda is stable and resource consumption is also very effective. It takes less resources compared to other tools.

It's very easy. The cluster setup is also very easy in Redpanda compared to others.

It's free. Everybody can use it, only support is paid.

We evaluated three: Kafka, Redpanda, and ActiveMQ .

I would recommend Redpanda to others because it's easy to set up, consumes less resources, and is stable compared to other tools.

Overall, I would rate it an eight out of ten because there is room for improvement. So far, it's nothing but a rewrite of Kafka. It needs improvement on some of the command-line tools. And the second thing is it doesn't have a management graphical user interface. That also needs improvement.

On-premises

Amazon Web Services  (AWS )

We use the tool for a simple use case. We use it for data streaming and data normalization. We receive a lot of messages from many different systems. We normalize them and highlight errors. We get 5000 to 6000 messages per minute, so we dump them in the database to handle the load. We use the tool for microservices.

The UI is modern. It is simple. We find what we are looking for.

The version control mechanism must be improved. Converting messages takes time. There is a version mechanism in the tool. It follows the coding standard, but sometimes, it gets confusing because I have to remove the older version to ensure the new version works fine.

I have been using the solution for more than a year. I am using the latest version of the tool.

The product is pretty stable.

We have around 20 users.

We used RabbitMQ before.

The setup is straightforward. The vendor takes care of everything. We just paid for the tool and started using it.

Redpanda is cheaper than its competitors. It is cheaper than Vanilla Kafka and Confluent.

We use the libraries in the different programming languages. We use the message format in Avro, convert the data in the format, and pass it on. We have contacted the sales team about the solution. So far, the product fulfills our requirements. We get lots of data. The data flows through Kafka to Redpanda. Then it goes to a database.

If our infrastructure goes down for a week, we will not lose data. It is an advantage of Redpanda. I recommend the product to others. We might choose a different solution if we find something cheaper. We used Redpanda because we did not want to self-host our software. Redpanda is a piece of infrastructure. There is not much of a learning curve.

Overall, I rate the tool an eight out of ten.

On-premises

What do you like best about the product?

Redpanda has been a powerful part of our infrastructure, it enables us to have the scalability and reliability that we need while being low touch around administration and maintenance. The Redpanda team has done as excellent job delivering a high quality product, as well as providing excellent support, guidance and continuous product and service improvements. BYOC has provided us the flexibility of running their managed service on our account, which combined with a very efficient Kafka platform, and their tooling and support, means we can focus our time and money on delivering even more value to our customers, while still having the flexibility of an onsite solution.

What do you dislike about the product?

If you're a Confluent or Open Source Kafka user, there will be some differences to adjust to with regards to running it yourself, as the underlying implementation is not the same, but I think overall an improvement for users in terms of management and configuration.

What problems is the product solving and how is that benefiting you?

Provides the backbone pipeline for our processing flows and connecting our various pieces of infrastructure together.