In my company, we are not using the tool for analytics and it is more for CDC processes, so we change the capture processes. It is used to extract data from a database and make it available in other parts of our systems or produce events that inform us of data updates.
External reviews
External reviews are not included in the AWS star rating for the product.
Works well in areas like maintenance and issue resolution
What is our primary use case?
What needs improvement?
There are some premium connectors, for example, available in Confluent, which you cannot access in the marketplace, so there are some limitations. From Confluent's point of view, I understand where they come from, but I believe its deployment model is a little strict. Confluent Platform can be installed in your own infrastructure. Even if you install Confluent Platform on your own platform, you need to use the components that Confluent offers. Otherwise, the support is very limited, and I think this is an idea of improvement for Confluent. Confluent is pretty solid, so I don't have much in terms of improvement.
For how long have I used the solution?
I have experience with Apache Kafka on Confluent Cloud.
What do I think about the stability of the solution?
Confluent gives you 99.99 percent availability, so I rate the tool's stability a nine out of ten.
What do I think about the scalability of the solution?
If you use Confluent Cloud, the platform's case can go up and down depending on your needs, and it is very easy from the point of view of storage as well because if you are getting more advanced, it basically scales up your storage. If you are given a number of events using your storage device, it is very easy. If you use Confluent Platform, you have a little bit more manual management there, although being a product that assists you with some side components like CFK.
How are customer service and support?
With Confluent, if you have its tools, I rate the support an eight out of ten, but if you have mixed products, then it is a six out of ten.
How would you rate customer service and support?
Neutral
Which solution did I use previously and why did I switch?
I have experience with Confluent Cloud and Amazon MSK. With Confluent Cloud, we are really happy with the ecosystem that is made available, along with the connectors, SQL, DB, and other such aspects. The tool can be provided in a very easy way, and it was really effective for the type of activities that we do. The tool presents quite a range of possibilities for integration between different sources and things.
While you use Confluent, all of the services that are needed to manage the enterprise-level EDA are available to you, and you have an integrated schema registry, together with the entire schema registry, and you have a portal for publishing your schema. You can do routing and filtering by configuration. You have CFK, which allows management of your cluster, allows monitoring of your cluster, and allows you basically to connect to the managed connectors on your cluster. Confluent is a full-fledged platform for an event-driven architecture that can be deployed at an enterprise scale, while Amazon MSK is just Kafka as a service from AWS.
How was the initial setup?
A part of the delivery team does the setup, but it was pretty easy on both sides, as with AWS and Confluent, the team didn't have much trouble.
What was our ROI?
The main return on investment was in the maintenance space because going for Confluent Cloud means you remove all the platform management that you have in terms of these resources that can be allocated to other tasks, where the tool takes basically ownership of all of these. We saw, at the end of the year's end, improvements that were substantial, especially when it comes to the need to resolve issues, as we can deploy the minimum team possible for Confluent because the support model allows for the Confluent team to take ownership of the issue. With AWS, the tool's team supports us, but we have to deploy the right people and take them out of all other initiatives. The most important part is the cost related to platform maintenance and issue resolution.
What's my experience with pricing, setup cost, and licensing?
Using Confluent, you have more licensing prices to account for when you calculate. I think the pricing is fair, but Confluent requires a little bit more thinking because the price can go up really quickly when it comes to premium connectors.
What other advice do I have?
Speaking about data security and privacy requirements, I would say that there are some BAA or legal agreements in the tool. We did not have issues in terms of security or breaches, but before any adoption, with PII or PHI type of data and before having this data flowing to other clouds or other platforms, the BAA needs to be signed because of IPAC.
Confluent Cloud handles data volume pretty well.
If you are starting to deploy a fully-fledged ETA platform where you do not just have information streaming and go for CDC, and you have some legacy systems that have to communicate on your systems, then I suggest you go for Confluent Cloud.
I rate the tool an eight out of ten.
Helps us manage transactions effectively and integrates seamlessly with our data analysis tools
What is our primary use case?
We use Apache Kafka with Confluent Cloud for specific real-time transaction use cases, both on-premise and in the cloud. We have been using Confluent Cloud for about five years.
We initially used it for data reputation, then expanded to microservices integration and Kubernetes, focusing on improving data quality and enabling real-time location tracking.
We configure it for data transactions across various topics and partitions, depending on the specific use case and required throughput.
From an IT perspective, I've used this product across all domains: system development, operations, data management, and system quality.
How has it helped my organization?
We have experience using Kafka on Confluent Cloud for data pipelines. We've implemented several techniques to optimize topic usage, integrated it with microservices, and even utilized change data capture (CDC) components.
What is most valuable?
We leverage topic configurations and partitions extensively. We simulate various use cases with different partition numbers, like high throughput scenarios with 45 partitions or high transaction environments with other configurations.
In our microservices architecture running on Kubernetes, Confluent Cloud helps us manage transactions effectively. Additionally, it integrates seamlessly with our data analysis tools like DataStage, Big Data, and Teradata, providing a smooth flow for large data volumes.
The overall integration with other tools and efficient transaction management are the key benefits I experience with Confluent Cloud for large-scale data streams.
What needs improvement?
I saw an interesting improvement related to the analytics environment.
For how long have I used the solution?
We have been using this solution since 2018.
What do I think about the scalability of the solution?
We have a well-defined process and platform for scaling big data solutions. When multiple providers propose their options, we configure a custom platform based on our current use cases.
However, we're planning to migrate to a new big data platform within the next fifteen months. This timeframe is due to our internal process for evaluating and deploying new platforms.
How was the initial setup?
In terms of configuring the product, specifically Confluent, understanding the design and configuring values for various parameters is something only I am familiar with. The initial setup, including the initial Non-Disclosure Agreement (NDA) and progress in implementation, is quite difficult.
We primarily use on-premises Kafka for high-transaction scenarios. If something crashes there, we handle data processing manually. It might not be the most efficient, but we haven't considered it a major concern.
For other use cases, we also prefer on-premises.
The implementation took us one year. It involved configuring the platform over a year. The time required for configuring or implementing use cases varies; some take longer, while others might also take up to a year.
What about the implementation team?
I attempted the deployment myself. However, there were three of us involved in these tasks within this analytical environment.
My role revolves around deploying use cases in analytics. I also operate within Architect areas, focusing on data architecture.
For maintenance, the same three people take care of it. We might need two more, but for now, three is sufficient.
What was our ROI?
The platform and container operations themselves provided significant value.
Senior Software Engineer
"A Game-Changer for Real-Time Data Management"
Flexibility of use
Product is good to find suitable software needed for user.
Heads on woth native Kafka
Easy to setup and do configurations very fast
The Best Of Data Streaming Platform
Unified Platform
Apache Kafka Integration
Scalability & Performance
Streaming capabilities