StreamSets is being used in the IT department to make sure that we have a stable solution and that our configuration is secure and running smoothly. We are using it for our data analytic tool as well as for real-time prediction for various real-life business use cases. It's helping us in generating new business ideas. It's a tool that allows us to share data between platforms, which also removes the dependency on other ETL tools, such as SSIS.
IBM StreamSets
IBM SoftwareExternal reviews
116 reviews
from
and
External reviews are not included in the AWS star rating for the product.
Review for StreamSets
What do you like best about the product?
There are various upsides of using StreamSets as Many users appreciate StreamSets for its user-friendly interface and intuitive design. The platform aims to simplify the process of designing and managing data pipelines. StreamSets is very useful in terms of Data Drift Handling, Scalability, Extensibility, Monitoring and Management, Community Support, Real-time Data Processing, Customer Support and in many more.
What do you dislike about the product?
THere are not such any significant downside of using StreamSets but there are a few features in which StreamSets is laking behind are as Documentation Quality, Limited Transformations, Community and Ecosystem Maturity, Resource Intensive.
What problems is the product solving and how is that benefiting you?
StreamSets solves the problem related to Real-time Data Processing as Traditional batch processing may not meet the requirements of applications that demand real-time data processing, StreamSets supports real-time data processing, enabling organizations to analyze and act upon data as it is generated. This is crucial for use cases requiring low-latency insights and decision-making. Along with this StreamSets also resolve problem related to Scalability as data volumes grow, scalability becomes a concern for traditional data integration solutions, StreamSets is designed to scale horizontally, accommodating large volumes of data and adapting to changing processing needs. This scalability ensures that the platform can handle increased data loads as organizations grow.
StreamSets data pipelines
What do you like best about the product?
StreamSets has lot of out of box features to use for data pipelines and connect AWS Kinesis, DB or Kafka and send to HDFS & Hive.
What do you dislike about the product?
Some of features like Aerospike connectors has deprecated and MapReduce has issues in control hub running in cluster out of Cloudera.
What problems is the product solving and how is that benefiting you?
Easy integration with big data tools and real time data ingestion.
Data transformation workflow in LCNC model
What do you like best about the product?
It is very easy with LCNC framework and every workflow creation is about drag and drop of the components. It's really reduce lot of developement efforts. Anyone can operate from Day1.
What do you dislike about the product?
SDC needs much more configurable parameters to maintain the buffer config etc. It's always must to have to tool tip on when needed and keep more guided at places in chosing the component
What problems is the product solving and how is that benefiting you?
Event framework and processing events in lightspeed is amazing and i would see streamset gives me a option to see my real time metrics gathered from multiple source got into visibility with lot of transformations.
Powerful and user-friendly data integration platform
What do you like best about the product?
The best feature of StreamSets is its intuitive visual interface, allowing us to effortlessly design, monitor, and manage data pipelines without the need for complex coding. This has significantly reduced our development time and made the process highly accessible to both technical and non-technical team members.
What do you dislike about the product?
Though StreamSets is an outstanding tool, one aspect that could be improved is the initial learning curve for new users. While the interface is user-friendly, understanding all the features and configurations may take some time for those unfamiliar with data integration platforms. However, the support documentation and community forum help to mitigate this issue to a large extent.
What problems is the product solving and how is that benefiting you?
StreamSets has been a game-changer for us, addressing several critical challenges in our data management process. Firstly, it simplifies data integration tasks through its intuitive visual interface, enabling both technical and non-technical team members to participate in designing, monitoring, and managing data pipelines. This has significantly reduced the learning curve and development time, improving our overall productivity.
Furthermore, StreamSets has helped us improve data quality and governance. Its monitoring and validation features allow us to track data quality metrics, identify anomalies, and ensure compliance with data privacy regulations and industry standards. By ensuring high-quality data, we can make more accurate and reliable business decisions.
Moreover, as our data volumes grow, StreamSets scales effortlessly, handling large-scale data processing without compromising on performance. This scalability has allowed us to handle increasing data demands and grow our business without worrying about data integration bottlenecks.
Furthermore, StreamSets has helped us improve data quality and governance. Its monitoring and validation features allow us to track data quality metrics, identify anomalies, and ensure compliance with data privacy regulations and industry standards. By ensuring high-quality data, we can make more accurate and reliable business decisions.
Moreover, as our data volumes grow, StreamSets scales effortlessly, handling large-scale data processing without compromising on performance. This scalability has allowed us to handle increasing data demands and grow our business without worrying about data integration bottlenecks.
It's lightweight and well-integrated, and it saves a lot of money and time
What is our primary use case?
How has it helped my organization?
StreamSets is straightforward to use for implementing batch, streaming, or ETL pipelines once you know how to use it. The pipeline can be integrated with Azure Key Vault, which eliminates the need of sharing credentials with developers. The same goes for parameters. It's very easy and straightforward.
It's easy for me to connect StreamSets to enterprise data stores such as OLTP databases and Hadoop, or messaging systems such as Kafka. I've got a good experience with it, and I've been working with it for a long time. It's very easy to connect and integrate for me. However, if you are a beginner, it might not go that well in the first step.
It's easy to move data into analytics platforms using StreamSets.
StreamSets enables us to build data pipelines without knowing how to code. We don't require the best coding skills. We can use the code-free environment to quickly create pipelines. It's very helpful for that.
StreamSets is a helpful tool for pipelines. It's very easy, so we can register data collectors to control hubs using provisioning agents.
StreamSets has helped to break down data silos within our organization. It hasn't negatively affected our business. It has fortunately enhanced our development time. We are able to develop secure, stable platforms faster and even remotely.
StreamSets has saved us a lot of time. It saved us the time that we were spending developing applications manually. One budget can be used by the team to come up with a stable solution. Our time savings are 30%. Out of five hours, it has saved us around two hours.
StreamSets has reduced our workload by 35%. It has also saved us money. When you subscribe to StreamSets, it seems very expensive, but when you get to know how their integration and documentation are and how things move, it's definitely efficient. It saves a lot of money. Before implementing it, we spent around 10,000 USD to hire experts. It has saved us 10,000 USD that we would have spent on hiring experts.
What is most valuable?
What I love the most is that StreamSets is very light. It's a containerized application. It's easy to use with Docker. If you are a large organization, it's very easy to use Kubernetes.
It has a very easy and user-friendly interface. It only takes a few days for new developers to start and deploy their first pipeline. It provides an easy and powerful integrated environment with different platforms such as Kafka, Salesforce, Oracle Database, REST API, etc. The user interface is a powerful feature of StreamSets.
What needs improvement?
There are so many things that need to be improved. For the StreamSets cloud user interface, there aren't enough use cases and examples for the main problems. In addition, the hybrid data sets cannot be joined in a data connector, which is a significant limitation.
There aren't enough hands-on labs, and debugging is also an issue because it takes a lot of time. Logs are not that clear when you are debugging, and you can only select a single source for a pipeline. It isn't helpful when you need to apply the same logic for multiple sources. It becomes difficult because you need to create more pipelines and then add coordination between them.
Initially, it's hard to find out or master the logic behind it. It can be hard if you aren't technical enough. There is scope for improvement because it's not straightforward. You need to go through the documentation and make sure that you understand every step. For me, it was a challenging model.
For how long have I used the solution?
I've been using StreamSets for two and a half years.
What do I think about the stability of the solution?
It's stable enough.
What do I think about the scalability of the solution?
It's good enough. We don't use it at multiple locations. We use it at one location, and it's being used by the IT and development departments. We have five users who are using it.
How are customer service and support?
Its deployment was hard. I had to contact them so that they could help me set things up. They are good people. They make sure that you are getting the best experience and that you are getting things in the right way. Their support is good and technical. I'd rate them a 10 out of 10 because of the fact that they were able to troubleshoot the issue.
How would you rate customer service and support?
Positive
Which solution did I use previously and why did I switch?
We did not use a different solution.
How was the initial setup?
In the beginning, it's very hard, but after reading the documentation, you can set up things easily. The documentation is very good and helpful.
For me, deployment was initially very hard because it required a lot of technical skills that I didn't have at that time. I had to contact the team, and they helped me with how to deploy it. The following day, I was able to set up everything. So, deployment is initially very hard, but after you become familiar with StreamSets, you can deploy it more easily.
What about the implementation team?
I deployed it myself. It doesn't require any maintenance because they take care of that.
What was our ROI?
There has been a great return on investment. We can use a single package of one thousand USD to have different applications with different people and different skills. It has saved us the money that we would have spent individually to develop those applications. Using StreamSets has saved us expenses. We have seen 40% ROI.
What's my experience with pricing, setup cost, and licensing?
It's not so favorable for small companies.
Which other solutions did I evaluate?
We didn't evaluate other options. We found StreamSets to be aligned with our expectations.
What other advice do I have?
To those evaluating this solution, I'd advise ensuring that they have someone who is an expert in StreamSets so that you can deploy it in less time. Otherwise, it won't be a great option.
I'd recommend StreamSets if you want to design a very good pipeline, but you also have to think about the budget. Its budget is not so favorable for small companies, but it's great software for businesses that want to create good data pipelines and have secure platforms. It will help your business in making sure that you are providing a stable solution to your clients.
Overall, I'd rate StreamSets a 10 out of 10.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Best Data Pipeline Building Platform
What do you like best about the product?
Stream Set is one of the leading Data Pipeline creating platforms and it is used by many tech giants also. Also, it is partnered with AWS, Snowflake, Google Cloud, and Azure. Which is very help full for Devops, Dataops and Data engineers. because it provides a comprehensive solution on one platform.
What do you dislike about the product?
I think it didn't have any downfall because the platform is so versatile. The only thing they can improve is by adding more regional servers around the world so that latency will reduce.
What problems is the product solving and how is that benefiting you?
I want to connect my Apache Kafka and Apache Nifi with data lake so I found this Platform and it really helped me, because of this amazing platform my work got complete in few click only.
Streamsets is a great product for dataops.
What do you like best about the product?
the ability to create a pipeline with with visual representation of the excecutions.
What do you dislike about the product?
this training provided is very basic and could be more specific.
What problems is the product solving and how is that benefiting you?
data engineering
Very Powerful and Easy Data Engineering platform. Capable to handle multiple platform and huge data.
What do you like best about the product?
StreamSets is very light. Since it is containerized app, it is easy to use with Docker if you are an individual developer. For organizations they can use Kubernetes.
They have a very easy and user-friendly user interface. It takes only a few days for new developers to start and deploy their first pipelines.
StreamSets provides easy and powerful stages(kind of connectors) to integrate StreamSets with different platforms such as Kafka, SalesForce, Oracle DB, Rest API, HTTPS connection, Data lakes and many more.
StreamSets uses regex expression for data transformation related operation which is really easy.
Monitoring StreamSets pipelines are very easy, you can register your Data collector to control hub using provisioning agents. After registering you can deploy pipelines to SCH and create jobs. All of this can be done using their Python SDK which can easily be integrated with ADO release pipelines.
After creating/deploying pipelines users can use SCH subscription to create alerts if pipelines/jobs changes their status.
For individual alerts pipeline have built-in capability to do so.
After their version 4.0.1 , sdc are merged with their data ops platform. This allows individual developers to have the feel of a Control Hub. It also remove platform dependancy.
They have very excellent security. Pipeline can be integrated with Azure Keyvaults which eliminates the needs of sharing credentials with Developers. Same goes for parametrs and runtime parameter. Developers can easily replace any value in pipeline with ADO library variables.
If you are an Organization they provide very extensive support, work instantly on any bug if found by an organization. They also have customer success team which will do anything to make sure your organisation's experience with StreamSets is seamless.
They have a very easy and user-friendly user interface. It takes only a few days for new developers to start and deploy their first pipelines.
StreamSets provides easy and powerful stages(kind of connectors) to integrate StreamSets with different platforms such as Kafka, SalesForce, Oracle DB, Rest API, HTTPS connection, Data lakes and many more.
StreamSets uses regex expression for data transformation related operation which is really easy.
Monitoring StreamSets pipelines are very easy, you can register your Data collector to control hub using provisioning agents. After registering you can deploy pipelines to SCH and create jobs. All of this can be done using their Python SDK which can easily be integrated with ADO release pipelines.
After creating/deploying pipelines users can use SCH subscription to create alerts if pipelines/jobs changes their status.
For individual alerts pipeline have built-in capability to do so.
After their version 4.0.1 , sdc are merged with their data ops platform. This allows individual developers to have the feel of a Control Hub. It also remove platform dependancy.
They have very excellent security. Pipeline can be integrated with Azure Keyvaults which eliminates the needs of sharing credentials with Developers. Same goes for parametrs and runtime parameter. Developers can easily replace any value in pipeline with ADO library variables.
If you are an Organization they provide very extensive support, work instantly on any bug if found by an organization. They also have customer success team which will do anything to make sure your organisation's experience with StreamSets is seamless.
What do you dislike about the product?
A few of the stages are a bit unstable. Like Oracle CDC client. They work fine but in some corner case scenario, it becomes a bit tricky. Logging mechanism is excellent and extensive but it could be simpler.
What problems is the product solving and how is that benefiting you?
I am in an organization where we are working on sharing Data between mutiple application running on different platoform. So we needed a tool/platform with can easily integrate with variety of technology and can adopt with this everchanging era.
StreamSets allowed us to share real time data between platfoms which also removed dependancy from heavier ETL tools like SSIS, Abinitio.
Since it is easier which allows our talent developement team enable our developers to use StreamSets.
StreamSets allowed us to share real time data between platfoms which also removed dependancy from heavier ETL tools like SSIS, Abinitio.
Since it is easier which allows our talent developement team enable our developers to use StreamSets.
Data Migration cross RDBMS and NO-SQL become very easy.
What do you like best about the product?
I found it very flexible and GUI-based configuration makes it very user-friendly.
What do you dislike about the product?
So good so far, didn't find anything wrong about streamsets as of now.
What problems is the product solving and how is that benefiting you?
Data Migration from RDBMS to RDBMS and RDBMS to NO-SQL.
By using StreamSets I am able to migrate data without any downtime and without any help from DBA. in the traditional way we were doing import and export for RDBMS to RDBMS which is not now needed. from RDBMS to NO-SQL I was using custom scripts to export data in CSV from Oracle and import it in Cassandra but now I have created a pipeline and all work is sorted now.
By using StreamSets I am able to migrate data without any downtime and without any help from DBA. in the traditional way we were doing import and export for RDBMS to RDBMS which is not now needed. from RDBMS to NO-SQL I was using custom scripts to export data in CSV from Oracle and import it in Cassandra but now I have created a pipeline and all work is sorted now.
Lead Data Engineer
What do you like best about the product?
The development speed for a Spark Application.
What do you dislike about the product?
The control hub must be available as part of trail version, with minimal feature
What problems is the product solving and how is that benefiting you?
Convert Spark coding into drag and dropable UI
Recommendations to others considering the product:
If you want to exploit the full power of Apache Spark and maintain it easily then Streamsets in the best way to do it.
showing 11 - 20