StreamSets is used for data transformation rather than ETL processes. It focuses on transforming data directly from sources without handling the extraction part of the process. The transformed data is loaded into Amazon Redshift or other data warehousing solutions.
IBM StreamSets
IBM SoftwareExternal reviews
116 reviews
from
and
External reviews are not included in the AWS star rating for the product.
Useful for data transformation and helps with column encryption
What is our primary use case?
What is most valuable?
The best thing about StreamSets is its plugins, which are very useful and work well with almost every data source. It's also easy to use, especially if you're comfortable with SQL. You can customize it to do what you need. Many other tools have started to use features similar to those introduced by StreamSets, like automated workflows that are easy to set up.
What needs improvement?
We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which was painful. Also, pipeline failures were common, and data drifting wasn't addressed, which made things worse. Licensing was another issue we encountered.
For how long have I used the solution?
I have been working with the product for five years.
What do I think about the scalability of the solution?
The tool's flexibility and performance are good. It allows for task dependency management so others won't be affected if one task fails. It can handle large volumes of data and supports features like change data capture for tracking changes.
Around six months ago, many people in my company were using StreamSets. In the US team, about 42 people across different projects were using it. Similarly, in 2021, there were around 43 users. About 16-18 people in Mumbai used it in my previous company.
How are customer service and support?
The tool's support is good.
How was the initial setup?
Installing StreamSets can take time because it has two versions: a data controller and a data transformer. The data controller is easier to install, but the transformer is more complicated and requires more steps, like setting up tasks and configurations.
It would be best to ensure the environment was ready, including that it worked well with other servers. The process can be both easy and difficult, but if you follow the documentation, it should be manageable.
What was our ROI?
Whether the tool is worth the money depends on the situation. If you don't want to spend a lot on competing products like Databricks or Glue, then StreamSets might be a better option. It's particularly valuable if you prefer not to invest heavily in training your team on new technologies. If your ETL developers or data engineers are comfortable with StreamSets, it can be worth the money.
What's my experience with pricing, setup cost, and licensing?
The licensing is expensive, and there are other costs involved too. I know from using the software that you have to buy new features whenever there are new updates, which I don't really like. But initially, it was very good.
What other advice do I have?
We use various tools and alerting systems to notify us of pipeline errors or failures. StreamSets supports data governance and compliance by allowing us to encrypt incoming data based on specified rules. We can easily encrypt columns by providing the column name and hash key.
If you're considering using StreamSets for the first time, I would advise first understanding why you want to use it and how it will benefit you. If you're dealing with change tracking or handling large amounts of data, it could be cost-effective compared to services like Amazon. It's easy to schedule and manage tasks with the tool, and you can enhance your skills as an ETL developer. You can easily migrate traditional pipelines built on platforms like Informatica or Talend to StreamSets. I rate the overall solution an eight out of ten.
StreamSets make data pipelining seamless
What do you like best about the product?
The support for multi cloud makes it a wonderful choice for having data pipelining seamless. In a organisation where multiple cloud infras are used and these data pipeline needs to be shared, StreamSets makes it seamless.
What do you dislike about the product?
Didn't really find any so far, they are doing great job.
What problems is the product solving and how is that benefiting you?
Multi cloud shareing the data pipelining
Streamsets Review & ratings
What do you like best about the product?
Best things i liked of streamsets is pipeline creation using mulyiple options of data sources and destination storage options & also very easy of processsing & transformation of data.
What do you dislike about the product?
Need more options in streamsets for data transformation & availability/integration of streamsets products in other products like GCP.
What problems is the product solving and how is that benefiting you?
Streamsets helped in transformation of data in very easy and efficient way. Also reusibility of pipelines into other pipelines helps making quick data transformation.
Best Enterprise Grade Modern Data Integration Platform
What do you like best about the product?
Best UI/UX in terms of modern day ETL tools, it enhanced the working machenism thinking and good product support.
What do you dislike about the product?
Transformer still misses some basic functionality, and doesn't work well in Google cloud.
What problems is the product solving and how is that benefiting you?
Majorly for all Batch and Streaming Scenarios we are designing StreamSets pipelines, few best suited and tried out use cases below :
1. JDBC to ADLS data transfer based on source refresh frequency.
2. Kafka to GCS.
3. Kafka to Azure Event.
4. Hub HDFS to ADLS data transfer.
5. Schema generation to generate Avro.
The easy to design Canvas, Scheduling Jobs, Fragment creation and utilization, an inbuilt wide range of Stage availability makes it an even more favorable tool for me to design data engineering pipelines.
1. JDBC to ADLS data transfer based on source refresh frequency.
2. Kafka to GCS.
3. Kafka to Azure Event.
4. Hub HDFS to ADLS data transfer.
5. Schema generation to generate Avro.
The easy to design Canvas, Scheduling Jobs, Fragment creation and utilization, an inbuilt wide range of Stage availability makes it an even more favorable tool for me to design data engineering pipelines.
Great data pipeline solution
What do you like best about the product?
The best I like about Streamsets is it's scalability . That it can scale horizontally and vertically making it easily usable.
What do you dislike about the product?
According to me, the procing model can be modified.
What problems is the product solving and how is that benefiting you?
The main problem Streamsets is solving is managing different data sources and formats. I benefit as a user from it is that it improves efficiency for operations.
Easy interface for users
What do you like best about the product?
personally i liked the interface of the Streamsets
What do you dislike about the product?
some times faced endpoint issue in etl , no more dislikes about it
What problems is the product solving and how is that benefiting you?
access to data is very fast, low risk and costs.
Streamsets revolution in data integration
What do you like best about the product?
StreamSets makes designing and managing data pipelines easy-peasy. Its versatility and vast array of connectors simplify the process of handling different data sources. It's like having a smooth operator for your data flow.
What do you dislike about the product?
While StreamSets offers many benefits, some users find its learning curve a bit steep initially. Additionally, the complexity of certain configurations might be overwhelming for beginners.
What problems is the product solving and how is that benefiting you?
StreamSets simplifies data integration and real-time processing, boosting operational efficiency and enabling faster insights from data.
Very smooth experience
What do you like best about the product?
All in one place solution for ETL transformations of various kinds
What do you dislike about the product?
Ui can be made more sophisticated, detailing can be added
What problems is the product solving and how is that benefiting you?
We tried it out for one of our spark related batch job, it performed really well. User friendly and sharing options
Best ETL tool to manage data
What do you like best about the product?
I really like how IBM StreamSets now works with watsonx—it makes building and managing data pipelines feel a lot smoother. The smart automation, clear data tracking, and built-in quality checks save time and reduce guesswork.
What do you dislike about the product?
One thing IBM StreamSets could improve is making the interface a bit more beginner-friendly. Also, faster and clearer error messages during pipeline issues would really help save time and reduce frustration.
What problems is the product solving and how is that benefiting you?
It is helping me in extraction transformation and loading of large data sets
And it saves a lot of time as most of process is automated.
And it saves a lot of time as most of process is automated.
My feedback on StreamSets
What do you like best about the product?
I like the updated ui which helps in visualizing the pipeline and debugging more easily.
What do you dislike about the product?
IBM streamsets serves much of my purpose, nothing as of now.
What problems is the product solving and how is that benefiting you?
It helps in data drift handling and real-time data streaming.
showing 11 - 20