dbt is used for data transformation and data engineering with multiple data transformations and engineering functions. It is also used for orchestrating data engineering pipelines. An example of this is ingesting data from Azure Blob or S3 sources and then transforming it into different layers in the data platform.
dbt Platform
dbt LabsExternal reviews
190 reviews
from
External reviews are not included in the AWS star rating for the product.
I can manage my own dependencies using dbt.
What do you like best about the product?
dbt runs well on Redshift, since that is what was mentioned over and over again in the notes; however, dbt simply compiles the SQL and the warehouse itself handles the heavy lifting. Using Git and Version Control for Data Models, is nice because it keeps the data model from exploding. dbt also integrates with our AWS infrastructure without requiring tears. The speed is sufficient, as it simply passes the work to the database; although, having the transformation logic in one location is helpful.
What do you dislike about the product?
The cost is becoming increasingly expensive and considering dbt is essentially a fancy SQL Compiler. dbt also has poor performance when handling un-structured data (although this may be due to Redshift); I'm unsure, everything seems to blend together. Additionally, the learning curve is very steep if you are not familiar with Jinja and setting-up YAML files properly.
What problems is the product solving and how is that benefiting you?
dbt allows us to scale the analytics engineering work so we are not running ad-hoc SQL scripts on a laptop. dbt separates the compute and storage logic, allowing us to define the "what", while it determines the "how". dbt automatically manages the dependency graphs, which is great, as I cannot handle tracking those manually.
Streamlined Data engineering and built-in lineages
What is our primary use case?
What is most valuable?
The best features of dbt include lineage and Jinja templating languages that make it easy for creating pipelines.
The built-in lineage feature provides a good understanding of the several layers where data is being loaded in dbt, allowing visibility from different layers into the end product.
dbt has positively impacted version controlling as it has different version control steps involved. The specific improvements seen with version control in dbt are that it has helped trace the data lineage, enabled faster trace and rollbacks, and enabled safe collaboration at every scale, which has improved data quality.
A return on investment has been seen from using dbt as the time has reduced while utilizing dbt in the form of data pipelines and ETL scripting. There is operational efficiency achieved, and data quality and governance have also been achieved with modular SQL and version controlling, which reduced duplication of data and data errors.
What needs improvement?
dbt is not as stable as preferred, as it has had a few outages in the current year itself, so improvement should be made in the outages section as it is not stable.
The copilot in dbt is not very comfortable for users, and my team has already tried using it but opted to move off from the dbt copilot to other copilots such as GitHub.
Improvement is needed in the tool itself in terms of the copilot, in terms of covering outages, in terms of testing, and in terms of quality reasons related to governance and collaboration.
For how long have I used the solution?
dbt has been used for about a year.
What do I think about the stability of the solution?
dbt is not as stable as preferred, as it has had a few outages in the current year itself, so improvement should be made in the outages section. Overall, dbt is stable.
What do I think about the scalability of the solution?
In terms of scalability, dbt has improved the scalability of the organization depending on different dimensions for team size, data, and complexity of transformations.
How are customer service and support?
The customer support from dbt was good and was identified and resolved by the customer support team when reached out to.
How would you rate customer service and support?
Neutral
Which solution did I use previously and why did I switch?
Initially, multiple solutions such as Talend Studio and Informatica were utilized for different projects before switching to dbt.
How was the initial setup?
The experience with pricing, setup cost, and licensing was that it was straightforward for the pricing setup and also on the licensing part for dbt.
What was our ROI?
A return on investment has been seen from using dbt as the time has reduced while utilizing dbt in the form of data pipelines and ETL scripting. There is operational efficiency achieved, and data quality and governance have also been achieved with modular SQL and version controlling, which reduced duplication of data and data errors.
What's my experience with pricing, setup cost, and licensing?
dbt was purchased through the AWS Marketplace.
Which other solutions did I evaluate?
Before choosing dbt, other options were evaluated, but dbt was the preferred choice as it was an open-source solution that was already on the track.
What other advice do I have?
My advice to others looking into using dbt is that it is a good tool for having ETL or ELT transformations done. To begin with, a pilot project can be added with modular SQL or modeling, Git workflows, and a standardized project structure from source, staging, intermediate, to the mart layers, which will optimize performance. I would rate this solution a seven out of ten.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
User-Friendly Data Modeling with Seamless Integration
What do you like best about the product?
DBT is great for organizing data models. It is user friendly, integrates well with other tools, and they had a great onboarding process.
What do you dislike about the product?
In dbt Cloud, I cant work on two different branches at the same time in different browsers.
What problems is the product solving and how is that benefiting you?
DBT allows us to take raw data from many sources and output it in clean, easy to use output tables that are used in our bi tool.
We finally found a solution for easier management of data models
What do you like best about the product?
The interesting fact about dbt is that it simplifies the process of managing data pipelines. It was implemented successfully and I depend on it on a daily basis and hence my frequency of use is high. The amount of features such as model testing, documentation, and version control is especially appreciated by me. It has minimized errors in our conversion processes and has simplified the process of teamwork a lot and has helped the team maintain pipelines which are uniform and structured across projects.
What do you dislike about the product?
The thing I dislike with dbt is that it may be difficult to troubleshoot model errors. The features are good, and error messages are not always helpful in disclosing the problem. High frequency of use implies that such moments have the capacity of derailing workflows since I use it frequently. There is responsive customer support but edge-case fixes are not always immediately available, so the team occasionally has to check outputs before proceeding.
What problems is the product solving and how is that benefiting you?
Dbt has resolved the problem of inaccurate or inconsistent transformations within our workflows. It has simple implementation and I use it frequently hence my usage frequency is also high. It has many features that can be used to test and keep track of the version that helps in uncovering errors at the earlier stages. It is lean cooperation throughout the team, reduced manual checks that have to be done multiple times, and ensures our data is reliable and can be used in reporting and business decisions.
Coding-focused data pipelines have accelerated delivery of in-house data products
What is our primary use case?
My main use case for dbt is for data transformation and data engineering.
A specific example of how I use dbt for data transformation and engineering is that we use it to connect and ingest data from our Azure blob and S3 buckets, then transform through our glorified serving layers into our data platform.
We use dbt to orchestrate our data engineering pipelines.
What is most valuable?
The best features dbt offers include built-in lineage, which is useful, and the Jinja templating language that makes it easy for creating pipelines.
The built-in lineage feature and the Jinja templating have made things easier for me because it's an easy language that most people pick up pretty quickly, and the ability to template SQL and generate SQL programmatically is useful.
dbt has positively impacted my organization by allowing us to create our data pipelines much faster, going from ingestion of data to creating a data product in weeks instead of months, and we can do it in-house with the skillset we already have.
I can share a specific outcome that resulted from using dbt: there was a data product a vendor did for us two years ago that took them six months to achieve, whereas we were able to do it in-house with dbt coupled with Snowflake in four to five weeks, which was much quicker.
What needs improvement?
dbt can be improved as I find the co-pilot in dbt is not very good, and my team has tried using it but opted to move off it and use other co-pilots such as GitHub. Additionally, the debugging capabilities in dbt are practically nonexistent, making it very hard to troubleshoot and debug if you write incorrect Jinja code.
dbt is not as stable as I would prefer, as there have been a few outages this year.
For how long have I used the solution?
I have been using dbt for about four years.
What do I think about the scalability of the solution?
dbt's scalability is fine, and I have never had any capacity problems.
How are customer service and support?
The customer support is average.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
I previously used a different solution; we used a bit of Talend and Wherescape Red.
What was our ROI?
I have seen a return on investment as it means we don't have to employ as many people; it's more about cost avoidance, but I have not removed employees.
What's my experience with pricing, setup cost, and licensing?
My experience with pricing, setup cost, and licensing was simple enough.
What other advice do I have?
dbt is easy to use and easy to learn, but it has some limitations that I would love to see mitigated; however, in general, most of my engineers are happy using dbt.
My advice for others looking into using dbt is that it is good if you have an organization with engineers who prefer to code and get hands-on, but if you have teams of engineers who prefer a mouse-driven, drag-and-drop type, less technical coding environment, other tools might be more appropriate.
I rate dbt overall an eight out of ten.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Reliable data project workflow
What do you like best about the product?
What I like about dbt most is that my modeling work is much more comfortable. I spend my entire time modifying logic or verifying the changes and the set of features is what I actually need. I operate it with Snowflake, and that integration ensures that my updates are regular. The installation was relatively fast and the number of times the tool has been used demonstrates the extent to which it has come to my rescue in order to maintain projects and ensure they are well organized.
What do you dislike about the product?
The thing I do not like is that there are some spots that do not provide me with the flexibility that I need when I work on large portions of work. It imposes additional procedures which disrupt my rhythm. These weak spots are visible since I am in dbt so many times. There is a good response of the customer support but still the restrictions influence my speed during peak weeks.
What problems is the product solving and how is that benefiting you?
Dbt addresses our problem of disorganized model upkeep in the team. I apply it in my daily activities keeping track of the changes and updating and ensuring that all goes in the right direction. The size of the features suits very well into our workflow and the frequency of use demonstrates how much easier our process is now. It has assisted us in preventing the instances of miscommunication and enabled all of us to be more certain of the work we drive forward.
DBT has absorbed all the stress while making my life a lot easier
What do you like best about the product?
I threw terabytes at DBT and expected the infrastructure to fail but DBT ran the distributed execution on its own with no intervention by me. The ability to run machine learning directly within SQL is strange but better than exporting to vertex.ai and dealing with cluster management myself. I also do not have to worry about cluster management as I can just write the query and wait for the results which in my opinion is very straightforward thing to do.
What do you dislike about the product?
Billing is a trap as well as if you run a generic query without a filter the costs jump up right away which can be very annoying. I had to re-write all of my stored procedures because the syntax isn't quite like pl/sql. And I really dislike reading the logs when a model fails and/or errors occur.
What problems is the product solving and how is that benefiting you?
DBT takes care of managing the server and I never had to manage it myself and also I used to have to sit around waiting for hours for my ETL to finish but with DBT now they complete really quickly. We just throw the dirty data into DBT and it will handle separating out the different types of storage without me worrying about it.
Streamlined Development and Reliable Data with Effortless DBT Orchestration
What do you like best about the product?
What I appreciated most was the elimination of duplicated code that used to be spread across various scripts. This change has significantly enhanced data reliability and now lets me implement business logic directly in pure SQL. I also value how much it accelerates development, and I find the orchestration and deployment with DBT to be exceptionally straightforward.
What do you dislike about the product?
I found the project management aspect challenging when dealing with hundreds of models, as the interface can at times be quite complicated.
What problems is the product solving and how is that benefiting you?
This tool has addressed our primary needs during the data transformation phase, enhancing data reliability and making development more efficient. It also acts as a central resource, ensuring that all teams use the same data management functions, even though it does have some shortcomings. Overall, DBT performs exceptionally well.
Efficient Data Management with Room for Documentation Improvement
What do you like best about the product?
I appreciate dbt for its secure practices in software engineering which I find crucial, particularly in ensuring integrity through data lineage, which plays a significant role in our security framework. The versatile templating system effectively enhances our data modularity, which amplifies the efficiency of our data processes. The intuitive templating also significantly improves the user experience by making our boards more operationally efficient.
What do you dislike about the product?
I wish the error messages were clearer. Sometimes, it's hard to identify the root of issues based on the current messages. Additionally, the documentation could be more beginner-friendly, as new users might find it challenging to navigate and understand.
What problems is the product solving and how is that benefiting you?
dbt resolves inconsistent data issues, making models easy to maintain. The templating boosts efficiency and data lineage ensures quality and security.
dbt keeps our data models clean, consistent and version controlled
What do you like best about the product?
I use dbt every day to transform raw data in our warehouse into clean, analytics ready tables and my workflow typically begins in VS Code, where I write sql models, then push them to Git for version control and run them through dbt Cloud. And overall it has also made collaboration between our team members much easier because dbt makes the whole process much more simpler.
What do you dislike about the product?
It's challenging when one change throws an entire run off track and the error messages are at best, vague. I also feel the need to defend is the handiwork of my contributor to dbt cloud. I have also encountered the overly relaxed strucure and the resulting chaotic command and environment specific configurations.
What problems is the product solving and how is that benefiting you?
I can onboard people more easily, which has dramatically increased the usage of my warehouse and decreased my reliance on fragile, one off sql scripts and we have a whole team of analysts, engineers, and product working to have the same versioned models just building and ready for use.
showing 1 - 10