My usual use case for Databricks as an end-user mostly involves exporting data. This typically entails writing directly into a web interface to get the data out, so probably with Python.
Databricks Data Intelligence Platform
Databricks, Inc.External reviews
External reviews are not included in the AWS star rating for the product.
Improved data governance has enabled sensitive data tracking but cost management still needs work
What is our primary use case?
What is most valuable?
The most significant benefit Databricks has brought to my company is the Unity Catalog. Previously, with our data warehouse, we weren't able to track where sensitive data was. The Unity Catalog has been a big improvement, even though we haven't gotten the rest right.
The user interface is very useful, especially in writing directly into a web interface.
From my perspective, the ability to export data effectively and use Python within Databricks are key valuable features.
What needs improvement?
I believe we could improve Databricks integration with cloud service providers. The impact of our current integration has not been particularly good, and it's becoming very expensive for us. The inefficiencies in our implementation, such as not shutting down warehouses when they're not in use or reserving the right number of credits, have led to increased costs.
We made several beginner mistakes, such as not taking advantage of incremental loading and running overly complicated queries all the time. We should be using ETL tools to help us instead of doing it directly in Databricks. We need more experienced professionals to manage Databricks effectively, as it's not as forgiving as other platforms such as Snowflake.
I think introducing customer repositories would facilitate easier implementation with Databricks.
For how long have I used the solution?
I have been working with Databricks for the last six months.
What do I think about the stability of the solution?
As a platform, Databricks is fine. However, our implementation isn't particularly reliable.
We've suffered from the lack of professionals with previous experience, which makes it difficult to dig ourselves out of the situation we've found ourselves in.
What do I think about the scalability of the solution?
The scalability level of Databricks at the moment exceeds our needs. It's not a problem for us.
The sky's the limit with Databricks.
How are customer service and support?
I have addressed technical support about our issue with Databricks. It was the team that engaged with them, and I believe our development teams also reach out for support, though I'm not sure what level of support they get.
Previously, when using Snowflake, we had customer reps who were really knowledgeable and helped us to avoid beginner mistakes. With Databricks, it seems we could have benefited from similar support. Our implementation team had no experience and made obvious mistakes. It may be that we opted not to have that support, but I believe we should have.
How would you rate customer service and support?
Positive
Which solution did I use previously and why did I switch?
Before Databricks, I used SQL Server.
The big decision to switch from SQL Server to Databricks was motivated by the lack of auditing, lineage, and tracking sensitive data in SQL Server, along with a need for more flexibility.
How was the initial setup?
I did not participate in the initial setup of Databricks.
What about the implementation team?
We use a consultancy, Avanade, for our Databricks implementation. They had previously done a Databricks implementation for another part of our organization. Our implementation team lacked experience which resulted in several beginner mistakes.
What was our ROI?
So far, we're not measuring any return on investment, such as saving time, money, or resources with Databricks. We're still in the phase where our old system and the new system are running simultaneously, so everything is twice as expensive and much effort is doubled. We haven't progressed far enough yet to realize any ROI.
What's my experience with pricing, setup cost, and licensing?
I believe that in terms of credits for Databricks, we're spending between £15,000 and £20,000 a month.
I think Databricks is priced correctly. If we managed our resources better, we wouldn't be paying anywhere near that amount. The issue is with our management of resources.
Which other solutions did I evaluate?
No other options were considered because we used the consultancy Avanade, who had done a previous Databricks implementation for another part of our organization. We used them to recreate our implementation.
What other advice do I have?
I'm probably not the best person to discuss certain aspects of Databricks since I haven't explored it deeply and am not part of the team developing it.
We haven't utilized Databricks' machine learning capabilities.
From my company, data ingestion and transformation are done with Databricks, though I don't do it directly.
I don't use Databricks' features for managing data, such as data lake and warehouse operations.
Most of our current work with Databricks isn't really live yet, so measuring savings in time and money or identifying any return on investment isn't applicable right now.
I would rate this review a 7 overall.
Collaborative Notebooks and MLflow Integration Boost Productivity
Databricks compute user
Great summit
New to BI/AI but growing it in out AI space
Bricks is gr8
Feature Parity for Govcloud
It was fantastic got to meet many peers
• Brings data from multiple sources (structured, unstructured, streaming) into a unified platform for centralized access.
2. Slow Time to Insights
• Enables faster data processing with Spark, Delta Lake, and optimized workflows to accelerate decision-making.
3. Scalability Issues
• Handles growing data volumes and user loads efficiently with cloud-native, distributed compute.