Databricks by Carahsoft Technology Corp [Private Offer Only]

The use case for Databricks  is that we use the clustering for high big data processing within the cluster.

I think it is difficult to determine which feature of Databricks  I enjoy the most since there are many valuable features.

What's valuable about Databricks to my organization is that it is more cost-effective and provides better performance than the current AWS  tools and services they offer.

I am uncertain about specific improvements for Databricks.

It would be beneficial to make Databricks even more cost-effective.

I have been using Databricks for two years.

My experience with Databricks has been smooth, and I haven't encountered any issues.

Databricks is definitely a very stable product and reliable.

I have not used Databricks customer service or support.

Positive

Before Databricks, I used Batch processing, Fargate, and possibly Kubernetes .

I switched from my previous solutions because they were either too expensive or too difficult to configure.

I have considered other solutions besides Databricks, such as Snowflake , but we haven't explored it extensively yet.

We are still early in our Snowflake  experience, so we don't know the pros and cons compared to Databricks.

My deployment model for Databricks is limited as I'm not a heavy user.

I am not the person who purchased Databricks, but it was possibly acquired through the AWS Marketplace .

I may not have utilized Databricks machine learning capabilities.

My experience with the pricing and licensing model is that it remains relatively expensive. Though it's less expensive than AWS , we still need a more cost-effective solution.

I would rate Databricks overall a nine out of ten.

A typical use case for the solution is to build the data lakehouse for the client because they have a variety of source systems, and they want to unify that data into the lakehouse platform, where they want to use the data for analytical purposes and insights.

The most valuable features of Databricks  are especially the Delta Lake and the Unity Catalog; those are the main features. The Unity Catalog is for data governance, and the Delta Lake is to build the lakehouse. Currently, they're coming up with workflow jobs, along with other supporting elements to create an end-to-end solution.

In my opinion, areas of Databricks  that have room for improvement involve the dashboards. Until recently, everyone used third-party systems such as Power BI to connect to Databricks for dashboards and reports, but they're now coming up with their IBI dashboard, and I think they're on the right track to improve that even further.

I have approximately four years of experience working with Databricks.

I would rate the stability of Databricks as highly stable, around nine out of ten.

I would rate the scalability of this solution as very high, about nine out of ten.

I rate the technical support as fine because they have levels of technical support available, especially partners who get really good support from Databricks on new features. For us, it's so far so good with no problems, and I would rate the support quality as eight out of ten.

Positive

The initial setup of the Databricks solution is reasonably fair enough. It doesn't give any trouble to implement the solution, and I think it's fairly easy to set up and work on Databricks.

I can't say if there's seen an ROI from the solution because I do not have exposure in that area, although I think the people who decided to implement Databricks might have done all this analysis and POCs.

My relationship with the vendor is that I'm not a partner of Databricks; I work for a client where we use the Databricks software for implementing the solutions.

My clients are usually enterprise-level organizations, but the area where they're implementing is medium level here, although it might go into enterprise level in the future.

Regarding the price of Databricks, I don't involve myself in those decisions.

I think Databricks is very good at facilitating AI and machine learning projects; they implement AI and machine learning models very well, and clients can run their models on Databricks. I believe they are in a better place compared to competitors such as Snowflake , and they are tying up with important companies such as SAP and Palantir.

Based on my experience, I would recommend Databricks to other people. Overall, I would rate this solution as one of the best, about eight out of ten, although I might not know some of the pitfalls; it's based on use case to use case, but for us, it's working well.

Neutral

I use Databricks  for various purposes, including data engineering, MLOps, machine learning training and deployment, the entire ML cycle, and dashboards. It serves different purposes for different projects.

Unity Catalog is a feature I am currently using extensively. I am migrating many projects to Unity Catalog. MLflow, which I use for model registering and creating the lineage of models, is also valuable. 

Additionally, Databricks  serves as a single platform for conducting the entire end-to-end lifecycle of machine learning models or AI ops. I don't need to switch between various tools, making it an all-encompassing solution for development and research. I use the lake house and utilize features effectively.

There has been a significant evolution in databases. One area of improvement is the Databricks File System (DBFS), where command-line challenges arise when accessing files. Standardization of file paths on the system could help, as engineers sometimes struggle. 

It would be beneficial to have utilities where code snippets are readily available. This would allow engineers to easily click a snippet and import it into the notebook, enabling quick modifications to variables or paths for fetching files, such as reading data from DBFS files. If I could right-click to copy absolute paths or to read files directly into a data frame, it would standardize and simplify the process.

I have used the solution for five years plus.

I would rate stability seven to eight out of ten.

I would rate scalability seven to eight out of ten.

I do not have any issues that require support. Many resources are available online.

Neutral

I use infrastructure as code on the cloud to deploy the infrastructure. I have all the Git  repositories and code repositories for deploying the code and models in the workspace. My setup includes a shared workspace, shared clusters, and integration with Unity Catalog.

I have a team of 100 engineers working with me, and I head the Center of Excellence (COE).

I believe it is competitive across clouds. When it comes to big data processing, I prefer Databricks over other solutions. Cost-wise, it is very competitive. The setup process is straightforward, thanks to the use of Spark clusters. This allows for faster turnaround times with Databricks.

The product rating is nine out of ten. 

Databricks serves as a single platform that can handle numerous end-to-end machine learning tasks. The configuration is simple, scalability is excellent, and monitoring cluster utilization facilitates informed business decisions. 

It's easy to schedule jobs, pipelines, and handle multiple use cases in parallel, providing countless benefits.

We work on three platforms. Databricks  is hosted on Azure  for us, so we work with ADFS, Azure Data Factory , and also the AWS  Cloud. We work for some customers.

The notebooks and the ability to share them with collaborators are valuable, as multiple developers can use a single cluster. This reduces costs. The scheduling part is managed by Databricks  itself, for example, when it is idle, it will automatically turn off. All these features are handled by Databricks, reducing costs. We do not need to schedule separately. 

For example, on AWS  EC2 , we have to create a Lambda function or use System Manager templates to schedule EC2  and EMRs. Here, it is taken care of, saving significant resources. 

Additionally, notebooks can be shared within the development team which saves effort. Developers can share their notebooks. Git  and Azure DevOps  integration on the Databricks side is also very helpful.

The API deployment and model deployment are not easy on the Databricks side. We use MLflow for managing MLOps, however, further improvement would be beneficial, especially for large language models and related tools. Moreover, the API deployment should be simplified for ease of deployment and consumption.

I have been using Databricks for approximately two and a half to three years.

We have not faced any shortages so far. The clusters are available on demand, thus we have not encountered any scalability issues.

We mostly had limited data support required from Databricks. Whenever we did need support, within two or three days the problem was solved. I would rate them ten out of ten.

Positive

We bought it as a service, which is why we never implemented it ourselves. We do not have any implementation team.

For companies focused solely on data transformation, transferring data between databases, and not tackling machine learning or deep learning problems, I recommend ADF. It would be sufficient and cost-saving compared to a full-fledged solution like Databricks. However, for data analytics and solving ETL problems, one should consider Databricks.

I would rate it nine out of ten.