Cohere Embed Light Model v3 - English

I have used Cohere in a RAG use case where I had to vectorize some data. I used multiple models in RAG to find a better model that could give superior results. I was trying to find a cloud-hosted model, and Cohere's Embed English v3.0 is a cloud-hosted model that took less time to embed the textual data. When I was trying to get the similarity search after embedding that data, Cohere provided much better results.

Let's suppose I had to embed 100 documents at a time. Most other models, including all-MiniLM-L6-v2, took more time when I was trying to embed using that model. When I tried Cohere, it was much faster. I would say it was more than 50 to 60% faster than those models. It was even somewhat faster than text-embedding-3, which is from OpenAI. So Cohere helped to reduce the development time and embedding times.

I believe Cohere offers excellent features, especially the cloud-hosted model and the API calls. The number of times I can call the API within a minute is very good. The ping is great; I have started a request to Cohere model, and it was very quick to respond. The best part was the free tier because most models do not provide a free tier.

Regarding benefits, Cohere is less costly than other models. If I talk about OpenAI or Google embedding models, they charge highly compared to Cohere. Regarding the training data, Cohere has the most data embedded or trained with the most English. Cohere's Embed English v3.0 has been trained with much more data than other models, including OpenAI. This gives an extra benefit to my organization.

One thing that Cohere can improve is related to some distances when I am trying similarity search. Let's suppose I have provided textual data that has been embedded. I have to use some extra process from numpy after embedding the model. In the case of OpenAI embedding models, I do not have to use that extra process, and they provide lower distances compared to my results from Cohere. I was getting distances of approximately 0.005 sometimes, but in the case of Cohere, I was getting distances around 0.5 or sometimes more than that. I think that can be improved. It was possibly because of some configuration or the way I was using it, but I am not exactly sure about that.

I have been using Cohere for the last seven or eight months.

The scalability was very good because of the response time. Even though I do not need that much processing at a time, I have had a good experience with Cohere so far.

Previously, I was using all-MiniLM-L6-v2 and switched to Cohere because all-MiniLM-L6-v2 needed to be locally deployed. That model was processing locally, and the results I was getting from that model, even though it was open source, I was not satisfied. That is why I switched to Cohere.

I can highlight two benefits. Cohere charges less than OpenAI, so it saves cost. In the second use case, the timing is significant. Cohere's Embed English model took less time to embed than OpenAI's embedding ada-002 model. In this case, it also saves time. These two benefits I can highlight.

I have evaluated OpenAI's Embed English v3 and text-embedding-3 models. I have evaluated multiple models, and I even evaluated some models from Hugging Face .

Cohere provides a free tier, and any developer who is starting their journey can use Cohere for RAG use cases. They can utilize the model benefits. After using Cohere, I got distances after the similarity search that were much lower compared to other vectorization and embedding models. The only model that performed better than Cohere was OpenAI's text-embedding-3-large. It was good, but Cohere was the second-best performing model in my use case.

I think Cohere's use cases are excellent, and I would suggest Cohere to others because of the less response time and time-saving in the process. It is also cheaper than other models. I would give this review a rating of eight out of ten.

Neutral

We founded this company two and a half years ago, and since the middle of 2022, we foresaw the trending of generative AI and large language models, so my startup is working on developing generative AI applications for our clients, including enterprises and a few other startups across America and Canada.

I started using Cohere when we first got information from the community about their reranking models almost one and a half years ago.

In some clients' projects, we were required to introduce reranking model in the RAG flow (Retrieval-augmented generation). In this flow, we use different components to allow users to select and pick up from the UI components, drag and drop to their flow to enhance their RAG pipeline. That's where we introduced Cohere models as one of the providers for reranking.

Cohere's reranking model helped us complete this request

From our data, I can tell that at least 15% of end users were actively using reranking to enhance their RAG pipeline because we have the UI to indicate that reranking is recommended as it can enhance the quality of the retrieval.

For clarification, I want to describe this data more clearly. As mentioned, 15% of end users chose to enable this module based on the fact that we have the pricing tier with an extra cost for their API call.

In general, I'm satisfied with the speed, and I can confirm this because we have the long fields to track all conversations, and we see that this loop for reranking actually costs relatively less time throughout the whole chat flow. Regarding quality, it's hard to tell because we don't have a benchmark. In our enterprise applications, we are trying to build up evaluation pipelines, do AB testing, and other analysis, but it's not a conventional computer science application, so it's very hard to build up evaluation pipelines with objective criteria. It's challenging for us to make a conclusion about quality, but the speed is good.

A direct benefit of using Cohere's reranking model is that we can tell clients we have this module rather than missing this piece, as reranking is a very important component that companies discuss to enhance RAG quality.

Although it's not impacting our business model, I'm pushing for the evaluation system because it can expand our business scope. We want to sell our system to clients, and while they may not be aware of evaluation initially, it's beneficial to have. Once we have these systems, we can showcase to end users that employing such a reranking system improves quality. We need proof to convince ourselves that after implementing reranking, we get better quality.

It would be better to have a dashboard for users to showcase how reranking helps improve quality. When end users choose the service, they want to see the actual output. The evaluation part is challenging for recent large language model applications but remains very important.

If Cohere could provide a dashboard where we can employ an LLM as a judge to check quality before and after reranking, that would be helpful. We could either have another large language model evaluate this part or allow UAT users to manually check with humans in the middle. As an enterprise provider, we want such features because when chatting with clients, we can demonstrate that employing Cohere's reranking model significantly improves results compared to not using it.

Documentation is not a major blocking issue for us as we are sophisticated software engineers. Integration and the API provided for reranking models are not complicated, so we can easily handle that. The documentation is good. The major point is to prove the value through evaluation. We need a sophisticated solution to showcase visibly to our clients and engineering team to convince them that using this model creates improvements.

I started using Cohere when we first got information from the community about their reranking models almost one and a half years ago.

That's only what we need in our product currently. I will communicate when we have other requirements.

We haven't had any issues to escalate to Cohere's support because reranking is an optional feature in our product, and we haven't seen any significant issues so far.

We don't observe many scaling problems because it's an enterprise application.  There are a few hundred people using this. The concurrent user rate is not significant, which might be why we don't see many scaling issues so far.

We haven't had any issues to escalate to Cohere's support because reranking is an optional feature in our product, and we haven't seen any significant issues so far.

Positive

For reranking, Cohere was our only solution.

I'm more focused on the speed and overall quality of the model itself and the chat flow as a whole solution. That's why I'm not in the position to comment on the price and setup cost as there are DevOps working on this piece.

Hard to estimate the overall ROI. but if you see the ROI for the feature of reranking, it's a positive number

I'm not in the position to answer that question because I was not the one who deployed that model, but I believe it is because we see the model name as ARN name, so it's most likely coming from Bedrock.

For reranking, Cohere is the only solution we have used so far.

As a feature developer, I'm more focused on the speed and overall quality of the model itself and the chat flow as a whole solution. That's why I'm not in the position to comment on the price and setup cost as there are DevOps working on this piece. My rating for this solution is 8 out of 10.

Positive