Fireworks AI (Pay-As-You-Go)

I used the latest-gen OSS  models.

It did not improve the organization—the billing experience caused serious harm. Despite subscribing to a plan explicitly labeled 'Pay-As-You-Go,' I was suddenly hit with a $40,000 charge through AWS Marketplace . Nothing ever disclosed that a pay-as-you-go account could incur a flat five-figure charge: no spending cap, no threshold alert, no notification, no warning. Receiving a $40,000 bill out of nowhere, for usage I never knowingly incurred and which my own dashboard does not reflect, has been an extremely stressful experience. This is misleading and unacceptable for a product marketed as pay-as-you-go.

The inference itself works fine. The model breadth and speed are reasonable. My problem is entirely with billing, not the technology.

Billing transparency and safeguards are urgently needed. A product sold as 'pay-as-you-go' should never produce a surprise $40,000 charge with no spending cap, no real-time threshold alert, and no notification. There must be hard spend limits, immediate alerts, and clear disclosure that large flat charges are possible. The AWS Marketplace  metering also appears mis-scaled, billed as flat $10,000 units, approximately 1,000 times the actual usage, which needs to be fixed so customers aren't blindsided.

I have used the solution for a few weeks on the 'Pay-As-You-Go' plan via AWS Marketplace.

We didn't switch from our primary inference, which runs on Amazon Bedrock . We were trialing Fireworks alongside it for its open-source model selection.

Be extremely cautious. Although it is marketed as 'Pay-As-You-Go,' there are no spending caps, no threshold alerts, and no notifications. We received a sudden $40,000 charge via AWS Marketplace, which we obviously never used, and we filed a case for it.

We used Amazon Bedrock  as our main platform and other hosted open-source inference providers. We were evaluating Fireworks as an additional option for OSS  models.

The inference technology itself is fine, but the billing is a serious risk. We were billed $40,000 on a pay-as-you-go plan for usage we never knowingly incurred and which our own usage dashboard does not reflect. Until there are enforceable spend limits, real-time alerts, and transparent metering, evaluate the billing exposure very carefully before relying on this in production.

Fireworks AI  hosts the large language model that we have trained, which is a large language model on behavior science and human capital data. We have a culture operating system, so whenever we need to do some kind of inferencing that goes via our large language model that we have trained, Fireworks AI  is hosting the LLM that we have trained. Whenever we need AI capabilities in our product, we fire a query or API call to Fireworks AI and then we get a response, with the inferencing happening on Fireworks AI model.

Building AI capabilities on the culture operating system data with Fireworks AI allows our managers to query the LLM for insights. For example, if a manager wants to know what their team trust score is right now, it will query the LLM and then it will get the answer. If a manager wants to deep dive into how they can improve, the inferencing will happen on Fireworks AI and generate an answer to improve the trust score or any vital sign score that is being generated by our LLM that is running on Fireworks AI.

The best feature Fireworks AI offers is speed. The speed of Fireworks AI stands out to me, as it is both the response time and scalability. The speed is very fast, so the inferencing happens very fast and we do not have to worry about the GPU running cost. Fireworks AI handles the scalability as well, so we have a few clients doing the inferencing at any point, and it is Fireworks AI's responsibility to scale up our GPU.

Fireworks AI has positively impacted our organization by increasing our AI response time by twenty to fifty percent, as we now have AI agents and AI features that return answers twenty to fifty percent faster. The engineering effort from the infrastructure side has been reduced, with our engineers not having to worry about hosting these trained models, resulting in a twenty to thirty percent reduction in engineering effort. The cost of hosting these models has gone down by fifteen to thirty-five percent.

We measure those improvements with Fireworks AI internally. Previously we used to host this model on our GPU on AWS  cloud and knew the latency and inferencing time. After switching to Fireworks AI, we compared the response time and found the reduction in speed.

Fireworks AI can be improved by addressing that costs can rise at scale. It is good when you have a few customers, but beyond a limit, the cost can be huge, and we do not have a cap on the uses.

The user experience is really good, and there is nothing there to improve. There are no other improvements needed for Fireworks AI that I have not mentioned.

I have been using Fireworks AI for quite some time, around six months.

Fireworks AI is stable.

Fireworks AI is pretty scalable, and you do not have to worry about it with a few customers using it at a single point in time.

I think the customer support is good, but we did not have any chance to connect with the support team. The documentation was thorough and complete, so it is straightforward and you will find all the answers there.

We previously hosted on AWS  GPUs manually, which was tedious and time-consuming, as our engineers spent lots of time maintaining those GPUs.

My experience with Fireworks AI regarding pricing, setup cost, and licensing is good, as it is pretty easy and the UI was simple. Our engineer was able to deploy it easily with no support needed from Fireworks—it was straightforward.

I have seen a return on investment with Fireworks AI. The speed of the response time has improved, and on the ROI side, we do not have to worry about engineering effort, leading to a twenty to thirty percent reduction in the engineering time for data engineers working on infrastructure.

Fireworks AI stands out in all the metrics that we were considering, so we went directly for it.

Regarding Fireworks AI's AI capabilities, its accuracy and reliability are pretty accurate, as the quality of output depends on the LLM that we are hosting on this platform. We have trained our LLM and tested it, and speed is something that has improved by hosting our model on Fireworks AI.

Fireworks AI's governance and security are pretty secure, as we have all the compliance certificates, including SOC 1 and SOC 2.

For others looking into using Fireworks AI, I advise you to know your costs if you are hosting. If you have one customer for in-house deployment, you do not have to worry about hosting. If you have few customers who want to use privately developed LLMs, then Fireworks AI is a very good place. I would rate my overall experience with Fireworks AI a ten out of ten.