Sold by: abyo software
Deployed on AWS
Free Trial
Serve more concurrent users and longer context per GPU. A vLLM v1 KV-cache backend that quantizes the KV cache to int4 (per-channel KEY, per-token VALUE), bit-identical to fp16 - about 3x more KV density.
Overview
S4 KV is a KV-cache backend for vLLM v1 paged attention that stores the attention KV cache in int4 instead of fp16, using KIVI-style quantization (per-channel asymmetric int4 for KEY, per-token int4 for VALUE). It is bit-identical to fp16 greedy decoding (verified on Qwen2.5 1.5B/3B/7B and Llama-3.2-3B / 3.1-8B), so you get roughly 3x more KV-cache density: more concurrent requests and longer context on the same GPU, with no measured quality loss. The image is a turnkey GPU server - an OpenAI-compatible vLLM endpoint with int4_kivi enabled - launched via the included CloudFormation template. Supported models: head_dim 128 with standard attention (Qwen2.5 and Llama-3 families); the backend fails fast at startup rather than serve incorrect output for an unsupported model. Priced on density and quality, not latency: at long context on datacenter GPUs the backend trades some decode latency for density. Runs on g5, g6, g6e, p4d, and p5 GPU instances.
Highlights
- About 3x KV-cache density vs fp16 (int4 KIVI: per-channel-asymmetric KEY + per-token VALUE) - more concurrent users and longer context per GPU.
- Bit-identical to fp16 (greedy match 1.0000, verified on Qwen2.5 and Llama-3 families) - density with no measured quality loss.
- Turnkey GPU AMI: an OpenAI-compatible vLLM server with int4_kivi enabled out of the box; launch via the included CloudFormation.
Details
Sold by
Categories
Delivery method
Delivery option
CloudFormation Quick Launch
64-bit (x86) Amazon Machine Image (AMI)
Latest version
Operating system
Ubuntu 22.04
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Free trial
Try this product free for 14 days according to the free trial terms set by the vendor. Usage-based pricing is in effect for usage beyond the free trial terms. Your free trial gets automatically converted to a paid subscription when the trial ends, but may be canceled any time before that.
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time. Alternatively, you can pay upfront for a contract, which typically covers your anticipated usage for the contract duration. Any usage beyond contract will incur additional usage-based costs.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Cost/hour |
|---|---|
g6.xlarge Recommended | $0.113 |
g5.xlarge | $0.141 |
g5.2xlarge | $0.17 |
g5.12xlarge | $0.794 |
g6.2xlarge | $0.137 |
g6.12xlarge | $0.644 |
g6e.xlarge | $0.261 |
g6e.2xlarge | $0.314 |
g6e.12xlarge | $1.469 |
p4d.24xlarge | $3.074 |
Dimensions summary
You pay an hourly software fee metered per GPU instance type you run. The eleven dimensions map to specific EC2 GPU instances across five families: g5, g6, g6e, p4d, and p5. Within each family, sizes range from xlarge to larger multi-GPU options like 12xlarge, 24xlarge, and 48xlarge. Larger instances handle more concurrent requests and longer context. You choose the instance that matches your throughput and model size. This software fee bills separately from your own EC2 compute charge. Pricing reflects capacity and quality, not latency.
Top-of-mind questions for buyers
The hourly fee covers the int4 KV-cache software running on your chosen GPU instance. You pay this fee separately from your own EC2 compute charge for that instance. The two charges appear on your AWS invoice per instance type you run.
Match the instance to your throughput and model size. Larger instances handle more concurrent requests and longer context per GPU. The vendor suggests sizing by concurrency multiplied by context length. Bigger multi-GPU options like 12xlarge, 24xlarge, and 48xlarge serve heavier loads.
The software fee meters per instance-hour, so it accrues while the instance runs. A stopped instance does not run the software and stops the hourly fee. Underlying AWS storage charges for the instance may still apply separately from the software fee.
abyo.net
Helpful?
Vendor refund policy
Software fees are billed by AWS Marketplace on your consolidated AWS bill. Refund requests are handled per the AWS Marketplace refund policy; for product-specific questions email aws-support@abyo.net .
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
64-bit (x86) Amazon Machine Image (AMI)
Amazon Machine Image (AMI)
An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.
Version release notes
Adds a CloudFormation Quick Launch delivery option. Software identical to version 0.1.0.
Additional details
Usage instructions
Launch via the included CloudFormation template (deploy/cfn-gpu-serving.yaml): it creates the security group + IAM role and starts the vLLM OpenAI server with int4_kivi enabled. Set ModelId to a head_dim-128 model (e.g. Qwen/Qwen2.5-7B-Instruct). The OpenAI-compatible API is on port 8000 at /v1; health at /health.
Support
Vendor support
Email support: aws-support@abyo.net
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Self-contained EC2 AMI of the S4 transparent S3 compression gateway with CPU codecs (zstd / gzip) preinstalled. Launch on any general-purpose or compute-optimized instance (t3 / m6i / m7i / c6i / c7i), point your S3 clients at it, and cut S3 storage bytes 50-80 percent for compressible data with zero application changes.
Self-contained EC2 AMI of the S4 transparent S3 compression gateway with NVIDIA nvCOMP GPU codecs preinstalled. Launch on a GPU instance (g4dn / g5 / g6), point your S3 clients at it, and cut S3 storage bytes 50-80 percent for compressible data with zero application changes.
Drop-in S3-compatible gateway that transparently compresses every object (CPU zstd or GPU nvCOMP), cutting S3 storage bytes 50-80 percent for compressible data with zero application changes. Includes pre-deployment savings estimation and measured-savings reporting.
Drop-in S3-compatible gateway that transparently compresses every object, cutting S3 storage bytes 50-80 percent for compressible data with zero application changes. This edition bills by measured savings: you pay per GB of backend storage avoided, per hour, at roughly one third of the avoided storage cost.
Cut your CloudWatch custom-metric bill: govern metric cardinality at ingest, then auto-baseline and roll up savings across your whole AWS Organization.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.