Listing Thumbnail

    LiteLLM LLM Gateway (Proxy Server)

     Info
    Sold by: LiteLLM 
    OpenAI Proxy Server (LLM Gateway) to call 100+ LLM APIs using the OpenAI format Bedrock, Huggingface, VertexAI, TogetherAI, Azure OpenAI, OpenAI, etc. Get started with Opensource LiteLLM here: https://github.com/BerriAI/litellm (12,000+ Github Stars)

    Overview

    With LiteLLM Proxy Server, you'll get access to a Proxy Server to call 100+ LLMs in a unified interface where you'll be able to track spend, set budgets per virtual key and users.

    You'll be able to set budgets & rate limits per project, API key, and model on OpenAI Proxy Servers.

    You can also translate inputs to the provider's completion, embedding, and image_generation endpoints as well as retry/fallback logic across multiple LLM deployments.

    Highlights

    • Call 100+ LLM APIs in OpenAI format https://docs.litellm.ai/docs/providers
    • Track metrics on Prometheus and send logs to s3, GCS Bucket, Langfuse, OpenTelemetry, Datadog https://docs.litellm.ai/docs/proxy/logging
    • Get started with Opensource LiteLLM here: https://github.com/BerriAI/litellm (12,000+ Github Stars)

    Details

    Delivery method

    Deployed on AWS

    Unlock automation with AI agent solutions

    Fast-track AI initiatives with agents, tools, and solutions from AWS Partners.
    AI Agents

    Features and programs

    Financing for AWS Marketplace purchases

    AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
    Financing for AWS Marketplace purchases

    Pricing

    LiteLLM LLM Gateway (Proxy Server)

     Info
    Pricing is based on the duration and terms of your contract with the vendor. This entitles you to a specified quantity of use for the contract duration. If you choose not to renew or replace your contract before it ends, access to these entitlements will expire.
    Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator  to estimate your infrastructure costs.

    12-month contract (1)

     Info
    Dimension
    Description
    Cost/12 months
    LiteLLM Enterprise
    All features under LiteLLM Enterprise License, SSO sign on, Feature Prioritization, Proffesional Support, Custom SLAs
    $30,000.00

    Vendor refund policy

    All fees are non-cancellable and non-refundable except as required by law.

    How can we make this page better?

    We'd like to hear your feedback and ideas on how to improve this page.
    We'd like to hear your feedback and ideas on how to improve this page.

    Legal

    Vendor terms and conditions

    Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

    Content disclaimer

    Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

    Usage information

     Info

    Delivery details

    Software as a Service (SaaS)

    SaaS delivers cloud-based software applications directly to customers over the internet. You can access these applications through a subscription model. You will pay recurring monthly usage fees through your AWS bill, while AWS handles deployment and infrastructure management, ensuring scalability, reliability, and seamless integration with other AWS services.

    Support

    Vendor support

    For dedicated support, please send an email to support@berri.ai 

    AWS infrastructure support

    AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

    Product comparison

     Info
    Updated weekly

    Accolades

     Info
    Top
    10
    In AIOps
    Top
    25
    In AIOps
    Top
    25
    In AIOps

    Customer reviews

     Info
    Sentiment is AI generated from actual customer reviews on AWS and G2
    Reviews
    Functionality
    Ease of use
    Customer service
    Cost effectiveness
    0 reviews
    Insufficient data
    Insufficient data
    Insufficient data
    Insufficient data
    11 reviews
    Insufficient data
    Positive reviews
    Mixed reviews
    Negative reviews

    Overview

     Info
    AI generated from product descriptions
    API Compatibility
    Supports calling 100+ Large Language Model APIs using a unified OpenAI-compatible interface
    Metrics and Logging
    Enables tracking metrics through Prometheus and supports log transmission to multiple observability platforms
    Multi-Provider Integration
    Provides translation and compatibility for completion, embedding, and image generation endpoints across different LLM providers
    Budget and Rate Control
    Implements configurable budget and rate limit settings at project, API key, and model levels
    Fallback Mechanism
    Supports retry and fallback logic across multiple Large Language Model deployments
    Prompt Engineering
    Side-by-side comparative analysis of prompts, parameters, models, and model providers across test case banks
    Workflow Automation
    Prototype and deploy AI workflows integrating business logic, data, APIs, and dynamic prompts for multiple use cases
    Model Evaluation
    Create comprehensive test case repositories to evaluate and identify optimal prompt and model combinations across diverse scenarios
    Request Tracking
    Reliable proxy mechanism for connecting applications and model providers with comprehensive request tracking for debugging and quality monitoring
    Contextual Search
    Advanced document retrieval system enabling company-specific data integration as context for large language model interactions
    AI Gateway
    Centralized control and routing mechanism for managing AI application interactions across multiple models
    Observability
    Comprehensive monitoring and tracking system for AI application performance and interactions
    Prompt Management
    Advanced configuration and control system for managing AI model input prompts and responses
    Security Compliance
    Multi-standard security compliance including ISO, SOC2, HIPAA, and GDPR certification frameworks
    Model Load Balancing
    Dynamic routing and distribution of AI requests across different language models for optimal performance

    Contract

     Info
    Standard contract

    Customer reviews

    Ratings and reviews

     Info
    0 ratings
    5 star
    4 star
    3 star
    2 star
    1 star
    0%
    0%
    0%
    0%
    0%
    0 AWS reviews
    No customer reviews yet
    Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.