Sold by: Shri Sai Tech LLCÂ
Overview
LLM Insight is a platform to manage, track and optimize LLM. The LLM Insight allows you to track your LLM expenses in real-time, offering visibility into the costs of each API call. You can monitor spending by model, department, or project to ensure you're staying within budget. The platform allows for setting custom cost limits, so you are alerted if expenses exceed thresholds. This feature helps businesses prevent unexpected overages and optimize their AI investments.
Highlights
- Benefits of LLMInsight.ai: - Governance & Control - Cost Monitoring/Spend Control - LLM Usage Insights Across Departments - Single Interface for All LLMs - Automated Alerts - Automated Invoices
- Single Chat Window: - Unified chat window in LLM Insight for interacting with multiple LLM providers and models through a single interface. - Streamlines workflows by eliminating the need to switch between different platforms. - Supports popular models such as OpenAI, Anthropic, and Gemini for easy comparisons and seamless integration.
- Chargeback and Invoicing: - The chargeback and invoicing system in LLM Insight enables businesses to allocate AI costs to specific projects, teams, or departments. - It provides detailed reporting on AI usage, helping organizations maintain accurate financial records. - The platform supports generating invoices based on usage, ensuring that each team pays for its exact share of the costs.
Details
Sold by
Delivery method
Delivery option
AMI with CFN template
Latest version
Operating system
Ubuntu 2204
Deployed on AWS
Unlock automation with AI agent solutions
Fast-track AI initiatives with agents, tools, and solutions from AWS Partners.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Pricing and entitlements for this product are managed through an external billing relationship between you and the vendor. You activate the product by supplying a license purchased outside of AWS Marketplace, while AWS provides the infrastructure required to launch the product. AWS Subscriptions have no end date and may be canceled any time. However, the cancellation won't affect the status of the external license.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Vendor refund policy
Refunds are generally not available. Exceptions are handled case-by-case through a support request.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA)Â .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
AMI with CFN template
This CloudFormation template deploys llminsights application by launching and configuring an EC2 instance, installing Docker, setting up environment variables, and running several Docker containers. Here’s a detailed description of this delivery option:
-
Infrastructure Setup: It create EC2 instance and runs already downloaded application docker images. It also defines a security group to manage inbound traffic, allowing SSH from a specified IP and HTTPS from anywhere.
-
EC2 Instance Configuration: An EC2 instance (t3.large) is provisioned with a 100 GiB EBS volume. The instance's UserData script handles essential configurations upon launch.
-
Parameterization: The template is highly customizable through parameters for subdomain, API keys, security credentials, and Cognito details, allowing for flexible deployments.
-
Outputs: Key deployment information like instance ID, public IP, private IP, and the ECR login role ARN are provided as stack outputs for easy reference.
CloudFormation Template (CFT)
AWS CloudFormation templates are JSON or YAML-formatted text files that simplify provisioning and management on AWS. The templates describe the service or application architecture you want to deploy, and AWS CloudFormation uses those templates to provision and configure the required services (such as Amazon EC2 instances or Amazon RDS DB instances). The deployed application and associated resources are called a "stack."
Version release notes
llminsight with aws enabled sso
Additional details
Usage instructions
To configure and run your software using this CloudFormation template, follow these steps:
- Prerequisites:
- An active AWS account.
- AWS CLI configured with appropriate permissions, or access to the AWS Management Console.
- An existing EC2 KeyPair in the region for SSH access to the instance.
- Your public IP address in CIDR notation (e.g., 203.0.113.25/32) to allow SSH access.
- An OpenAI API Key.
- Cognito User Pool details: User Pool ID, App Client ID, Region, and App Client Secret.
- AWS Client ID and Client Secret, representing an IAM user or application with necessary permissions to access EC2, S3, VPC, and Cognito services.
- Deploying the CloudFormation Stack:
- You can deploy the stack via the AWS Management Console
- Navigate to the CloudFormation service in the AWS Management Console.
- Click "Create stack" and then "With new resources (standard)".
- For "Specify template", choose "Upload a template file" and upload the llminsight-cfn.yaml file. Click "Next".
- For "Specify stack details":
- Stack name: Enter a unique name for your stack (e.g., llminsight-deployment).
- Parameters: Fill in the following parameters with your specific values:
- Subdomain: Enter your desired customer subdomain name (e.g., mycompany). - This will form URLs like mycompany.llminsight.ai and mycompany-api.llminsight.ai.
- OpenAIAPIKey: Provide your OpenAI API key.
- KeyName: Select an existing EC2 KeyPair from the dropdown.
- MyIP: Enter your IP address in CIDR notation (e.g., 203.0.113.25/32).
- ImageId: You can leave the default
- CognitoUserPoolId: Your Cognito User Pool ID.
- CognitoClientId: Your Cognito App Client ID.
- CognitoRegion: The AWS region where your Cognito User Pool is located (e.g., us-east-1).
- CognitoClientSecret: Your Cognito App Client Secret.
- AwsClientId: Your AWS Client ID.
- AwsClientSecret: Your AWS Client Secret.
Click "Next". For "Configure stack options", you can leave the defaults or add tags. Click "Next". For "Review", review all the details. Check the box "I acknowledge that AWS CloudFormation might create IAM resources." Click "Create stack".
LLMinSight is offered under a BYOL (Bring Your Own License) model with a 2-month free tier. Please be aware that the license cost does not include any charges for AWS infrastructure resources such as EC2 instances, data transfer, or any usage costs related to OpenAI or other generative AI APIs that the user configures. These costs will be incurred separately and must be paid directly by the user.
Resources
Vendor resources
Support
Vendor support
For any support-related questions, please reach out to us through our website or feel free to email us directly at sstofficial@sstglobal.netÂ
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Gladia Speech-To-Text
By Gladia
Gladia provides a speech-to-text and audio intelligence API, powering companies with transcription, translation, and insights powered by best-in-class generative AI models.
Fiddler AI (Lite Version)
By Fiddler AI
Fiddler is the pioneer in AI Observability for responsible AI. The unified environment provides centralized controls and actionable insights to operationalize ML models, GenAI, and LLM applications with trust. Monitoring, analytics, and security capabilities address the unique challenges of building in-house and secure LLM and MLOps at scale. Fiddler helps you grow into advanced capabilities and build a framework for responsible AI practices.
Fortune 500 organizations use Fiddler across pre-production and production to deliver high performance AI, reduce costs, and be responsible in governance.
Fiddler AI (Lite Version) includes observability for 1 model, .5 GB of Data/Monthly, and our lowest level of support. No LLM functionality is included.
Zendata Privacy
By zendata
Zendata offers comprehensive data protection, providing data usage context, identifying third-party risks, and ensuring policy alignment. Using AI, it unifies data sources for deeper insights while maintaining security and compliance. Zendata delivers preventive, detective, and corrective controls, and deploys privacy controls on LLM models to prevent data breaches.
Zendata Privacy
By zendata
Zendata offers comprehensive data protection, providing data usage context, identifying third-party risks, and ensuring policy alignment. Using AI, it unifies data sources for deeper insights while maintaining security and compliance. Zendata delivers preventive, detective, and corrective controls, and deploys privacy controls on LLM models to prevent data breaches.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.