With Frosty AI, you can:
-
LLM Routing Interface: Easily connect, configure, and switch between different LLMs using a web-based UI.
-
Prompt Management: Centrally manage your prompts, including versioning and tracking, to maintain consistency and efficiency. (Coming Soon)
-
Analytics Dashboard: Log and visualize interactions, success rates, and other key metrics to gain insights into your AI operations.
-
AI Recommendations: Leverage AI-driven insights to improve your prompts and overall system performance. (Coming Soon)
-
Continuous Updates: Our team continuously adds and updates supported providers and models to keep you ahead in AI innovation.
Frosty AI Terms
Organization
The top-level entity that a company or enterprise creates when they sign up for Frosty AI. This is where all company-wide settings and management take place.
Workspaces
Sub-divisions within an Organization, tailored for different departments, teams, or projects. Workspaces serve as isolated environments where specific groups can manage their LLM interactions, analytics, and configurations.
Routers
The core feature within a Workspace. Routers are responsible for directing requests to different LLM providers. They allow users to configure which LLM to use and manage the flow of information without altering the underlying code.
Multiple Routers in a Workspace
In a single Workspace, you can create multiple Routers. This is particularly useful when your team or project needs to interact with different LLMs for various tasks. For example:
-
Task-Specific Routing: You might have one Router dedicated to handling customer support queries using a conversational LLM, while another Router is optimized for generating technical documentation using a different, more specialized LLM.
-
A/B Testing: You can set up multiple Routers to perform A/B testing between different LLM providers or configurations. This allows you to compare performance, response quality, and cost-effectiveness across different models, helping you make informed decisions on which LLM best suits your needs.
-
Redundancy and Failover: Having multiple Routers also provides redundancy. If one LLM provider experiences downtime, you can quickly switch to another Router connected to a different provider, ensuring continuous operation without any interruptions.
By leveraging multiple Routers, you can maximize flexibility, optimize performance, and ensure that your AI-driven workflows are robust and efficient.
Providers
The various LLMs that can be connected and switched between through Routers. These could include services like OpenAI, Anthropic, Meta, MistralAI, and more coming soon with the flexibility to bring your own model (coming soon).