AI Gateway
Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API.
AI Gateway is now available in Otoroshi Managed instances and Serverless products.
Integrating AI in your API Gateways allows seamless access to machine learning models and AI services.
Our AI Gateway securely manage requests, process data, and deliver AI-powered insights efficiently.
Whether you're incorporating predictive analytics, natural language processing, or other AI functions, Cloud APIM’s AI Gateways make integration easy, scalable, and secure.
Use our all-in-one interface : Simplify interactions and minimize integration hassles
10+ LLM providers supported right now, a lot more coming. Use OpenAI, Azure OpenAI, Ollama, Mistral, Anthropic, Cohere, Gemini, Groq, Huggingface and OVH AI Endpoints
Speed up repeated queries, enhance response times, and reduce costs.
Ensure optimal performance by distributing workloads across multiple providers
Manage LLM tokens quotas per consumer and optimise costs
Every LLM request is audited with details about the consumer, the LLM provider and usage. All those audit events are exportable using multiple methods for further reporting
An AI Gateway is similar to an API Gateway but is designed specifically for handling requests that involve artificial intelligence or machine learning tasks. It routes, manages, and secures AI-based interactions, such as calls to machine learning models, ensuring smooth, efficient, and secure integration of AI services into applications.
Cloud APIM AI Plugins are integrated in both our offers Serverless and Otoroshi Managed instances :)
You can use OpenAI, Azure OpenAI, Ollama, Mistral, Anthropic, Cohere, Gemini, Groq, Huggingface and OVH AI Endpoints. 10+ LLM providers supported right now, a lot more coming
Yes, Semantic cache is included in our offers Otoroshi Managed instances and Serverless. With semantic cache you can speed up repeated queries, enhance response times, and reduce costs.