AI Gateway

Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API.

AI Gateway is now available in Otoroshi Managed instances and Serverless products.

The logo of the authify

Integrating AI in your API Gateways allows seamless access to machine learning models and AI services.

Our AI Gateway securely manage requests, process data, and deliver AI-powered insights efficiently.

Whether you're incorporating predictive analytics, natural language processing, or other AI functions, Cloud APIM’s AI Gateways make integration easy, scalable, and secure.

Benefits

...

Unified interface

Use our all-in-one interface : Simplify interactions and minimize integration hassles

...

Supports Multiple providers

10+ LLM providers supported right now, a lot more coming. Use OpenAI, Azure OpenAI, Ollama, Mistral, Anthropic, Cohere, Gemini, Groq, Huggingface and OVH AI Endpoints

...

Semantic cache

Speed up repeated queries, enhance response times, and reduce costs.

...

Load balancing

Ensure optimal performance by distributing workloads across multiple providers

...

Custom quotas

Manage LLM tokens quotas per consumer and optimise costs

...

Observability and reporting

Every LLM request is audited with details about the consumer, the LLM provider and usage. All those audit events are exportable using multiple methods for further reporting

Frequently Asks Questions

An AI Gateway is similar to an API Gateway but is designed specifically for handling requests that involve artificial intelligence or machine learning tasks. It routes, manages, and secures AI-based interactions, such as calls to machine learning models, ensuring smooth, efficient, and secure integration of AI services into applications.

Cloud APIM AI Plugins are integrated in both our offers Serverless and Otoroshi Managed instances :)

You can use OpenAI, Azure OpenAI, Ollama, Mistral, Anthropic, Cohere, Gemini, Groq, Huggingface and OVH AI Endpoints. 10+ LLM providers supported right now, a lot more coming

Yes, Semantic cache is included in our offers Otoroshi Managed instances and Serverless. With semantic cache you can speed up repeated queries, enhance response times, and reduce costs.