Skip to main content

Cloud APIM - LLM Tokens rate limiting

cp:otoroshi_plugins.com.cloud.apim.otoroshi.extensions.aigateway.plugins.LlmTokensRateLimitingValidator

This plugin limits the number of LLM used on a period of time.

official documentation from otoroshi manual

categories:​

  • Cloud APIM
  • AI - LLM
  • AccessControl

default configuration:​

{
  "window_millis" : "10000",
  "throttling_quota" : "1000",
  "group_expr" : "${route.id}"
}