Portkey-AI / gateway

What it does

• Fallbacks. Fallback to another provider or model on failed requests using the LLM gateway. You can specify the errors on which to trigger the fallback. Improves reliability of your application.
• Automatic Retries. Automatically retry failed requests up to 5 times. An exponential backoff strategy spaces out retry attempts to prevent network overload.
• Load Balancing. Distribute LLM requests across multiple API keys or AI providers with weights to ensure high availability and optimal performance.
• Request Timeouts. Manage unruly LLMs & latencies by setting up granular request timeouts, allowing automatic termination of requests that exceed a specified duration.
• Multi-modal LLM Gateway. Call vision, audio (text-to-speech & speech-to-text), and image generation models from multiple providers — all using the familiar OpenAI signature