LiteLLM Replica - Documentation

📚 Documentation

LiteLLM Replica Documentation

Complete guide to using the LiteLLM Replica unified gateway for 100+ LLM providers

Documentation

Navigate through our guides

🚀 Getting Started

Quick start guide to using LiteLLM Replica

Installation

pip install litellm-replica
# or
docker run -p 8000:8000 litellm-replica

Basic Usage

Start making requests to any LLM provider through our unified API:

curl -X POST https://ds36u4p80.drytis.ai/v1/api/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Hello!"}]
}'

📖 API Reference

Complete API documentation

Chat Completions

POST
/v1/chat/completions

Create a chat completion with any supported model

Models

GET
/v1/models

List all available models across providers

Health Check

GET
/health

Check system health and status

🔌 LLM Providers

Supported LLM providers and configuration

OpenAI (GPT-3.5, GPT-4)

Supported

Anthropic (Claude)

Supported

Azure OpenAI

Supported

Google (Gemini)

Supported

AWS Bedrock

Supported

Cohere

Supported

Hugging Face

Supported

Ollama (Local)

Supported

🔧 MCP Integration

Model Context Protocol integration guide

LiteLLM Replica supports Model Context Protocol (MCP) for extending LLM capabilities with tools and functions.

# Configure MCP server
POST /mcp/servers
{
"name": "file-system",
"endpoint": "ws://localhost:3001",
"capabilities": ["read_file", "write_file"]
}

🔐 Authentication

API key management and security

Secure your API access with authentication tokens and rate limiting.

curl -H "Authorization: Bearer YOUR_API_KEY" \
https://ds36u4p80.drytis.ai/v1/api/v1/chat/completions

💡 Examples

Code examples and use cases

Python Example

import requests
response = requests.post(
"https://ds36u4p80.drytis.ai/v1/api/v1/chat/completions",
json={
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Hello!"}]
}
)

JavaScript Example

const response = await fetch('/v1/chat/completions', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'gpt-3.5-turbo',
messages: [{ role: 'user', content: 'Hello!' }]
})
})

🔍 Troubleshooting

Common issues and solutions

Connection Issues

If you're experiencing connection issues, check your network configuration and firewall settings.

API Key Errors

Ensure your API keys are properly configured for each provider in the admin panel.

Rate Limiting

If you're hitting rate limits, consider upgrading your plan or implementing request queuing.