LiteLLM

Różne notatki. Prace w trakcie.

aiohappyeyeballs==2.7.1
aiohttp==3.14.3
aiosignal==1.4.0
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1  
APScheduler==3.11.3
attrs==26.1.0 
azure-core==1.41.0       
azure-identity==1.25.3
azure-storage-blob==12.30.1
backoff==2.2.1  
boto3==1.43.89
botocore==1.43.89
certifi==2026.7.22       
cffi==2.1.1          
charset-normalizer==3.5.1
click==8.5.0 
croniter==6.2.4
cryptography==50.0.1
distro==1.9.0               
dnspython==2.8.0    
email-validator==2.3.0  
Expression==5.7.0
fastapi==0.141.1
fastapi-sso==0.22.0
fastuuid==0.14.0
filelock==3.32.5
frozenlist==1.8.0    
fsspec==2026.7.0
granian==2.8.2   
gunicorn==23.0.0
h11==0.16.0       
hf-xet==1.6.0
hiredis==3.4.1
httpcore==1.0.9  
httpx==0.28.1        
httpx-sse==0.4.3
huggingface_hub==1.30.0
idna==3.19        
importlib_metadata==8.9.0
inquirerpy==0.3.4
isodate==0.7.2          
Jinja2==3.1.6            
jiter==0.16.0 
jmespath==1.1.0
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
litellm==1.99.0
litellm-enterprise==0.1.59
litellm-proxy-extras==0.4.89
markdown-it-py==4.2.0
MarkupSafe==3.0.3                                                                          
mcp==1.29.1
mdurl==0.1.2
msal==1.38.0
msal-extensions==1.3.1
multidict==6.7.1
nodeenv==1.10.0
numpy==2.5.2
oauthlib==3.3.1
openai==2.54.0
orjson==3.12.0
packaging==26.3
pfzy==0.3.4
polars==1.44.1
polars-runtime-32==1.44.1
prisma==0.13.0
prompt_toolkit==3.0.53
propcache==0.5.2
pycparser==3.0
pydantic==2.13.5
pydantic-settings==2.15.0
pydantic_core==2.46.5
Pygments==2.21.0
PyJWT==2.13.0
PyNaCl==1.6.2
pyroscope-io==0.8.16
python-dateutil==2.9.0.post0
python-dotenv==1.2.3
python-multipart==0.0.32
PyYAML==6.0.3
redis==8.1.0
referencing==0.37.0
regex==2026.9.3
requests==2.34.2
RestrictedPython==8.5
rich==13.9.4
rpds-py==2026.6.3
rq==2.12.0
s3transfer==0.19.2
six==1.17.0
sniffio==1.3.1
soundfile==0.14.0
sse-starlette==3.4.11
starlette==1.6.0
tiktoken==0.14.0
tokenizers==0.23.2
tomlkit==0.15.1
tqdm==4.70.0
typing-inspection==0.4.4
typing_extensions==4.16.0
tzlocal==5.4.4
urllib3==2.7.0
uvicorn==0.52.4
uvloop==0.22.1
wcwidth==0.8.3
websockets==15.0.1
yarl==1.24.5
zipp==4.1.0

model_list:
  - model_name: granite
    litellm_params:
      model: ollama/granite4.2:8b
      api_base: http://si.local:11434
 
  - model_name: gemma
    litellm_params:
      model: ollama/gemma4:12b
      api_base: http://si.local:11434
 
  - model_name: qwen
    litellm_params:
      model: ollama/qwen3.5:9b
      api_base: http://si.local:11434
 
  - model_name: bielik
    litellm_params:
      model: ollama/bielik-11b:Q5_K_M
      api_base: http://si.local:11434
 
router_settings:
  routing_strategy: latency-based-routing
  num_retries: 2
  timeout: 601
  fallbacks:
    - granite: ["qwen"]
    - bielik: ["gemma"]

general_settings:
  master_key: admin
  store_prompts_in_spend_logs: true
  ###disable_env_credential_login: true
  database_url: "postgresql://[rola]:[haslo]@[baza.local]:5432/[nazwa_bazy]"

# \c litellm
You are now connected to database "litellm" as user "postgres".
# grant all ON database litellm to litellm;
# grant CONNECT on database litellm to litellm;
# GRANT ALL PRIVILEGES ON SCHEMA public TO litellm;
# ALTER ROLE "litellm" WITH LOGIN;

Będąc w katalogu z venv.

prisma generate --schema lib/python3.12/site-packages/litellm/proxy/schema.prisma

litellm --config litellm_config.yaml --port 4040

Można się zalogować jako użytkownik admin z hasłem admin na stronie http://127.0.0.1:4040/ui/login/ . Potem na stronie http://127.0.0.1:4444/ui/api-keys/ można wyklikać klucz API, powiązać go z modelami, grupami i użytkownikami.

Uruchomienie AiderAI bez połączenia z Ollama, ale z połączeniem do LiteLLM.

AIDER_ANALYTICS_DISABLE=true AIDER_CHECK_UPDATE=false AIDER_YES_ALWAYS=true AIDER_DARK_MODE=true LITELLM_PROXY_API_BASE=http://127.0.0.1:4040 LITELLM_PROXY_API_KEY=sk-[token_w_litellm] aider --model=litellm_proxy/granite