====== LiteLLM ======
Różne notatki. Prace w trakcie.
==== requirements.txt ====
aiohappyeyeballs==2.7.1
aiohttp==3.14.3
aiosignal==1.4.0
annotated-doc==0.0.5
annotated-types==0.8.0
anyio==4.15.1
APScheduler==3.11.3
attrs==26.1.0
azure-core==1.41.0
azure-identity==1.25.3
azure-storage-blob==12.30.1
backoff==2.2.1
boto3==1.43.89
botocore==1.43.89
certifi==2026.7.22
cffi==2.1.1
charset-normalizer==3.5.1
click==8.5.0
croniter==6.2.4
cryptography==50.0.1
distro==1.9.0
dnspython==2.8.0
email-validator==2.3.0
Expression==5.7.0
fastapi==0.141.1
fastapi-sso==0.22.0
fastuuid==0.14.0
filelock==3.32.5
frozenlist==1.8.0
fsspec==2026.7.0
granian==2.8.2
gunicorn==23.0.0
h11==0.16.0
hf-xet==1.6.0
hiredis==3.4.1
httpcore==1.0.9
httpx==0.28.1
httpx-sse==0.4.3
huggingface_hub==1.30.0
idna==3.19
importlib_metadata==8.9.0
inquirerpy==0.3.4
isodate==0.7.2
Jinja2==3.1.6
jiter==0.16.0
jmespath==1.1.0
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
litellm==1.99.0
litellm-enterprise==0.1.59
litellm-proxy-extras==0.4.89
markdown-it-py==4.2.0
MarkupSafe==3.0.3
mcp==1.29.1
mdurl==0.1.2
msal==1.38.0
msal-extensions==1.3.1
multidict==6.7.1
nodeenv==1.10.0
numpy==2.5.2
oauthlib==3.3.1
openai==2.54.0
orjson==3.12.0
packaging==26.3
pfzy==0.3.4
polars==1.44.1
polars-runtime-32==1.44.1
prisma==0.13.0
prompt_toolkit==3.0.53
propcache==0.5.2
pycparser==3.0
pydantic==2.13.5
pydantic-settings==2.15.0
pydantic_core==2.46.5
Pygments==2.21.0
PyJWT==2.13.0
PyNaCl==1.6.2
pyroscope-io==0.8.16
python-dateutil==2.9.0.post0
python-dotenv==1.2.3
python-multipart==0.0.32
PyYAML==6.0.3
redis==8.1.0
referencing==0.37.0
regex==2026.9.3
requests==2.34.2
RestrictedPython==8.5
rich==13.9.4
rpds-py==2026.6.3
rq==2.12.0
s3transfer==0.19.2
six==1.17.0
sniffio==1.3.1
soundfile==0.14.0
sse-starlette==3.4.11
starlette==1.6.0
tiktoken==0.14.0
tokenizers==0.23.2
tomlkit==0.15.1
tqdm==4.70.0
typing-inspection==0.4.4
typing_extensions==4.16.0
tzlocal==5.4.4
urllib3==2.7.0
uvicorn==0.52.4
uvloop==0.22.1
wcwidth==0.8.3
websockets==15.0.1
yarl==1.24.5
zipp==4.1.0
==== Przykładowy plik litellm_config.yaml ====
model_list:
- model_name: granite
litellm_params:
model: ollama/granite4.2:8b
api_base: http://si.local:11434
allow_client_keepalive_override: true
temperature: 0.6
- model_name: gemma
litellm_params:
model: ollama/gemma4:12b
api_base: http://si.local:11434
allow_client_keepalive_override: true
temperature: 0.8
- model_name: qwen
litellm_params:
model: ollama/qwen3.5:9b
api_base: http://si.local:11434
allow_client_keepalive_override: true
temperature: 0.8
- model_name: bielik
litellm_params:
model: ollama/bielik-11b:Q5_K_M
api_base: http://si.local:11434
allow_client_keepalive_override: true
temperature: 0.8
router_settings:
routing_strategy: latency-based-routing
num_retries: 2
timeout: 601
fallbacks:
- granite: ["qwen"]
- bielik: ["gemma"]
general_settings:
master_key: admin
###disable_env_credential_login: true
###store_model_in_db: true
store_prompts_in_spend_logs: true
database_url: "postgresql://rola:haslo@host:5432/bazadanych"
==== Rola i uprawnienia w bazie Postgres ====
# \c litellm
You are now connected to database "litellm" as user "postgres".
# grant all ON database litellm to litellm;
# grant CONNECT on database litellm to litellm;
# GRANT ALL PRIVILEGES ON SCHEMA public TO litellm;
# ALTER ROLE "litellm" WITH LOGIN;
==== Utworzenie schematu bazy ====
Będąc w katalogu z venv.
prisma generate --schema lib/python3.12/site-packages/litellm/proxy/schema.prisma
==== Utworzenie aplikacji ====
litellm --config litellm_config.yaml --port 4040
Można się zalogować jako użytkownik //admin// z hasłem //admin// na stronie http://127.0.0.1:4040/ui/login/ . Potem na stronie http://127.0.0.1:4040/ui/api-keys/ można wyklikać klucz API, powiązać go z modelami, grupami i użytkownikami.
==== AiderAI ====
Uruchomienie AiderAI bez połączenia z Ollama, ale z połączeniem do LiteLLM.
AIDER_ANALYTICS_DISABLE=true AIDER_CHECK_UPDATE=false AIDER_YES_ALWAYS=true AIDER_DARK_MODE=true AIDER_SHOW_MODEL_WARNINGS=false AIDER_MAP_TOKENS=1024 LITELLM_PROXY_API_BASE=http://127.0.0.1:4040 LITELLM_PROXY_API_KEY=sk-[token_w_litellm] aider --model=litellm_proxy/granite
==== Shell GPT ====
Plik konfiguracyjny //~/.config/shell_gpt/.sgptrc// komunikujący klienta z LiteLLM.
CHAT_CACHE_PATH=/tmp/chat_cache
CACHE_PATH=/tmp/cache
CHAT_CACHE_LENGTH=100
CACHE_LENGTH=100
REQUEST_TIMEOUT=60
DEFAULT_MODEL=gemma
DEFAULT_COLOR=magenta
ROLE_STORAGE_PATH=/home/[uzytkownik]/.config/shell_gpt/roles
DEFAULT_EXECUTE_SHELL_CMD=false
DISABLE_STREAMING=false
CODE_THEME=dracula
OPENAI_FUNCTIONS_PATH=/home/[uzytkownik]/.config/shell_gpt/functions
OPENAI_USE_FUNCTIONS=false
SHOW_FUNCTIONS_OUTPUT=false
API_BASE_URL=http://127.0.0.1:4040
PRETTIFY_MARKDOWN=true
USE_LITELLM=false
SHELL_INTERACTION=true
OS_NAME=auto
SHELL_NAME=auto
OPENAI_API_KEY=sk-[klucz w LiteLLM]
Działa to tak:
((sgpt) ) [[uzytkownk]@[host] sgpt]$ sgpt 'Which user am I?'
To determine which user you are currently logged in as on your CentOS Stream 10 system, you can use the following command in your bash shell:
whoami
Alternatively, you can check the USER environment variable:
echo $USER
These commands will return the username of the current session. If you are running with elevated privileges (e.g., via sudo), whoami will show the effective user identity.
----
{{:wiki:logi_litellm.jpg?400|}}
{{:wiki:litellm_06.jpg?400|}}