LiteRegistry CLI¶
Entry point installed by the package:
All subcommands are Python Fire functions:
--help works on the root and on each subcommand (literegistry gateway --help).
Subcommands at a glance¶
| Subcommand | Role | Deeper docs |
|---|---|---|
redis |
Start Redis (registry backend) | Registry, Runtimes |
vllm |
Launch vLLM + register | vLLM & SGLang |
sglang |
Launch SGLang + register | vLLM & SGLang |
gateway |
Canonical composable gateway with strict affinity | Gateway, Podman |
old-gateway |
Legacy gateway retained for compatibility | Gateway |
podman |
Rootless stateful container server registered in Redis | Podman |
code |
Stateless Python workers | Code & Terminal |
terminal |
Restricted log pipelines | Code & Terminal |
console |
Streamlit ops dashboard | Console |
summary |
Print replica counts per model | Registry |
detail |
Print URI + metadata per replica | Registry |
Registry URL convention¶
Almost every command takes --registry:
| Value | Backend |
|---|---|
redis://host:port |
Redis |
sqlite:///absolute/path/registry.sqlite3 |
SQLite database file |
file:///absolute/path/to/dir |
Filesystem KV store |
Pass the same --registry to workers, gateway, and inspect commands.
literegistry redis¶
Start a Redis server, publish a Redis-PING-verified endpoint, and print both
REDIS_URL=redis://hostname:PORT and its head-registry URI.
literegistry redis --port 6379
literegistry redis --runtime local --foreground --port 6379
literegistry redis --runtime local --foreground \
--head_registry=file:///shared/deployment/bootstrap
| Argument | Default | Meaning |
|---|---|---|
port |
6379 |
Listen port |
runtime |
apptainer |
apptainer or local |
foreground |
False |
Stay attached to this process |
log |
None |
Optional log file |
image |
redis_7-alpine.sif |
Apptainer image |
image_source |
docker://redis:7-alpine |
Pull source |
pull_image |
True |
Pull if SIF missing |
redis_server_path |
None |
Host binary when runtime=local |
advertise_host |
node FQDN | Host stored in the Redis endpoint URL |
head_registry |
unique file:// registry in /tmp |
Stable file://, sqlite://, or redis:// registry containing the current Redis endpoint |
coordination_dir |
None |
Backward-compatible name for head_registry |
coordination_ttl_seconds |
60 |
Endpoint TTL |
coordination_refresh_interval |
30 |
PING/refresh interval |
coordination_startup_timeout |
600 |
Startup health deadline |
coordination_healthcheck_timeout |
2 |
Per-PING timeout |
persistence |
True |
Enable AOF persistence |
data_dir |
derived for file/SQLite heads | AOF storage location; explicit shared path recommended with a Redis head |
appendfsync |
everysec |
always, everysec, or no |
workdir / bind / env |
None |
Apptainer workdir, binds, env |
apptainer_cleanenv |
True |
--cleanenv |
apptainer_executable |
apptainer |
Binary name |
apptainer_extra_args |
None |
Extra Apptainer flags |
The head record is a bootstrap/liveness endpoint, not a normal routable service registration. On clean shutdown it is removed; after a crash its TTL expires.
For cross-node Beaker resumption, put head_registry or data_dir on shared
storage. Disable durable state explicitly with --persistence=False.
literegistry vllm / literegistry sglang¶
Start a model server, register under model_path=<model>, heartbeat while healthy.
Extra flags are forwarded to the backend (--tensor-parallel-size, --tp-size, …).
literegistry vllm \
--model meta-llama/Llama-3.1-8B-Instruct \
--registry redis://login-node:6379 \
--tensor-parallel-size 4
literegistry sglang \
--model meta-llama/Llama-3.1-8B-Instruct \
--registry redis://login-node:6379 \
--tp-size 1
| Argument | Default | Meaning |
|---|---|---|
model |
Llama 3.1 8B Instruct | Model id/path → registry model_path |
host |
0.0.0.0 |
Backend bind host |
registry |
path or Redis (wrapper default) | Registration target |
port |
random 8000–12000 |
Backend port |
runtime |
apptainer |
local or apptainer |
image / image_source |
backend-specific SIF / Docker URI | Container image |
pull_image |
True |
Pull if missing |
bind / env / workdir |
None (+ auto HF binds/env) |
Container mounts & env |
apptainer_nv |
True |
GPU (--nv) |
apptainer_cleanenv |
True |
--cleanenv |
apptainer_executable |
apptainer |
Binary |
apptainer_extra_args |
None |
Extra flags |
**kwargs |
— | Passed through as --kebab-case flags |
Full flag tables and image names: vLLM & SGLang, Runtimes.
literegistry gateway¶
OpenAI-compatible proxy with bandit load balancing.
literegistry gateway --registry redis://login-node:6379 --port 8080
literegistry gateway --registry redis://login-node:6379 --port 8080 --workers 4
| Argument | Default | Meaning |
|---|---|---|
registry |
cluster Redis URL | Discovery backend |
port |
8080 |
Listen port |
workers |
1 |
Uvicorn workers (>1 = multi-process) |
timeout |
61 |
Model request timeout (s) |
python_timeout |
20 |
/python timeout |
python_max_retries |
3 |
/python attempts |
python_retry_budget_seconds |
20 |
/python retry wall budget |
terminal_timeout |
20 |
/terminal timeout |
terminal_max_retries |
2 |
/terminal attempts |
terminal_retry_budget_seconds |
20 |
/terminal retry wall budget |
Endpoints: /health, /session-stats, /v1/models, /v1/completions,
/classify, /python, /terminal — see Gateway.
literegistry gateway / literegistry podman¶
The gateway owns the strict-affinity lifecycle used by Podman: handshake,
terminal commands, and close/delete. The Podman server registers as
model_path=podman.
literegistry gateway \
--registry redis://registry:6379 \
--port 8080 \
--workers 8
literegistry podman \
--registry redis://registry:6379 \
--host 0.0.0.0 \
--port 8091 \
--allow-non-loopback=True \
--advertise-host podman-node \
--advertise-port 8091 \
--image python:3.12-slim
The gateway prints GATEWAY_URL=.... For managed host-network deployments,
the Podman server requires the explicit --allow-non-loopback=True flag. See
Podman for the request lifecycle and rootless requirements.
literegistry code¶
Stateless Python executor; registers as model_path=python.
| Argument | Default | Meaning |
|---|---|---|
host |
0.0.0.0 |
Bind host |
port |
1212 |
Listen port |
registry |
Redis URL | Registration target |
heartbeat_interval |
4 |
Heartbeat period (s) |
pool_size |
CPU count | Process pool size |
max_queue_size |
None |
Admission queue cap |
default_tools |
csv of common libs | Tools in default namespace |
preimport_tools |
all known specs | Pre-imported at boot |
title |
Stateless Python Code Executor |
Service title |
outer_timeout_grace_seconds |
1 |
Extra seconds past max_runtime |
log_level |
INFO |
Log level |
Request shape via gateway: Code & Terminal.
literegistry terminal¶
Restricted stdin pipelines; registers as model_path=terminal.
| Argument | Default | Meaning |
|---|---|---|
host |
0.0.0.0 |
Bind host |
port |
1213 |
Listen port |
registry |
Redis URL | Registration target |
heartbeat_interval |
30 |
Heartbeat period (s) |
max_output_bytes |
1048576 |
Stdout capture cap |
max_response_chars |
None |
Optional response truncate |
command_path |
None |
Extra PATH dir for tools |
Allowed commands: rg, grep, awk, sed, jq, xsv, pandoc, sort,
uniq, tr, cut, head, tail, wc, cat, nl, echo. Details:
Code & Terminal.
literegistry bm25¶
Local Lucene BM25 search over a JSONL corpus. It registers under
model_path=localsearch:<corpus stem> when registry is provided.
python -m pip install 'literegistry[bm25]'
literegistry bm25 \
--corpus_jsonl=/data/corpus.jsonl \
--lucene_index_dir=/data/lucene-index \
--registry=redis://login-node:6379 \
--service_name=localsearch:corpus \
--port=1214
| Argument | Default | Meaning |
|---|---|---|
corpus_jsonl |
environment | Source JSONL corpus |
lucene_index_dir |
environment | Existing Pyserini/Lucene index |
registry |
optional | Registration target |
service_name |
corpus stem | Registered model_path |
advertised_host |
FQDN | Host published to the registry |
host |
0.0.0.0 |
Bind host |
port |
8000 |
Listen port |
workers |
1 |
Uvicorn workers; registered mode requires one |
The service accepts direct query/topn requests and gateway-compatible
mode="query" or mode="url" requests.
literegistry console¶
Streamlit dashboard for gateway / vLLM / registry views.
literegistry console \
--logs /path/to/logs \
--registry redis://login-node:6379 \
--server-address 127.0.0.1 \
--server-port 8765
| Argument | Default | Meaning |
|---|---|---|
server_address / address |
127.0.0.1 |
Streamlit bind host |
server_port / port |
8765 |
Streamlit bind port |
ngrok |
True |
Try ngrok tunnel if installed |
logs |
derived | Log root(s) to scan |
logs_dir |
package gateway logs | Legacy single gateway path |
registry |
cluster Redis URL | For summary polling |
slurm_logs_dir / vllm_logs_dir |
package defaults | vLLM/Slurm logs |
seed_recent |
True |
Seed from recent tails |
poll_seconds |
0.5 |
Log poll interval |
window |
"1 hour" |
Gateway chart window |
refresh / refresh_seconds |
True / 5 |
UI auto-refresh |
poll_registry / registry_poll_seconds |
True / 5 |
Run summary periodically |
show_vllm |
True |
vLLM panel |
vllm_newest_files |
80 |
Max newest log files |
vllm_tail_lines |
1000 |
Tail depth |
vllm_window |
"2 hours" |
vLLM chart window |
Full guide: Console.
literegistry summary¶
Print model_path : replica_count for active servers.
literegistry summary --registry redis://login-node:6379
literegistry summary --registry /shared/fs/registry
literegistry summary --head_registry sqlite:///weka/shared/head.sqlite3
literegistry summary --registry head+file:///weka/shared/head
| Argument | Default | Meaning |
|---|---|---|
registry |
cluster Redis URL | Backend to inspect |
head_registry |
None |
Stable file, SQLite, or Redis head that advertises the live data-plane Redis |
timeout |
10 seconds |
Maximum time to resolve and query the live registry |
With a head registry, summary first resolves and verifies its current Redis
publisher, then reads the roster from that live Redis. It prints the stable
head URI, resolved live Redis URI, publisher ID, and publication age before
the normal replica counts. Passwords in either URI are redacted.
literegistry detail¶
Same grouping as summary, plus each replica’s uri and metadata.
literegistry detail --registry redis://login-node:6379
literegistry detail --head_registry file:///weka/shared/head
| Argument | Default | Meaning |
|---|---|---|
registry |
cluster Redis URL | Backend to inspect |
head_registry |
None |
Stable file, SQLite, or Redis head that advertises the live data-plane Redis |
timeout |
10 seconds |
Maximum time to resolve and query the live registry |
detail performs the same live resolution as summary, then prints each
service URI and metadata from the resolved Redis roster.
Fire usage tips¶
# Flags: both styles usually work
literegistry gateway --port 8080
literegistry gateway --port=8080
# Booleans
literegistry redis --foreground
literegistry console --ngrok=False
# Lists / nested values
literegistry console --logs='["/a/logs","/b/logs"]'
literegistry vllm --bind='["/data:/data","/models:/models"]'
# Underscores become kebab-case for backend passthrough
literegistry vllm --tensor_parallel_size=4 # → --tensor-parallel-size 4
Equivalent module forms (same Fire mains):
literegistry gateway --registry redis://… --port 8080
python -m literegistry.gateway --registry redis://… --port 8080
python -m literegistry.services.vllm_wrapper --model … --registry redis://…
Minimal cluster recipe¶
# 1) Registry
literegistry redis --port 6379
# note REDIS_URL=...
# 2) Workers (on GPU nodes)
literegistry vllm --model meta-llama/Llama-3.1-8B-Instruct \
--registry redis://LOGIN:6379 --tensor-parallel-size 1
literegistry code --registry redis://LOGIN:6379
literegistry terminal --registry redis://LOGIN:6379
# 3) Front door
literegistry gateway --registry redis://LOGIN:6379 --port 8080 --workers 4
# 4) Inspect / watch
literegistry summary --registry redis://LOGIN:6379
literegistry console --logs /path/to/logs --registry redis://LOGIN:6379
Next: docs index · Registry · Gateway