GemAI Bot v2

GemAI Bot v2 is an open-source Telegram AI assistant framework focused on resilient real-world bot operations: multi-provider AI routing, streaming recovery, agentic web research, multimodal processing, long-term graph memory, Telegram Mini App surfaces, Docker deployment, and operator tooling.

Maintainer / OSS Status

• Primary maintainer: Le-Who
• Repository: https://github.com/Le-Who/gemaibotv2
• Production branch (checked-in workflows): vps_testai
• License: MIT, see LICENSE.
• Security policy: see SECURITY.md.
• Contributing guide: see CONTRIBUTING.md.
• Maintainer roadmap: see ROADMAP.md.
• Maintainer queue: see docs/MAINTAINER_QUEUE.md.

This is a production-oriented Telegram bot codebase with VPS deployment automation. It includes provider failover, user-facing recovery from model/API failures, authenticated Mini App surfaces, persistent memory and background-task ownership. Current deployment health must be verified separately.

Documentation status

Reviewed against checkout f44541a7 on 2026-09-20. This is a maintained application with deployment automation; a checkout alone does not establish current production health, traffic, provider availability or an open-PR count.

Start with the documentation index, agent agreements, architecture and audit findings. The changelog records history, not a current runtime specification.

Feature Overview

• Chat with direct OpenAI, Gemini, Opencode, OpenRouter and FreeTheAI routing, model selection, key rotation and recovery from partial responses. Interactive Gemini chat can start a second model after three seconds without text; at most two model routes run at once, and the first to produce text wins. Only the primary attempt races keys. A reserve uses one model and one key; further reserves run sequentially. A slow primary pair may overlap one reserve, with at most three provider calls per routed request. Independent requests and background Daily preparation retain their own execution budgets. One first-text deadline covers all keys and models: 12 seconds for Flash at low/off thinking, 20 at auto/medium, and 30 at high. Inline uses 6 seconds, or 10 with Search, and returns a ready standby immediately. See the latency policy and evidence limits. Router deadlines do not suspend otherwise healthy keys.
• The OpenAI live-answer profile uses Luna Flex without reasoning for ordinary chat and quick search, and Luna Standard for inline and Crocodile judgement. A local classifier routes text tasks with multiple constraints/dependencies to the separate chat.complex process, initially Terra Flex with low reasoning. The dashboard selects the model, reasoning override and Standard/Flex separately. OpenAI reasoning is controlled by the administrator; an absent override uses the model default, and saved user thinking cannot change OpenAI requests. User preferences remain available for Gemini. Flex waits up to 15 seconds for model text, including native GPT-6 web search. Telegram delivery has a separate budget. A Flex capacity error offers a manual retry without switching to Standard.
• Quick search (?), bounded agentic research (??), URL/document/photo understanding and deterministic weather/currency/crypto shortcuts.
• Document Q&A preserves the file explicitly selected in /documents; an unavailable selected file is not silently replaced with the latest upload. PDF/DOCX uploads have a 10 MB processing limit, checked before download when Telegram provides the size. Extracted text is capped at 100,000 characters; a truncated document is explicitly identified in the upload result.
• Typed streamed/completed response delivery, preserving final actions; long responses use Redis Reader, optional public Telegraph, or Telegram splitting.
• Inline answers, response tabs, collaborative boards and image generation. Inline text and image paths have their own routing; they are not a fixed promise that a particular model will win a race or meet a latency target.
• /draw and natural-language image requests with canvas controls. Gemini native image generation accepts legacy Imagen selections as aliases. Pollinations requires a server-side key and Pollen budget; FreeTheAI has a separate image path.
• Voice transcription, intent-aware chat/search routing, queued TTS and Mini App Live Audio. Confirmation depends on the voice flow; it is not universal.
• If a chat response has no first text after a minute, its placeholder offers a safe stop-and-retry action. Optional memory query expansion is limited to eight seconds so slow recall cannot hold up the main response indefinitely.
• Consent-gated private long-term memory, hybrid vector/text recall, provenance-aware graph writes and account-data controls. Group messages are not implicitly saved into private LTM.
• /memory offers full-text private reading with pagination, return to the list and individual deletion. Expired/deleted facts cannot be reopened; reading does not publish the fact through Reader or Telegraph.
• Daily Crocodile, 2048 and trivia. Players can change their daily game from any of the three Mini App screens or the bot's daily messages; the choice controls /dailycroc and their daily subscription. Reopening a completed game in chat shows the player's current result and leaderboard. The admin choice remains the default and is prepared seven days ahead, while the other games are prepared for today and tomorrow. Trivia saves answers in order and calculates ranked scores on the server from the game's pinned question revision. Reopening an unfinished trivia game restores the saved question and score; repeated submissions do not add points.
• Scheduled briefings, reminders, tarot and horoscopes. Horoscope settings provide on-demand today/tomorrow readings.
• Feature-gated natal reports with local city lookup, PyEphem planet calculations, local equal-house/angle math and optional LLM interpretation.
• Admin dashboard and daily-content controls, metrics, dependency validation and exact-SHA VPS deployment automation.

The public command/help catalog is app/bot_commands.py. Model defaults below describe code configuration, not independently verified provider entitlements. See Crocodile operations and natal readiness for their boundaries.

Non-Goals / Limitations

• Direct OpenAI uses the official SDK and a separate credential pool. OpenAI-compatible gateways retain their own routing and credentials.
• No ORM: persistence uses asyncpg and numbered SQL.
• Redis absence selects local fallbacks only where implemented; it does not make Reader, deferred delivery or every multi-replica feature equivalent.
• Provider quotas, response time, quality and model access are deployment-specific.
• Private-data erasure cannot retract copies already exported, shared or published to third-party services. Telegraph is disabled by default.
• Natal calculations support equal houses, not Placidus/Koch or a claim of Swiss Ephemeris parity; rollout still needs environment-specific verification.

Architecture

• Bot Container (tg-bot): A single async event loop runs both the Telegram webhook updater and the Quart web server via Hypercorn. Webhook mode: Quart receives updates at a derived-hash webhook path, validates the optional secret header, deduplicates updates and enqueues them for UserScopedUpdateProcessor(50). Rejected or backpressured requests do not receive an unconditional success response. Falls back to long-polling if WEBHOOK_URL is unset.
• Local Bot API Server (tg-api): Self-hosted telegram-bot-api container (aiogram/telegram-bot-api:latest) communicates with Telegram via MTProto. The bot sends HTTP requests to http://tg-api:8081/bot instead of api.telegram.org. Shared Docker volume (tg-api-data) enables zero-copy file access for voice/photo/document processing. File limit: 2 GB (vs 50 MB cloud API). Timezone: Europe/Kyiv.
• Media Cleanup Cron (tg-media-cleanup): Alpine-based sidecar container that runs a 60s-tick loop: (1) chmod -R g+rX on the shared volume to fix permission conflicts between telegram-bot-api (UID 101) and the bot container (GID 101), and (2) deletes cached media files older than 7 days every 24 hours to prevent disk exhaustion.
• Database (PostgreSQL): Source of truth for users, chats, messages, metrics, roles, and pgvector embeddings.
• Cache (Redis): Optional high-speed layer for caching rate limits and transient states.
• Third-Party APIs: Google Gemini (native SDK), Opencode Go (HTTPX), OpenRouter (HTTPX), JINA AI (HTTPX), Tavily (HTTPX).

Docker networking: All three containers share a tg-net bridge network. The shared volume tg-api-data is mounted at /var/lib/telegram-bot-api in both tg-api and tg-bot. The bot's web server binds to 127.0.0.1:$PORT on the host (not 0.0.0.0) — Caddy/Nginx reverse proxy is expected in front.

graph TD;
    User-->TelegramCloud[Telegram Cloud];
    TelegramCloud-->LocalAPI["tg-api<br/>Local Bot API Server<br/>MTProto ↔ REST"];
    LocalAPI-->BotHandler["tg-bot<br/>Python Bot + Quart"];
    Admin-->QuartServer["Quart Web Server<br/>/metrics, /health, /webapp"];

    BotHandler-->ProviderRouter;
    ProviderRouter-->Gemini[Google Gemini];
    ProviderRouter-->OpencodeGo[Opencode Go];
    ProviderRouter-->OpenRouter[OpenRouter];

    BotHandler-->Tavily[Tavily Search];

    BotHandler-->Cache[(Redis)];
    BotHandler-->DB[(PostgreSQL/pgvector)];
    QuartServer-->DB;

    LocalAPI-.->SharedVolume["tg-api-data<br/>/var/lib/telegram-bot-api<br/>Shared Docker Volume"];
    BotHandler-.->SharedVolume;
    Cleanup["tg-media-cleanup<br/>Alpine cron<br/>chmod + 7d prune"]-.->SharedVolume;

Loading

Repository Structure

Path	Purpose
app/	Core application logic (bot, web server, DB layer, handlers).
app/handlers/	Telegram command and message processors (ai_chat, ai_search, commands, inline).
app/repos/	Database repository pattern implementations (queries for chats, memory, keys).
app/providers/	AI provider abstraction layer (base, Gemini, Opencode, OpenRouter, FreeTheAI, Pollinations, Imagen, TTS, router).
app/response_delivery/	Typed provider-event coordination and the single owner of Telegram response rendering/finalization.
app/core/	Agentic research engine — multi-step query decomposition and tool use.
app/context/	Context assembly subsystem (assembler, summarizer, compression, token budget).
app/documents/	Document processing: chunking strategies, parsers, document repository.
app/middleware/	Request pipeline middleware (debounce aggregation, dedup prevention).
app/adapters/	Distributed/local concurrency primitives.
app/observability/	Bounded log pipeline, correlation, redaction, operational events and incident export.
ops/observability/	Independent private Grafana/Loki/Alloy/Docker-proxy deployment and runbook.
app/db/	Database bootstrap: schema validation, migrations runner, RLS, seed.
app/utils/	Shared utilities (formatting, keyboards, background tasks, image utils, reader SSR, etc.).
app/templates/	HTML Jinja2 templates for admin dashboard and Telegram Mini App.
app/games/	Crocodile, daily 2048/trivia, shared AI authoring and Telegram delivery.
app/bot_commands.py	Canonical public command catalog used by /help and Telegram's slash-command menu.
app/bot_instance.py	PTB Bot singleton — allows non-PTB code (WebSocket handlers) to call Bot API methods.
app/web_miniapp.py	Quart Blueprint for Telegram Mini App (initData auth, memory/settings/game API).
app/deferred_response.py	Redis-backed deferred AI generation worker for background retry after total outage.
app/intent_router.py	Lightweight LLM-bypass for weather/currency/crypto queries via WeatherAPI.com, ExchangeRate-API & CoinGecko.
app/state.py	Process-local user state, local locks and debounced PostgreSQL persistence.
docs/	Extended architectural documentation.
scripts/migrations/	Numbered SQL migration files — single source of truth for all DDL.
tests/	Comprehensive test suite (Unit and Integration).
bot.py	Main application entry point uniting Quart and the Telegram updater.

Tech Stack

Layer	Technology	Purpose
Runtime	Python 3.14-slim	Execution environment
Bot Framework	python-telegram-bot	Async interaction with Telegram APIs
Web Server	Quart + Hypercorn	Lightweight dashboard & Prometheus /metrics
Database	asyncpg	High-performance Async PostgreSQL driver
Vector DB	pgvector	Storing and querying semantic memories
Data Validation	pydantic	Configuration and strictly-typed object models
Observability	structlog	Structured JSON logging & context correlation

Setup

1. Clone the repository.
2. Use Python >=3.14,<3.15; the Docker image uses the slim variant. Supply PostgreSQL with pgvector available. CI uses PostgreSQL 17 with pgvector.
3. Install the pinned package manager and synchronize the committed dependency graph:

   python -m pip install "uv==0.13.5"
   uv sync --locked

4. Create an untracked UTF-8 .env using the configuration reference below; this checkout does not ship .env.example. The application reads process environment; local uv run --env-file .env or Docker --env-file loads the file explicitly.
5. Create PostgreSQL database with pgvector extension. Pending numbered SQL migrations from scripts/migrations/ are applied automatically on startup; migration files are kept idempotent so fresh deploys and repeated bootstrap runs remain safe.

Configuration

Configuration is explicitly read from environment variables in app/config.py and infrastructure modules. The following is an operator reference; load_settings() defines effective defaults. Deployment Secrets and persisted admin overrides can change them. The source inventory tracks literal readers and explicit Docker forwarding; python scripts/check_env_registry.py checks for drift without reading live values.

Important

Variables marked ✅ are required — the application will refuse to start if they are absent. Variables marked ⚙️ are optional and will use the listed defaults.

---

📔 Core / Authentication

Variable	Required	Format / Example	Default	Notes
TELEGRAM_BOT_TOKEN	✅	123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11	—	Obtained from @BotFather. Must match the running bot.
ADMIN_ID	⚙️	123456789	0	Set a real administrator ID for admin features, seeding and natal smoke. The loader permits omission; this does not configure a usable administrator.
ADMIN_SECRET	⚙️	Any secure random string, e.g. openssl rand -hex 24	—	Password for the admin web dashboard login form. Also used as encryption seed for stored API keys in the DB. Keep stable across restarts — changing it breaks decryption of stored keys.

---

🗄️ Database

Variable	Required	Format / Example	Default	Notes
DATABASE_URL	✅	postgresql://user:pass@localhost:5432/gemaibotv2	—	Standard asyncpg/libpq DSN. Must have the pgvector extension available — the bot won't start without it. Use a DSN reachable from the bot container; localhost inside a container is that container, not the VPS host.
DB_POOL_MIN_SIZE	⚙️	2—10	2	Minimum open asyncpg connections. Increase on high-traffic deployments to avoid connection storms.
DB_POOL_MAX_SIZE	⚙️	10—50	10	Maximum open asyncpg connections. Keep below the PostgreSQL max_connections limit (default 100). Tune from observed connection pressure and the database connection budget.
TEST_DATABASE_URL	⚙️	Same DSN format, pointing to a test DB	—	Used only during integration test execution (pytest -m integration). Completely isolated from production data.

RLS deployment boundary: the current startup path uses DATABASE_URL for both schema migrations and runtime queries. A PostgreSQL superuser, BYPASSRLS role, or table owner can bypass ordinary ENABLE ROW LEVEL SECURITY policies, so the built-in RLS layer is defense-in-depth rather than a hard tenant boundary when such a DSN is used. Application queries still set a transaction-local tenant context and scope by user_id. For hard database isolation, first split the current single-DSN bootstrap/runtime path: run migrations and RLS provisioning through a separate owner/migrator, then run the bot through a NOSUPERUSER NOBYPASSRLS non-owner role with only required DML, sequence, and function privileges. Validate cross-tenant denial against that role before enabling FORCE ROW LEVEL SECURITY; the current code does not yet provide that two-role bootstrap automatically.

Supabase Data API: the bot accesses PostgreSQL directly. The game, natal report and tarot tables protected by migration 079, and the roles catalog, do not grant access to the anon or authenticated API roles. A future Data API client requires deliberately reviewed grants and RLS policies; the bot's transaction-local settings are not Supabase Auth IDs. Migration 081 enables RLS on every expected application table before runtime policy setup.

Migrating from Supabase to a local DB:

# 1. Dump from Supabase
pg_dump "postgresql://postgres.xxx:PASS@aws-1-eu-north-1.pooler.supabase.com:5432/postgres" \
  --no-owner --no-acl -Fc -f backup.dump

# 2. Create local DB with pgvector
createdb -U postgres gemaibotv2
psql -U postgres gemaibotv2 -c "CREATE EXTENSION IF NOT EXISTS vector;"

# 3. Restore data
pg_restore -h localhost -U postgres -d gemaibotv2 --no-owner --no-acl -Fc backup.dump

After restore, update DATABASE_URL to postgresql://postgres:pass@localhost:5432/gemaibotv2 and redeploy. Validate restore completeness, extensions, grants and application behavior before switching traffic; keep the backup for recovery.

---

📦 Cache & Queue (Redis)

The VPS deployment uses the isolated Redis stack documented in Redis operations. Its connection comes from the root-owned /opt/gemaibot-redis/bot.env, not the legacy GitHub REDIS_URL secret. Redis data, credentials and local bounded backups survive bot deployments. Local/development runs still read REDIS_URL normally.

Variable	Required	Format / Example	Default	Notes
REDIS_URL	⚙️	redis://localhost:6379/0 or rediss://user:pass@host:6379	—	Used for: distributed LLM semaphores (multi-replica safety), Telegram Mini App Reader page cache (24h TTL), Gemini image per-key counters. Some locks/counters fall back locally if absent; Redis-backed Reader/deferred features and cross-process coordination need separate validation. Use rediss:// scheme for TLS connections.

---

🌐 Network & Web Server

Variable	Required	Format / Example	Default	Notes
PORT	⚙️	10000	10000	Port the Quart web server and admin dashboard binds to. On VPS, expose via Caddy/Nginx reverse proxy — do not bind directly to 0.0.0.0 in production.
ENABLE_WEB_SERVER	⚙️	true / false	true	Disabling this skips starting the Quart server entirely. Set false only for local dev without the dashboard.
WEBHOOK_URL	⚙️	https://bot.example.com	—	If set, the bot registers itself as a Telegram Webhook at this URL and stops long-polling. Must be HTTPS. Required for production webhook deployments. If absent, the bot uses long-polling (simpler for single-server setups).
WEBHOOK_SECRET_TOKEN	⚙️	1–256 ASCII letters, digits, _ or -	""	Optional Telegram secret header. Invalid configured tokens fail validation; when enabled, missing/malformed/mismatched inbound headers are rejected with constant-time comparison for valid strings.
WEBHOOK_MAX_CONNECTIONS	⚙️	Integer	40	Telegram webhook registration clamps the value to 1–100.
UPDATE_QUEUE_MAXSIZE	⚙️	Integer	1000	Bound on all admitted unfinished PTB updates, including fetched updates waiting on processing; released at task_done(). Construction enforces a minimum of 100.
WEBAPP_BASE_URL	⚙️	https://bot.example.com	""	Public URL from which the Telegram Mini App settings panel and reader are served. Must equal WEBHOOK_URL in most deployments. If empty, oversized responses use the optional Telegraph fallback or safe Telegram splitting.
TELEGRAPH_PUBLICATION_ENABLED	⚙️	true / false	false	Explicit privacy opt-in for publishing long responses and natal mirrors as public telegra.ph pages. Keep disabled unless users are informed that anyone with the link can read the page.
MINIAPP_SHORT_NAME	No	gemaibotv2	""	Short name for the Telegram Mini App deep links (t.me/<bot>/<short_name>). Configure the matching Mini App short name in BotFather; do not confuse it with a menu button URL.

---

Natal Reports

Variable	Required	Format / Example	Default	Notes
NATAL_REPORTS_ENABLED	⚙️	true / false	false	Enables natal input and hosted report generation. Enable explicitly for a controlled candidate rollout; the deploy readiness and maintenance gates must pass before public release.
NATAL_REPORT_TTL_DAYS	⚙️	365	365	Retention window for hosted natal reports. PostgreSQL storage keeps shared links stable by default.
NATAL_GEOCODER_PROVIDER	⚙️	local / nominatim	local	Birth-place resolution uses the local GeoNames-backed city catalog. Set nominatim explicitly to allow network fallback for unresolved free text.
NATAL_CITY_OVERRIDES_PATH	⚙️	/srv/bot/natal-city-overrides.json	""	Optional UTF-8 JSON file for admin-reviewed city records missing from GeoNames. Records are loaded into the local autocomplete/geocoding path; see docs/natal-city-overrides.example.json.
NATAL_SEND_RAW_BIRTH_DATA_TO_LLM	⚙️	false	false	Privacy guard. Keep false: LLM prompts should receive derived chart data, not raw birth date/place.

Natal city autocomplete uses GeoNames data via the geonamescache Python package. GeoNames data is licensed under CC BY 4.0 and should be credited in public product materials.

When NATAL_REPORTS_ENABLED=true, the VPS deploy workflow runs configuration, local city catalog, external offline fixture accuracy, storage and report smoke checks before maintenance and removal of the preserved previous container. The same readiness command, from the candidate image with its environment, is:

python scripts/natal_readiness.py \
  --check-config --check-storage --require-external \
  --reference-fixtures docs/natal-reference-fixture.moira-jpl.json \
  --smoke --webhook-url "$WEBHOOK_URL" \
  --min-city-count 30000 --max-city-warmup-ms 3000 --max-city-search-ms 300

Smoke uses an existing registered ADMIN_ID (or explicit --user-id), invokes report generation and writes a synthetic report; run it only in an authorized rollout window. It checks retrieval and hosted HTML rendering from storage. A manual Telegram/mobile/desktop check must still verify the public HTTPS link. Live NASA/JPL comparisons use the separate --check-horizons option and are not a dependency of every deploy. See the natal release contract for privacy settings, failure recovery and checks the smoke does not cover.

---

🖥️ Local Bot API Server (Optional)

When configured, the bot communicates with a self-hosted Local Bot API Server instead of api.telegram.org. This supports shared-volume media access and larger-file workflows; it does not remove all network or processing latency.

Variable	Required	Format / Example	Default	Notes
TELEGRAM_API_ID	⚠️	12345678	—	From my.telegram.org. Required only when running the Local Bot API Server container.
TELEGRAM_API_HASH	⚠️	0123456789abcdef...	—	From my.telegram.org. Required only when running the Local Bot API Server container.
TELEGRAM_LOCAL_SERVER_URL	⚙️	http://tg-api:8081/bot	""	URL of the Local Bot API Server. When set, enables local_mode=True in PTB. When empty (default), the bot uses the official Telegram cloud API.

---

🤖 Gemini Models (Primary LLM Provider)

Variable	Required	Format / Example	Default	Notes
GEMINI_API_KEYS	✅	key1,key2,key3	—	Comma-separated Google AI Studio API keys. Fresh selection spreads recent traffic across eligible keys before comparing daily usage. Each key has a local daily budget tracked in the DB; Google's actual RPM/TPM/RPD limits are shared per project. Minimum 1 key required. Live Audio no longer consumes this pool.
GEMINI_AVAILABLE_MODELS	⚙️	gemini-3.7-flash,gemini-3.6-flash,gemini-3.5-flash-lite	gemini-3.6-flash,gemini-3.5-flash-lite	Exact, ordered Gemini list shown in /model. Future syntactically valid gemini-* IDs are accepted without a code release. Role models such as DEFAULT_MODEL are not added to this selector automatically. Use the single token none for an intentionally empty list.
DEFAULT_MODEL	⚙️	gemini-3.6-flash	gemini-3.6-flash	Internal model used for standard conversational messages. It may be intentionally hidden from the user-selectable list.
QNA_MODEL	⚙️	gemini-3.5-flash-lite	gemini-3.5-flash-lite	Internal model used for quick Q&A web search queries (? prefix).
INLINE_MODEL	⚙️	gemini-3.5-flash-lite	gemini-3.5-flash-lite	Gemini inline role, loaded separately from the selectable chat catalog.
RESEARCH_MODEL	⚙️	gemini-3.6-flash	gemini-3.6-flash	Internal model used for synthesizing Tavily search results into a final research answer.
URL_SELECTION_MODEL	⚙️	gemini-3.5-flash-lite	gemini-3.5-flash-lite	Lightweight internal model that scores and filters candidate URLs during agentic web research before full content extraction.
TAXONOMY_MODEL	⚙️	gemini-3.5-flash-lite	gemini-3.5-flash-lite	Internal model used by MemPalace to classify memories into the Wing/Room taxonomy and to judge temporal contradictions (LLM-as-Judge).

---

---

Vertex AI (Primary Inline Driver + Optional Live Internet Route)

When VERTEX_AI_KEY is set, Vertex AI Express is used by specialized inline/game paths and eligible chat races; the key carries its own project metadata. Without that key, ordinary chat does not use the service-account credentials mounted for Vertex Live. Explicit Crocodile text-model selection follows the shared Gemini setting described in the operations notes. Live Audio remains separate by default: the default Mini App voice path runs through the Gemini GenAI Live API (gemini-3.8-live), with Rasalgethi first in its 30-voice catalog. Continuous conversation uses automatic speech detection with a 1.5-second silence window and does not automatically interrupt AI replies. A manual turn-ending mode remains available in Live settings on each device. See the Live conversation guide. The experimental Vertex Live route on gemini-live-2.5-flash-native-audio with Google Search grounding is available only when VERTEX_LIVE_ENABLED=true; otherwise it is hidden from settings and its WebSocket rejects connections.

For that experimental live route, VERTEX_AI_PROJECT and VERTEX_AI_LOCATION are not enough by themselves. The bot container must also receive readable ADC credentials, typically by mounting a service-account JSON and exporting GOOGLE_APPLICATION_CREDENTIALS to that in-container path. The included GitHub Actions deploy flow supports this via the VERTEX_LIVE_SERVICE_ACCOUNT_JSON secret and mounts it read-only into the bot container.

A readable service-account file can still refer to an account that no longer exists. If token refresh reports invalid_grant / account not found, replace VERTEX_LIVE_SERVICE_ACCOUNT_JSON with a valid account before using the opt-in Vertex Live route. Ordinary chat uses only VERTEX_AI_KEY for Vertex Express and does not consume this ADC file.

Variable	Required	Format / Example	Default	Notes
VERTEX_AI_KEY	No	GCP API key string	""	Used by configured Vertex Express paths; selection order depends on the caller. This API key is distinct from ADC service-account credentials for the optional Live route.
VERTEX_AI_PROJECT	No	my-gcp-project-123	""	GCP project ID for the regional Vertex Live route; not required by the Express client.
VERTEX_AI_LOCATION	No	us-central1	us-central1	Vertex AI region. Must match where your models are available.
VERTEX_LIVE_ENABLED	No	true or false	false	Enables the experimental Vertex Live route only after its service-account credentials work. Forwarded from the GitHub Actions repository variable.

Opencode Go Models

Variable	Required	Format / Example	Default	Notes
OPENCODE_API_KEYS	⚙️	sk-abc,sk-xyz	[]	Comma-separated Opencode Go API keys. When set and PRIMARY_PROVIDER=opencode, routes standard chat/search/inline through opencode.ai/zen/go/v1 using Bearer auth. Key rotation works identically to Gemini keys.
OPENCODE_AVAILABLE_MODELS	⚙️	opencode-go/minimax-m2.7,opencode-go/qwen3.5-plus	Role-model defaults	Exact, ordered Opencode list shown in /model. Use none for an intentionally empty list.
PRIMARY_PROVIDER	⚙️	opencode / gemini / openrouter / freetheai	opencode	Legacy default provider selection; persisted /set_provider overrides take precedence. The live-answer profile in /controls pins all covered processes, including existing users, independently of their saved model. Use that profile for a complete OpenAI/Gemini switch.
OPENAI_API_KEYS	⚙️	synthetic-key-one,synthetic-key-two	[]	Direct OpenAI credentials, separate from OpenRouter; comma-separated pool with rotation. OPENAI_API_KEY is the local single-key fallback when the pool is empty. Deployment forwards OPENAI_API_KEYS.
OPENAI_AVAILABLE_MODELS	⚙️	openai-direct/gpt-6-luna,openai-direct/gpt-6.1-terra	Luna + Terra	Exact ordered direct-OpenAI catalog. none keeps an intentionally empty selector catalog without disabling internal process routes. openai/… retains its existing OpenRouter meaning.
OPENAI_DEFAULT_MODEL	⚙️	openai-direct/gpt-6.1-terra	openai-direct/gpt-6.1-terra	Initial OpenAI profile for document/image and other live answers. Ordinary chat/search use Luna Flex, complex chat uses Terra Flex, and Crocodile judgement uses Luna Standard. Role overrides also accept bare IDs; saved dashboard plans take precedence.
OPENAI_QNA_MODEL	⚙️	openai-direct/gpt-6.1-terra	openai-direct/gpt-6.1-terra	Legacy OpenAI baseline for search. The active OpenAI profile starts with Luna Flex and native web search; saved dashboard plans take precedence. Gateways and legacy models keep Jina retrieval.
OPENAI_INLINE_MODEL	⚙️	openai-direct/gpt-6-luna	openai-direct/gpt-6-luna	Initial fast OpenAI profile for inline and short live classifiers. Crocodile answer checking has its own Luna Standard profile default. Daily preparation keeps its existing Gemini routes.
OPENCODE_DEFAULT_MODEL	⚙️	opencode-go/qwen3.5-plus	opencode-go/qwen3.5-plus	Default Opencode chat role.
OPENCODE_QNA_MODEL	⚙️	opencode-go/qwen3.6-plus	opencode-go/qwen3.6-plus	Model for quick Q&A search synthesis (? prefix).
OPENCODE_RESEARCH_MODEL	⚙️	opencode-go/glm-5.1	opencode-go/glm-5.1	Configured research-role model included in Opencode selector defaults. It does not switch the specialized ?? Gemini agent loop to Opencode.
OPENCODE_URL_SELECTION_MODEL	⚙️	opencode-go/big-pickle	opencode-go/big-pickle	Internal URL-selection role; not automatically added to the selectable model defaults.
OPENCODE_VISION_MODEL	⚙️	opencode-go/mimo-v2-omni	opencode-go/mimo-v2-omni	Vision-capable model — supports image_url natively. Exempt from Gemini vision redirect.
OPENCODE_INLINE_MODEL	⚙️	opencode-go/minimax-m2.5	opencode-go/minimax-m2.5	Lighter model for inline mode generation.
JINA_API_KEY	⚙️	jina_xxx...	""	API key for JINA AI Search. Used as grounding backend for ? quick search when Opencode is active. Also used by agentic research engine for page reading via r.jina.ai. Without this key, Opencode search queries run without web context.

Model IDs are configurable; this table is not a provider-side availability catalog.

---

OpenRouter Models

Variable	Required	Format / Example	Default	Notes
OPENROUTER_API_KEYS	⚙️	sk-or-v1-abc,sk-or-v1-xyz	[]	Comma-separated OpenRouter API keys. Required for OpenRouter generation. Rotated same as Gemini keys. Note: OpenRouter is disabled for multimodal (image) requests — Gemini is always used for vision.
OPENROUTER_AVAILABLE_MODELS	⚙️	stepfun/step-3.5-flash:free,qwen/qwen3-4b:free	OPENROUTER_DEFAULT_MODEL	Exact, ordered OpenRouter list shown in /model when at least one OpenRouter key is configured. Use none for an intentionally empty list.
OPENROUTER_DEFAULT_MODEL	⚙️	stepfun/step-3.5-flash:free	stepfun/step-3.5-flash:free	Default model for standard chat on OpenRouter.
OPENROUTER_QNA_MODEL	⚙️	stepfun/step-3.5-flash:free	stepfun/step-3.5-flash:free	OpenRouter model for quick Q&A search synthesis.
OPENROUTER_RESEARCH_MODEL	⚙️	stepfun/step-3.5-flash:free	stepfun/step-3.5-flash:free	Research-role mapping in provider-routed use cases (app/agent_use_cases.py); not a replacement for the specialized Gemini agent loop.
OPENROUTER_URL_SELECTION_MODEL	⚙️	stepfun/step-3.5-flash:free	stepfun/step-3.5-flash:free	OpenRouter model for URL scoring during agentic research.

---

🚀 FreeTheAI (Multimodal Router)

Variable	Required	Format / Example	Default	Notes
FREETHEAI_API_KEYS	⚙️	sk-freetheai-1,sk-freetheai-2	[]	Comma-separated API keys for FreeTheAI. Provides advanced multimodal image generation and Lyria-based audio music models.
FREETHEAI_AVAILABLE_MODELS	⚙️	cat/claude-4-6-sonnet,yng/deepseek-r1	FREETHEAI_DEFAULT_MODEL	Exact, ordered FreeTheAI chat-model list shown in /model. Image/audio IDs are not selectable here. Use none for an intentionally empty list.
FREETHEAI_DEFAULT_MODEL	⚙️	cat/claude-4-6-sonnet	cat/claude-4-6-sonnet	Default chat model via FreeTheAI.

For every *_AVAILABLE_MODELS variable, an unset or whitespace-only value uses the provider defaults; a space is therefore not a way to clear the list. Set the value to the single case-insensitive token none to make that provider's user-selectable list empty. /models changes create an explicit database override, which takes precedence over env/Secrets across restarts. The provider view shows the active source. Reset to .env deletes that override and immediately reloads the current env/Secret value. Legacy automatically seeded DB lists are discarded so they cannot mask a newly deployed Secret.

---

📔 Search & Web Research

Note: JINA_API_KEY is documented in the Opencode Go Models section above — it serves dual purpose as both Opencode search grounding and agentic page reader.

Variable	Required	Format / Example	Default	Notes
TAVILY_API_KEYS	✅	tvly-key1,tvly-key2	—	Comma-separated Tavily Search API keys. Used by search/research services; quick-search grounding depends on the active provider. Credit usage is tracked by the application.
WEATHER_API_KEY	No	abc123...	""	API key for WeatherAPI.com. Powers intent-direct weather query handler — weather queries intercepted before reaching the LLM. Without this key they fall through to the LLM.
EXCHANGE_RATE_API_KEY	No	abc123...	""	API key for ExchangeRate-API.com. Powers intent-direct currency conversion queries. Without this key currency queries fall through to the LLM.

---

🧠 Agentic Research Engine

Variable	Required	Format / Example	Default	Notes
AGENTIC_MODEL	⚙️	Gemini model ID	"" (falls back to RESEARCH_MODEL)	Used when neither a handler model override nor the current chat model is set. It does not override an explicit chat selection.
AGENTIC_MAX_ITERATIONS	⚙️	5—15	5	Maximum research loop cycles before the agent is forced to synthesize an answer. Each iteration = one round of query → search → read → reflect. Higher = deeper research, higher API cost.
AGENTIC_MAX_PAGES	⚙️	3—10	3	Shared admission limit on read_page across the research fallback-chain. Slots are reserved before a batch executes.
AGENTIC_MAX_TOKENS	⚙️	100000—500000	100000	Known usage plus reservations for missing usage gate the next iteration/fallback. An admitted call and final synthesis may overshoot; not a billing ceiling.
AGENTIC_TIMEOUT_SECONDS	⚙️	90—300	90	Shared cooperative async deadline for research attempts/tools/fallback, with min(15 seconds, timeout / 6) reserved for synthesis. Cleanup may add time; initial key lookup and final Telegram delivery are outside the interval.
AGENTIC_PAGE_CONTENT_LIMIT	⚙️	4096—16384	8192	Maximum characters extracted from each web page before truncation. Higher = more context per page, more LLM tokens consumed.
ADAPTIVE_THINKING_ENABLED	⚙️	true / false	true	Enables the Gemini chat thinking resolver: explicit user level/off, then model-specific defaults, then the heuristic classifier. OpenAI uses administrator reasoning or the model default; automatic complex-chat routing is separate.

The agentic loop in app/core/agentic.py uses the Gemini SDK directly. Its handler chooses model_override → chat_state.model → AGENTIC_MODEL → RESEARCH_MODEL; selection alone does not guarantee that a model has a usable Gemini key/capability. The handler shares one budget across at most three agent runs. Missing usage reserves 4,000 tokens per call for admission and is reported as uncertain, never as confirmed spend. A synthesized answer after a cutoff carries a visible limit notice. Ordinary provider-routed chat/search and specialized research execution have distinct paths.

---

📊 Rate Limits & Concurrency

Variable	Required	Format / Example	Default	Notes
DAILY_LIMITS	⚙️	JSON model-to-limit map or comma-separated model:limit entries	Model-specific map in app/config.py	Local Gemini RPD budgets per key and model, reserved atomically in PostgreSQL before dispatch, including TTS. Reset at midnight Pacific Time. Admin model-limit overrides take precedence; models without a configured cap still accumulate usage.
MAX_CONCURRENT_HEAVY_REQUESTS	⚙️	4—32	4	Global asyncio semaphore limiting simultaneously active LLM/TTS requests. Backed by Redis for multi-replica safety. Tune against measured load, provider quota and memory limits.
MAX_CONCURRENT_ULTRA_HEAVY_REQUESTS	⚙️	1—8	1	Separate semaphore for agentic research (??) sessions. These are memory-intensive due to iterative context accumulation. Tune against measured resource use.
LRU_STATE_CACHE_SIZE	⚙️	1000—50000	1000	Maximum number of UserState objects held in the in-process LRU cache. Tune from measured state size and cache misses.
DB_POOL_MIN_SIZE	⚙️	2—10	2	See Database section.
DB_POOL_MAX_SIZE	⚙️	10—50	10	See Database section.

---

🎨 Image Generation

Variable	Required	Format / Example	Default	Notes
IMAGE_MODELS	⚙️	Comma-separated IDs/aliases	flux,zimage,gptimage-1-5,gptimage,qwen-image,wan-image,klein	Configured canvas/inline selections. Daily admin additionally loads the live image catalog; saved selections are retained if unavailable.
DEFAULT_IMAGE_MODEL	⚙️	zimage	zimage	Which Pollinations model is pre-selected by default in the Canvas keyboard.
POLLINATIONS_API_KEY	⚙️	sk_...	—	Required for all Pollinations generation. Keep server-side; manage keys and Pollen budget at enter.pollinations.ai. Runtime /keys overrides remain supported.
IMAGE_GEN_DAILY_LIMIT	⚙️	10—100	10	Per-user daily cap for Gemini native image generations via Google API. Counted separately from Pollinations.
IMAGE_GEN_RPD_PER_KEY	⚙️	25	25	Requests-per-day budget per Gemini API key for native image generation. This is a local budget, not a provider entitlement. Tracked in Redis with in-memory fallback. Prevents image quota from consuming keys needed for LLM chat.
IMAGE_GEN_TIMEOUT	⚙️	30.0—120.0	60.0	Max seconds to wait for a Gemini native image API response before timing out and rotating to the next key.
IMAGE_GEN_MAX_RETRIES	⚙️	1—5	3	Number of Gemini key rotation attempts on quota/error before failing image generation entirely.

---

🔊 Voice / TTS (ElevenLabs & Gemini)

Variable	Required	Format / Example	Default	Notes
ELEVENLABS_API_KEYS	⚙️	sk_abc,sk_xyz	[]	Comma-separated outbound TTS keys, tried in order; failed/exhausted keys are skipped for remaining chunks of that synthesis. If the pipeline cannot finish, the voice engine can fall back to Gemini TTS for the entire response. Provider-side quotas are not defined by this repository.
ELEVENLABS_VOICE_ID	⚙️	XB0fDUnXU5powFXDhCwa	XB0fDUnXU5powFXDhCwa	Baseline ElevenLabs voice; the admin catalog and an active user preference take precedence. Default is Charlotte. Browse voices at elevenlabs.io/voice-library.
ELEVENLABS_MODEL	⚙️	eleven_multilingual_v2	eleven_multilingual_v2	Model passed to the ElevenLabs synthesis path.
GEMINI_TTS_RPM_PER_KEY	⚙️	Positive integer	3	Local rolling 60-second TTS request budget per key and model. Redis coordinates replicas; without Redis the window is process-local. Set against the independent projects' AI Studio quotas. Forwarded by the VPS workflow; restart required.

The admin dashboard's Voices and TTS section (/controls#audio) manages voice IDs, names, ordering, defaults, Gemini voice prompts and synthesis parameters. The Mini App and voice engine consume the same catalogs. Model chains and provider order remain in Model routes, and shared pronunciation/style/pacing prompts remain in Prompts. The default Gemini TTS chain is gemini-3.8-flash-lite-tts → gemini-3.8-flash-tts; admin policies can override it. See voice controls and model inventory for parameters, precedence and the separate ASR and Live boundaries.

Explicit recitation such as Озвучь: ваш текст or Read aloud: your text goes straight to the speech queue without a text-model call or a text echo. Replying Озвучь это сообщение selects the replied-to text/caption. A forwarded message plus an authored recitation request in the same batch selects its raw text. Missing or competing sources produce a clarification. Requests to generate an answer and speak it retain the chat route. See the speech request guide for examples and limits.

Long direct recitation retains each original Telegram message: full sources get separate audio replies in source order. A small final continuation can join the preceding source when their combined input fits one request; the tail receives an explanatory reply. Collection waits visibly for possible continuations; repeated text in different message IDs remains part of the source. See the speech guide for the bounded collection window and ambiguous final parts.

Gemini TTS counts the transcript and speech metadata with the selected TTS model's countTokens endpoint. Fitting input stays whole; verified input overflow or reported output truncation triggers necessary pieces, synthesized sequentially into one file for that source. A matching scoped count receipt prevents counting a merged input again in the worker. The old 2621-byte/duration estimate is removed. Key rotation is sequential; duplicate key races are disabled. Explicitly incomplete or token-limited output is discarded and retried with smaller pieces within bounded limits. A failed source never delivers partial audio. ElevenLabs also sends a whole transcript when it fits the selected model, splitting only at known model character limits (10,000 for Multilingual v2, 40,000 for Flash v2.5). Separate requests can differ in prosody; those necessary long-form joins retain neighbouring text context in ElevenLabs. The queue's 64 KiB UTF-8 source limit is separate from Gemini's per-call token budget. Audio validation detects reported truncation and malformed data; it does not verify every spoken word.

RPM admission precedes atomic RPD admission. An unsent minute reservation is refunded; dispatched requests retain RPM/RPD usage on errors and cancellation. SDK retries are disabled for TTS so each reservation covers one HTTP attempt. Keys are chosen from fresh least-used ordering, with exhausted keys skipped across the whole pool. If all usable keys are minute-limited, the job waits under its deadline. RPD values come from configured model budgets, not automatic discovery of provider account limits; missing caps rely on provider quota errors and rotation.

These tables cover supported operator settings, not every environment read in the repository. A runtime setting is not automatically forwarded by the VPS workflow: for example, INLINE_MODEL, OPENCODE_URL_SELECTION_MODEL, ELEVENLABS_MODEL, WEBHOOK_MAX_CONNECTIONS and UPDATE_QUEUE_MAXSIZE are loaded by code but are not currently passed through that workflow. Adding a GitHub Secret alone does not activate them. Conversely, legacy USE_OPENROUTER and MAX_CONCURRENT_HEAVY_CALLBACKS are forwarded but not loaded by load_settings(); the latter currently uses the handler's fallback of 4. Provider selection uses PRIMARY_PROVIDER and its persisted override. See the audit.

---

Logging & Observability

The default is one correlated UTF-8 NDJSON stream with structured exceptions, request/trace/job IDs, release metadata and bounded non-blocking output. Every actually selected provider key is represented by its last four characters plus a fingerprint on both request and terminal/error events; full credentials are always scrubbed. Protected operational logs default to bounded scrubbed message text (LOG_CONTENT_MODE=full, up to 2048 characters) plus size/fingerprint metadata. Operators can explicitly switch to metadata, while preview remains a selector-scoped diagnostic mode. Message-bearing rows are restricted evidence, not CI/public/alert payloads; DEBUG never relaxes credential rules.

Use canonical LOG_FORMAT=json|text; STRUCTURED_LOGGING and LOG_PRETTY are compatibility aliases. Production deploys private Grafana/Loki/Alloy search with 100-row/15-minute browser defaults and a 500-row server ceiling; access is through an SSH tunnel. See the operator/agent runbook, log viewer runbook, event catalog, and dated audit.

🧪 Testing Only

Variable	Required	Format / Example	Default	Notes
TEST_DATABASE_URL	⚙️	postgresql://user:pass@localhost:5432/test_db	—	Postgres connection string used exclusively by pytest -m integration. Must point to a clean, separate database — integration tests run destructive DDL and DML. Never set this to the production DB.

Run

Local Python (dev, long-polling):

uv run --locked --env-file .env python bot.py

Docker Compose (legacy local, single container):

docker compose -f docker-compose.yml up -d

Production VPS (application containers plus private log-search stack via GitHub Actions CI/CD):

The canonical deployment is automated by .github/workflows/deploy.yml. It starts after a successful completed CI run for a push to this repository's vps_testai branch. Before checkout, and again before preparing SSH secrets, it verifies that workflow_run.head_sha is still the current branch tip. PR runs, foreign source repositories and reruns of superseded commits cannot authorize deployment. It builds that exact SHA-tagged image in GHCR and then SSH-deploys the application containers.

CI requires lint/format and documentation checks, Mypy, unit/E2E tests, isolated PostgreSQL/Redis migration and integration tests, Chromium regressions, and a production-image build/smoke. These jobs run independently; Python dependencies use a lockfile-keyed uv cache, and image builds share a BuildKit cache with deploy. The separate CI Advisory workflow runs on the same pushes and PRs. It retains the production dependency audit, SBOM/license artifacts and disposable private log-stack checks. Its failures remain visible and require investigation, but its completion and result do not block application deployment. Dependency inventories are generated before vulnerability auditing and uploaded even when that audit fails. Deploy-time migration, readiness, authentication and health gates remain mandatory.

The application deployment contains:

1. tg-api — Local Telegram Bot API Server (aiogram/telegram-bot-api:latest). Requires TELEGRAM_API_ID and TELEGRAM_API_HASH from my.telegram.org. Before starting or reusing the local API server, each deployment runs scripts/release_cloud_bot_api.py; the script safely releases any cloud Bot API session without relying on a stale host marker.
2. tg-bot — The Python bot container. Connected to tg-api via TELEGRAM_LOCAL_SERVER_URL=http://tg-api:8081/bot. Mounts the shared volume tg-api-data at /var/lib/telegram-bot-api for zero-copy file access.
3. tg-media-cleanup — Alpine cron sidecar. Runs chmod -R g+rX every 60s (fixes UID 101 permission conflicts) and find -mtime +7 -delete every 24h (prevents disk exhaustion from cached media).

The application containers share the tg-net Docker bridge network and TZ=Europe/Kyiv. The same successful deployment starts an independent, resource-bounded Grafana/Loki/Alloy/Docker-proxy project before replacing tg-bot; Grafana listens only on VPS 127.0.0.1:3000.

If VERTEX_LIVE_SERVICE_ACCOUNT_JSON is configured in GitHub Actions secrets, the deploy script also writes that service-account JSON to the VPS, mounts it into tg-bot as /run/secrets/vertex-live-sa.json, and exports GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/vertex-live-sa.json. The mounted file must remain readable by the non-root bot process; the workflow now prepares it with read permissions suitable for the container user instead of root-only mode.

For a manual deployment, adapt the checked-in workflow for the target host and verified image SHA. The workflow is the complete reference for credentials, migration ordering, health gates, media retention and eligible rollback; a short docker run example is not equivalent.

Schema Management

All database DDL is managed via numbered SQL migration files in scripts/migrations/.

Component	Role
scripts/migrations/000_init_schema.sql	Initial bootstrap only; later migrations are required for the current schema
scripts/migrations/001-019_*.sql	Incremental schema changes (ALTER, indexes, RLS, triggers, cleanup)
scripts/migrations/020_add_trgm_hybrid_search.sql	Enables pg_trgm extension + GIN index for hybrid keyword+semantic memory search
scripts/migrations/024_upgrade_gemini_v2_768.sql	Migrates embeddings to halfvec(768) (gemini-embedding-2-preview)
scripts/migrations/025_add_temporal_graph_edges.sql	Adds updated_at + unique constraint to memory_edges for temporal upserts
scripts/migrations/026_add_core_persona_edges.sql	Adds is_core BOOLEAN + partial index to memory_edges for Core Persona Protection
scripts/migrations/026b_add_predicate_embedding.sql	Adds predicate_embedding halfvec(768) + HNSW index for Semantic Edge Deduplication
scripts/migrations/027_add_edge_provenance.sql	Adds source_memory_ids BIGINT[] + GIN index to memory_edges for HippoRAG 2 Dual-Node provenance
scripts/migrations/032_add_wing_room_taxonomy.sql	Adds wing, room, hall_type to long_term_memory and wing, room to memory_nodes with B-tree + partial HNSW indexes
scripts/migrations/033_add_role_diaries.sql	Adds role_diaries JSONB DEFAULT '{}' to user_state for MemPalace persistent role diaries
scripts/migrations/034_global_settings.sql	Creates global_settings key-value table for runtime configuration; seeds inline_thinking_level default
scripts/migrations/018_add_missing_table_definitions.sql	Backfill migration for databases that applied 000 without all tables
scripts/migrate.py	Standalone runner with strict manifest validation; deployment applies pending versions before starting the candidate.
app/db/migrations.py	Internal migration framework. Stops before legacy DDL when any numbered migration fails.
app/db/schema.py	Fail-closed startup validation — verifies all expected tables and critical runtime columns after migrations
app/db/rls.py	Row Level Security policy management
app/db/seed.py	Initial data seeding (admin user, API keys, indexes)

Workflow:

• Deploy-time: scripts/migrate.py runs in an ephemeral container. If any migration fails, deployment aborts.
• CI: apply the complete migration chain to clean pgvector PostgreSQL, rerun it as a no-op, then require --check to report no pending migration versions. This is not a checksum or full-schema drift audit.
• Startup-time: init_db() -> run_migrations() -> validate_schema() -> setup_row_level_security() -> seed. Any numbered migration, schema, or RLS failure aborts startup.

Diagnostic caveat: even scripts/migrate.py --check and --status may create the migration-tracking table. They use DATABASE_URL, not TEST_DATABASE_URL; never run them against production as casual read-only checks.

Adding new tables: Create a new numbered .sql file in scripts/migrations/, add the table name to EXPECTED_TABLES in app/db/schema.py, and add RLS configuration to app/db/rls.py if needed.

Long-Term Memory Architecture

LTM is implemented in app/repos/memory*.py, not app/memory_manager.py. It combines pgvector halfvec(768), hybrid text/vector retrieval, graph traversal and tiered context. Current model names, thresholds and budgets are defined in app/repos/memory_config.py and the relevant retrieval/handler code, not fixed latency or recall-quality guarantees.

Private-memory work carries a durable consent epoch and renewable provider lease. Disabling LTM or erasing an account invalidates stale work. Group messages are not implicitly captured into private LTM. Retrieval treats remembered content as data, not trusted instructions.

Extraction and consolidation prepare provider results outside the write transaction, then share a caller-owned, tenant-bound graph writer. Durable source memories, nodes/edges, normalized memory_edge_sources and compatibility snapshots are committed consistently. Cleanup follows live provenance so unsupported edges do not remain usable. See ADR 0002.

Use /memory, /clearmemory, /settings, /mydata and the confirmed private-chat /deleteme flow for user controls. Do not substitute ad-hoc SQL deletion for the repository workflows or claim these controls establish legal compliance. Exports and third-party published copies are outside automatic local erasure.

Scripts

Command	Purpose
uv run --locked ruff check .	Run the repository-wide lint rules.
uv run --locked ruff format --check .	Enforce the same formatting gate used by CI.
uv run --locked mypy app bot.py	Type-check the production Python surface.
uv run --locked python scripts/check_encoding.py	Detect UTF-8 corruption in protected documentation.
uv export --locked --no-dev --output-file production-requirements.txt	Export the exact production graph for external audit tools.

Dependency maintenance

pyproject.toml is the only manually edited dependency manifest and uv.lock is the committed exact graph. Use the pinned tool version for every lock operation:

python -m pip install "uv==0.13.5"
uv lock --check
uv sync --locked
uv run --locked python scripts/dependency_frontier.py audit

The Dependency Frontier Audit workflow wakes weekly but a fixed UTC epoch gate permits scheduled resolution only every 14 days; a manual dispatch runs immediately. It resolves the policy-constrained and unconstrained stable frontiers twice for Linux production and Windows development with a seven-day release cooldown. Its report is discovery evidence, not proof of compatibility. Dependabot groups post-1.0 patch/minor updates, while major and current 0.x updates remain separate and are never auto-merged.

The manual Dependency Live Canary workflow accepts a same-repository dependency PR that changes only pyproject.toml and uv.lock. Before the protected job is entered, an unprivileged job rejects non-PyPI/direct sources, disables source builds, installs both locked graphs, and verifies exact environment parity. Both checkouts discard persisted Git credentials. The protected job repeats the wheel-only locked installation without secrets; dedicated credentials exist only in the probe step. The dependency-canary GitHub Environment must require a reviewer and provide dedicated, low-quota credentials:

Environment entry	Purpose
CANARY_TELEGRAM_BOT_TOKEN	Dedicated bot that can access only the canary chat.
CANARY_TELEGRAM_CHAT_ID	Dedicated chat where one silent tagged message is sent and deleted.
CANARY_GEMINI_API_KEY	Revocable, low-quota Gemini canary key.
CANARY_TAVILY_API_KEY	Revocable, low-quota Tavily canary key.
CANARY_GEMINI_MODEL	Optional environment variable; defaults to gemini-2.5-flash.

Baseline and candidate probes run in one approved window. Missing credentials, cleanup failure, baseline failure, transient provider ambiguity, cancellation, or any skipped comparison cannot publish a passing candidate status.

The dedicated canary is optional infrastructure, not a substitute for production verification. If no separate canary bot exists, do not treat that gate as passed or reuse the production bot token in it. For a deliberate production dependency trial, keep the dependency changes in one reviewed PR so CI and the running bot exercise the exact combined graph. The deploy workflow then:

1. serializes production transitions without cancelling an in-flight deploy;
2. proves that the successful CI commit belongs to exactly one merged same-repository PR;
3. allows automatic rollback only when both dependency manifests and an explicit set of non-runtime review files changed;
4. preserves the current tg-bot container and independently verifies that its image SHA equals the PR base SHA;
5. starts the candidate, checks /health, runs enabled natal configuration/city/external-fixture/storage/report-smoke gates and maintenance, and only then deletes the preserved container;
6. restores and health-checks the previous container after an eligible candidate failure while leaving the deployment failed for investigation.

This rollback protects startup and health-gated failures. It cannot detect a semantically wrong provider response after /health succeeds, and it deliberately refuses automatic rollback when code or migrations changed. After deploying a frontier trial, exercise the real bot during a controlled low-traffic window:

• ordinary text in both streaming and non-streaming paths;
• provider fallback plus agentic/web search;
• photo and document understanding;
• voice transcription and TTS;
• /draw image generation and /live audio when enabled;
• memory write/read, a Redis-backed callback or game action, and an admin/background test send;
• structured logs, provider error rates, duplicate replies, stuck streams, and dropped updates.

Any functional regression, schema/type error from a provider, unexplained error-rate increase, duplicate delivery, or stalled response rejects the candidate even when all automated checks passed. Keep the PR under observation for 24–48 hours; a post-health functional failure requires an explicit redeploy of the recorded base SHA because the workflow has no reliable signal for automatic application-level rollback.

API / Events / Contracts

Telegram Commands:

• Public command catalog (the same catalog powers Telegram's menu and /help):
  • Start and chat: /start, /help, /newchat.
  • Personalization: /model, /roles, /setprompt, /thinking, /settings.
  • Search and briefings: /res toggles web search; /subscribe and /unsubscribe manage the daily intelligence briefing checked by the hourly scheduler.
  • Documents, images, and voice: /documents, /draw, /live. Image-generation aliases /img, /image, and /generate remain accepted but are not duplicated in the public menu.
  • Conversation history: /save, /conversations, /switch, /rename, /delete, /export, /stats.
  • Memory and privacy: /memory, /clearmemory, /mydata, /deleteme. Private-data commands require a direct chat; irreversible deletion requires explicit confirmation.
  • Games and reminders: /games, /dailycroc, /daily2048, /trivia, /remind.
  • Tarot and astrology: /tarot, /natal, /horoscope_settings, /horoscope_stop.

Administrative and developer commands are deliberately excluded from app/bot_commands.py and Telegram's public slash-command menu.

• Admin Commands (Requires ADMIN_ID):
  • /admin — Central administration hub.
  • /listmodels, /listusers — List configured models and registered users.
  • /adduser, /deluser — Manual user management.
  • /metrics, /rolemetrics — Detailed system and role-based usage telemetry.
  • /cachestats, /queuestats, /docstats, /groupstats — Performance monitoring for different subsystems.
  • /clearcache, /clearoldmetrics, /clearolddocs — System maintenance and cleanup. /clearcache removes search results from local memory and the qna / search Redis namespaces; it preserves queues, locks and Reader content. Backend failure is reported instead of confirming a successful clear.
  • /updatetavilykeys, /checktavilykeys — Hot-swap and verify search API keys.
  • /checkgeminikeys — Async parallel health check of all Gemini API keys against the Google API.
  • /set_inline_thinking <level> — Set inline generation thinking_level at runtime (stored in global_settings DB table, no restart required). Valid: minimal, low, medium, high.
  • /set_provider <name> — Switch the primary LLM provider at runtime without restart. Valid: opencode, gemini, openrouter, freetheai. Stored in global_settings; individual generation paths still apply their capability and key checks.
  • /models — Runtime model management wizard for Gemini, Opencode, OpenRouter, and FreeTheAI. Add or remove individual selectable models, inspect whether env or an admin override is active, and reset by deleting the override and restoring the current env/Secret list. New Gemini additions are checked for generateContent support before persistence.
  • /registergroup — Authorize the bot for use in a specific Telegram group.
  • /reloadconfig — Trigger an immediate hot-reload of the environment configuration.

Web Dashboard (Quart HTTP Routes):

• GET /, GET /login, POST /login, GET /logout — UI interface (requires ADMIN_SECRET authentication and uses Cookie Sessions).
• GET /health — Unauthenticated process/database status: 200 when the database connection flag is healthy, 503 when it is not, 500 on an internal check error. Redis status is reported but does not determine the overall status. This does not probe Telegram or AI providers.
• GET /ready — The same bounded status probe, additionally requiring configured, reachable Redis. VPS deployment uses this endpoint so unavailable queue/quota coordination cannot pass candidate readiness.
• GET /metrics — Unauthenticated, rate-limited Prometheus telemetry text. Restrict exposure at the reverse proxy when required by the deployment.
• GET /api/dashboard — Aggregated batch endpoint (replaces 8 individual fetches with 1 RTT). Auth required.
• GET /api/overview, /api/keys, /api/errors, /api/cache, /api/queue, /api/database, /api/circuit-breakers, /api/memory — Individual JSON data endpoints for dashboard charts (requires auth cookie or X-Auth-Token header).
• GET /api/key-health — Per-key health diagnostics: status, failure count, suspension info. Auth required.
• GET /api/events — Authenticated Server-Sent Events stream (5s interval) for real-time CPU, memory, queue, and request metrics. Service-health probe results come from /api/dashboard.
• /controls#queues — Shared job status, bounded pagination, revision-checked cancellation/retry and separate admission/execution pause controls. Unsupported PTB, TaskManager and TTS actions remain read-only. See the queue lifecycle and cutover runbook.
• /admin_daily#jobs — Read-only eight-day readiness calendar and tracked Crocodile, Trivia and Tarot operations. Job admission uses an operation ID; lost responses are reconciled through status lookup without automatically submitting another generation. Model/prompt edits use revision checks and retain conflicting drafts.
• /admin_daily#broadcast — Recipient preview, tracked offer batches and delivery history. Confirmed delivery, known failure and unknown acknowledgment remain distinct; retry excludes sent and uncertain recipients. Offers and actual daily deliveries have separate counters and an explicit history start. See the broadcast runbook.

The dashboard separates completed user operations from recorded outbound API attempts. Select today, the last 7 or the last 30 calendar days in Europe/Kyiv; the panel shows exact boundaries and marks API periods preceding instrumentation as partial. Gemini model groups start collapsed, support model/masked-key search and sorting, and show per-key local RPD usage when expanded. These quotas retain Pacific Time day boundaries. See metrics and dashboard for counter ownership, historical limits, recovery and performance metric scopes. Stage percentiles use a bounded 15-minute process window and show sample counts; source cards distinguish fresh, stale, partial and unavailable data. Dashboard HTTP and SSE readers share source refreshes. The bot lifecycle comes from the supervisor; a live web connection alone does not mean Telegram processing is ready.

Main User Flows

• Start with /help for the current categorized RU/EN public catalog.
• In private chat, standalone натальная opens the natal menu, таро or расклад opens Tarot, and гороскоп opens the horoscope menu without a slash. Case, surrounding whitespace and final ?!. are accepted. These exact menu words also switch out of an active input form; ordinary phrases retain normal chat routing.
• Standard chat combines selected model, history and consent-gated memory.
• An idle Tarot session offers a choice to continue Tarot or send the text as an ordinary request, including the first message after state restoration. A successful model selection, /newchat, normal /start, or the new-chat/ new-topic buttons end the Tarot session and remove its reply keyboard.
• ? requests quick search; ?? requests bounded agentic research.
• Enable inline mode in BotFather and use the deployed bot's username. Deferred AI answers also require chosen-result feedback; immediate compatibility results work without it.
• Inline compatibility accepts @<bot_username> совместимость мужчина скорпион женщина водолей, совм скорпион водолей and compatibility man scorpio woman aquarius. The result has adjacent «Подробнее о паре» and Tarot buttons. The first opens private chat with one «Заполнить данные пары» button for the compatibility Mini App when an HTTPS WEBAPP_BASE_URL (or webhook base) is configured. The form collects two dates, optional exact/approximate/range times and local-catalog cities, then provides editable confirmation and one calculation button. Known time requires a city; each partner may independently choose unknown time and skip birthplace. With no HTTPS Mini App base, the existing private two-date chat flow remains. In private chat, /compatibility, /совместимость, совместимость and совм open a two-sign picker with the same instant brief reading and a detailed-reading entry. /compatibility is also listed in Telegram's command menu and /help. Dates are supported from 1900 through today. Local synastry reuses natal astronomy for inter-chart aspects, Ascendant/MC and equal-house overlays; unknown-time aspects and placements are shown only when stable across the sampled interval. Missing time for one partner preserves the other's houses. Detailed results from both the Mini App and the chat date flow pass these local facts to AI for a connected relationship interpretation. The model receives only derived positions, aspects, symbolic matrix data and accuracy limits, without raw dates, times or birthplaces. Calculation-owned limits are appended separately; an unavailable model produces a retry notice rather than a template reading. Process compatibility and prompt compatibility.interpretation use the existing runtime model and prompt controls; the default model follows Tarot chat (gemini-3.1-flash-lite). Brief sign compatibility remains entirely local. Signed and currently authorized Mini App identity determines private delivery; results are not publicly hosted. Birth input is not persisted or sent to an interpretation provider. The chat fallback clears raw dates on completion, cancellation, mode changes or after 30 minutes; contextual Tarot stores derived positions only in its existing bounded 30-minute context. Readings are symbolic. Standalone private date messages in DD.MM.YYYY or YYYY-MM-DD form, including invalid calendar dates and edited messages after a restart or eviction of the bounded tracking markers, are intercepted before ordinary AI processing and message logging. The reply directs the user to «Подробнее о паре» or /natal; active compatibility and natal forms keep their own input handlers. This guard covers date-only messages, with no general scrubbing promise for dates embedded in arbitrary prose. Contextual tarot loads the saved UserState before applying the pair context. A button expired after 30 minutes explains the expiry and preserves active natal input. The tarot handler applies its new session and input cleanup only after successful welcome delivery. A failed contextual callback keeps the previous session, input and compatibility context available for a retry. A new command still cancels compatibility date input independently, as described above.
• The natal Mini App offers natal, destiny matrix and combined reports. Matrix-only input asks for a date; natal and combined input retain the time-precision and birthplace steps, including the option to omit an unknown birth time.
• /horoscope_settings supports subscriptions and on-demand readings; /horoscope_stop stops scheduled delivery.
• The admin web dashboard requires configured authentication; /admin_daily#croc manages shared/per-process Crocodile text models, per-puzzle images, the hourly automatic image quota and background preparation of both difficulties for an arbitrary day. Readiness checks do not invoke providers; explicit preparation can invoke paid providers and bypasses the automatic image quota. See the Daily Croc runbook.

Troubleshooting

• Conflict Error on Startup: Usually signifies another bot instance is currently polling the Telegram API using the same Token. Requires closing duplicate instances if not using Webhooks.
• decryption_error traces: Usually caused by attempting to load the database on a new host without providing the exact prior ADMIN_SECRET (the code derives the Fernet key; the secret itself need not be base64).
• Search features hanging: Check TAVILY_API_KEYS exhaustively or verify the circuit_breaker state at /api/circuit-breakers.

Operational Documentation Caveats

• The Dockerfile health start period is 40 seconds. Legacy Compose overrides it to 120 seconds only when that Compose file is used; it does not control the VPS workflow.
• The legacy Compose file exposes its published port differently from the loopback-only VPS workflow. Review network exposure before using it outside local development.
• The VPS deployment state, configured credentials and public provider catalogs must be verified separately; the repository does not prove current health.

Architecture Decisions

Use the architecture reference for current boundaries, ADR 0001 for single-owner delivery, and ADR 0002 for transactional graph writes. ROADMAP.md is future work, not implemented behavior; dated specs and plans retain their original design context.

Testing

See CONTRIBUTING.md for exact locked commands, test isolation and CI gates. Unit/E2E checks exclude both the integration directory and integration marker; integration tests also exist outside that directory. Use an explicitly disposable PostgreSQL/Redis target for DB tests. Rollback fixtures do not make arbitrary migration tests safe for production.

pytest.ini enables async auto mode, xdist and per-test timeout by default. Historical test counts are not fresh validation evidence. For documentation-only changes, check encoding, links, source accuracy and git diff --check.

Contributing

See CONTRIBUTING.md for setup, validation, pull-request, and secret-handling guidance.

Minimum maintainer checks for behavior changes:

1. Create a descriptive PR.
2. Verify the relevant pytest checks pass.
3. Run python -m ruff check . and python -m ruff format --check ..
4. Run python -m mypy app bot.py for production-code changes.
5. Update user-facing docs when public behavior or deployment changes.

License

MIT, see LICENSE.