Skip to content

perf: reduce image memory overhead across lifecycle - #10117

Open
w31r4 wants to merge 10 commits into
masterfrom
perf/image-memory-lifecycle
Open

w31r4 wants to merge 10 commits into
masterfrom
perf/image-memory-lifecycle

Conversation

@w31r4

@w31r4 w31r4 commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Reuse one image-input size guard across provider images, quoted images, plugin and MCP tool images, file reads, and tool screenshots.
  • Restore the existing 1 MiB compression trigger and remove the separate 2 MiB reuse threshold.
  • Keep the configured encoded-output budget and dimension limits in the shared preparation path.
  • Persist newly created history images as content-addressed references and materialize only selected references for a provider request.
  • Preserve coordinate-sensitive CUA PNG/JPEG inputs byte-for-byte when they are correctly oriented; animated inputs still use the bounded montage path.

Why

An image could pass the pixel limit while still creating a large encoded payload. Some producers also decoded data before the shared preparation path, so one image could occupy several full-size representations at once. The change rejects known oversized inputs before decoding, avoids duplicate size-limit logic, and keeps prepared image bytes out of ordinary conversation history.

Memory and lifecycle

  • Local paths and file URIs are checked with stat before reading.
  • Inline data URIs, base64:// references, and legacy base64 references are sized from their encoded length without decoding them first.
  • Downloaded images are checked after being written to the temporary file and before Pillow opens them.
  • File-read and tool-image cache paths use the same guard before their raw base64 decode.
  • New history entries store durable image references; only images selected for the active context are materialized for the request.
  • Existing inline history remains readable and is not rewritten during normal request handling.

No new runtime dependency is added. Full streaming transport remains outside this change.

Validation

  • Focused image/media lifecycle tests: 362 passed.
  • Full Python suite in WSL: 3482 passed, 5 skipped, 2 failures.
  • uv run ruff format .
  • uv run ruff check .
  • git diff --check

The two full-suite failures are inherited from the rebased #9669 changes: the Azure OpenAI test fixture has no endpoint, and the upload-format fixture does not define content_length. Native Windows and macOS runs remain unverified.

SweetenedSuzuka and others added 4 commits August 14, 2026 00:55
…y layer (#9663)

AsyncOpenAI/AsyncAzureOpenAI inherit the SDK default max_retries=2, which
nests underneath AstrBot's tenacity-based retry_provider_request(). As a
result provider_settings.request_max_retries no longer reflects the true
attempt count (e.g. request_max_retries=1 still sends up to 3 HTTP
requests) and delays fallback-provider switchover. Pass max_retries=0 so
AstrBot's retry layer is the single source of truth for retries.
…retry tests

Address review feedback on #9669: merge the near-identical official/Azure
client construction tests into a parametrized test, and add a regression
test asserting _query() performs exactly request_max_retries attempts
through retry_provider_request() when the call keeps failing, guarding
against multiplicative retries being reintroduced.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @w31r4, your pull request is larger than the review limit of 150,000 diff characters

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 17, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
astrbot-docs 843f171 Commit Preview URL

Branch Preview URL
Sep 17 2026, 11:15 AM

RC-CHN and others added 6 commits September 17, 2026 17:49
为模型图片输入与聊天附件上传增加大小护栏 / Add size guards for model
image inputs and chat attachment uploads:

- _convert_image_bytes_sync 复用原始编码字节的分支增加 2MB 上限;更大的
  输入重编码,输出始终受像素尺寸与质量约束
- prepare_model_image 在读入前检查文件大小,超过 32MB 跳过并记录日志
- 聊天附件上传上限 100MB:路由先检查 Content-Length,服务层落盘后复核
  实际大小

- Reuse the original encoded bytes in _convert_image_bytes_sync only for
  stills up to 2MB; larger inputs are re-encoded so the output stays
  bounded by pixel size and quality.
- prepare_model_image checks the file size before reading and skips
  inputs above 32MB with a warning log.
- Chat attachment uploads are capped at 100MB: the route checks
  Content-Length first, and save_uploaded_file re-checks the on-disk
  size after saving.

Tests: tests/test_media_utils.py, tests/test_chat_route.py
@w31r4
w31r4 force-pushed the perf/image-memory-lifecycle branch from 533d64e to 843f171 Compare September 17, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants