Python: [Bug]: ChatMiddleware has no supported way to make a durable change to the message list, so compaction silently no-ops behind any middleware that replaces messages

[jjwong0915]

Description

Follow-up to #7744, fixed by #7912. This is not a request to revert that design — the goal of "reconcile summaries without persisting unrelated rewrites" is right. The gap is that there is now no way for a middleware to declare a rewrite as related, so in practice "don't persist unrelated rewrites" behaves as "don't persist any rewrite made by a middleware that constructs new Message objects."

The rule as it stands

_reconcile_compaction_summaries (_compaction.py:378) accepts an inserted summary only if every id in its SUMMARY_OF_MESSAGE_IDS appears in source_message_ids — the ids of the caller's list (_compaction.py:384, 403-412). When it does not accept, it does not merely skip: it reverts the exclusions (_compaction.py:413-421).

So a ChatMiddleware can durably change message objects (in-place mutation of additional_properties survives, since the objects are shared) but cannot durably change the list, and loses even the object-level changes if any other middleware in the pipeline replaced a message.

Why this bites in a supported configuration

Replacing messages in middleware is not an exotic thing to do — the framework's own client asks for it. agent_framework_openai/_chat_completion_client.py:1139-1146:

OpenAI Chat Completions API does not support rich content (images, audio) in tool results. Rich content items will be omitted. Use the Responses API client for rich tool results.

For deployments pinned to Chat Completions, the answer is a middleware that moves those items into a user message so the vision model can still see them. That produces a message with no counterpart in the caller's list. A compaction middleware ordered after it is then silently disabled: the summary is rejected, the exclusions are reverted, the conversation is unchanged — and because a ChatMiddleware runs once per model call, this repeats on every iteration of the tool loop, paying for a summarizer model call each time.

There is no exception and no log record. The only visible symptom is a context window that keeps growing while compaction appears to be running.

Expected

A ChatMiddleware has some supported way to make a durable change to the message list — opt-in, so the default stays conservative.

Actual

Durable list changes are possible only for a middleware that no other middleware downstream of it disturbs, and the failure to persist is silent.

Steps to reproduce

Run the script below; it needs no network and no model. Only the middleware order differs between the two runs.

Code Sample

"""Repro: a ChatMiddleware cannot make a durable change to the message list."""
import asyncio

from agent_framework import (
    EXCLUDED_KEY,
    GROUP_ANNOTATION_KEY,
    SUMMARIZED_BY_SUMMARY_ID_KEY,
    SUMMARY_OF_MESSAGE_IDS_KEY,
    BaseChatClient,
    ChatMiddleware,
    ChatMiddlewareLayer,
    ChatResponse,
    Message,
    annotate_message_groups,
    included_messages,
)
from agent_framework.observability import ChatTelemetryLayer


class EchoClient(ChatMiddlewareLayer, ChatTelemetryLayer, BaseChatClient):
    async def _inner_get_response(self, *, messages, stream, options, **kwargs):
        return ChatResponse(messages=[Message(role="assistant", contents=["ok"])])


class RewriteMiddleware(ChatMiddleware):
    """Stands in for any middleware that adapts messages for a provider's wire format,
    e.g. moving images out of a tool result for the Chat Completions API."""

    async def process(self, context, call_next):
        context.messages = [
            Message(role=m.role, contents=m.contents) if m.role == "tool" else m
            for m in context.messages
        ]
        await call_next()


class CompactMiddleware(ChatMiddleware):
    """Summarizes everything but the newest group, as CompactionStrategy documents."""

    async def process(self, context, call_next):
        messages = list(context.messages)
        annotate_message_groups(messages)
        summarized = messages[:-1]
        summary = Message(
            role="user",
            contents=["<summary of earlier conversation>"],
            message_id="summary_1",
            additional_properties={
                GROUP_ANNOTATION_KEY: {
                    SUMMARY_OF_MESSAGE_IDS_KEY: [m.message_id for m in summarized if m.message_id]
                }
            },
        )
        for m in summarized:
            m.additional_properties.setdefault(GROUP_ANNOTATION_KEY, {})[
                SUMMARIZED_BY_SUMMARY_ID_KEY
            ] = "summary_1"
            m.additional_properties[EXCLUDED_KEY] = True
        messages.insert(0, summary)
        context.messages = list(included_messages(messages))
        await call_next()


async def main(order: str) -> None:
    middleware = (
        [RewriteMiddleware(), CompactMiddleware()] if order == "rewrite-first"
        else [CompactMiddleware(), RewriteMiddleware()]
    )
    client = EchoClient(middleware=middleware)
    conversation = [
        Message(role="user", contents=["q1"], message_id="m1"),
        Message(role="tool", contents=["tool output"], message_id="m2"),
        Message(role="user", contents=["q2"], message_id="m3"),
    ]
    await client.get_response(conversation)
    excluded = [m.message_id for m in conversation if m.additional_properties.get(EXCLUDED_KEY)]
    has_summary = any(m.message_id == "summary_1" for m in conversation)
    print(f"  order={order:14} caller list len={len(conversation)}  "
          f"summary reconciled={has_summary}  still excluded={excluded}")


print("After one compacted call, what survived in the CALLER's list:")
asyncio.run(main("compact-first"))
asyncio.run(main("rewrite-first"))

Output:

After one compacted call, what survived in the CALLER's list:
  order=compact-first  caller list len=4  summary reconciled=True  still excluded=['m1', 'm2']
  order=rewrite-first  caller list len=3  summary reconciled=False  still excluded=[]

Error Messages / Stack Traces

None — and that is part of the report. A rejected reconciliation produces no exception and no log record.

Package Versions

agent-framework-core: 1.17.0, agent-framework-openai: 1.14.2, agent-framework-anthropic: 1.0.0b260827

Python Version

Python 3.14.6

Additional Context

Proposed fix: let a middleware declare provenance

The reconciliation already resolves a transitive support graph — source_dependencies (_compaction.py:423-432) expands a summary's dependencies through other accepted summaries, so a summary of a summary is handled. A replacement message is the same shape of edge, and the existing fixed-point loop would resolve it unchanged.

Concretely: let a middleware record, on a message it constructs, which source message ids it supersedes — e.g. a SUPERSEDES_MESSAGE_IDS_KEY in the group annotation, the mirror of SUMMARY_OF_MESSAGE_IDS_KEY. source_message_ids would then be seeded with ids reachable through those edges, and a summary covering a promoted message would be accepted because its provenance terminates in the caller's list.

This keeps #7912's guarantee intact: a middleware that says nothing still cannot persist a rewrite. Only a middleware that explicitly claims provenance participates.

Alternatives considered

• Expose the durable list on ChatContext (e.g. context.source_messages) so a middleware that means to persist can operate on it directly. Simpler, but gives up the ownership check entirely.
• Pass the caller's list as ChatContext.messages, as FunctionInvocationContext.tools already does — that context documents the live-list aliasing as the feature (_middleware.py:435-438, and _tools.py:3772-3773). This was effectively the ask in
  Python: [Bug]: #4991 regresses when any ChatMiddleware is attached — compaction's summary is lost while its exclusion flags persist #7744; Python: Fix compaction persistence, thresholds, and observability #7912 chose reconciliation instead, so I am not re-raising it.

Minimum useful outcome

If none of the above is wanted, two much smaller changes would still have saved us the investigation:

1. Document the ordering constraint on ChatMiddleware and CompactionStrategy: a middleware that compacts must run upstream of any middleware that replaces Message objects.
2. Log when a summary is rejected and exclusions are reverted. A single warning at _compaction.py:413-421 would turn a silent, repeating, billable no-op into something diagnosable.

Relationship to open work

#8099 / #8117 generalize _reconcile_compaction_summaries so the function-invocation loop uses it too. That is adjacent but distinct: it addresses summaries not reaching AgentResponse, whereas this is about a middleware's own rewrites being unable to participate in reconciliation at all. Worth confirming the two designs agree before #8117 lands.