Skip to content

fix(converter): export only the current turn as workflow and agent input (SAO-19572) - #306

Open
etserend wants to merge 3 commits into
mainfrom
fix/sao-19572-orchestration-input-history
Open

etserend wants to merge 3 commits into
mainfrom
fix/sao-19572-orchestration-input-history

Conversation

@etserend

@etserend etserend commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem

When an app re-sends the whole conversation (as graph state or as a message list), every agent and workflow span
exported all earlier turns in gen_ai.input.messages. Each new trace's input repeated the previous inputs and outputs,
and O11y Cloud rendered them as one block that grows with the conversation. The output side already trims history; the
input side did not.

Change

_set_orchestration_content now trims the exported input to the messages from the last user message onward, when the
input is either a {"messages": [...]} state container or a bare list of role-tagged messages (the shape the LangChain
agent middleware logs). Tool-call and tool-response messages of the current turn are kept. The untrimmed input is still
used for the existing output history dedup.

Unchanged: partial update containers, single-turn inputs, inputs with no user message, and LLM (chat) span inputs
(see Notes).

Verification

Multi-turn runs (3 turns, the full history re-sent each turn), spans captured on a local collector. Message counts of
the root agent or workflow span input per trace, before → after:

Framework Root input Result
LangGraph via the callback handler (healthcare assistant example) 1 / 3 / 5 → 1 / 1 / 1 fixed
LangChain create_agent + SplunkAOMiddleware 1 / 5 / 9 → 1 / 1 / 1 fixed
OpenAI Agents SDK 1 / 1 / 1 → 1 / 1 / 1 unaffected (string inputs)
CrewAI not run the dependency mirror had no chromadb wheel for macOS arm64

Inner chatbot, tools_condition and tools spans of the LangGraph run now show only their own turn, and the lab0
session view shows each question once. Chat span inputs are identical before and after in every run.

  • tests/test_attribute_mapping.py: 75 passed, including 15 new cases (behaviours and guards, with most run for both
    workflow and agent spans). The cases for the first change failed before it.
  • ruff check, ruff format --check on the changed files, and mypy (invoke type-check) pass.
  • The full suite shows 24 failures and 5 errors in exporter diagnostics, middleware tracing, OpenAI/OpenAI Agents,
    serialization and two collection errors. The identical set fails on unmodified origin/main, so none come from
    this change.

Notes

  • Behaviour change: earlier turns are no longer in workflow and agent span inputs; they remain in the session and in
    LLM span inputs.
  • Chat spans keep the full prompt. An LLM span's gen_ai.input.messages records the exact messages the framework
    sent to the model (system prompt, earlier turns, tool calls and results), so it is not trimmed. In the
    verification run, the third turn's chat span input held 6 messages (system, then the whole conversation), and its
    follow-up call after the tool call held 8. Only agent and workflow spans carry app state rather than a prompt, so
    only they are trimmed. This is how the LLM span path already worked (set_llm_attributes writes span.input as
    is), and this PR does not change it.
  • Trade-off: a bare message list cannot be told apart from conversation state, so a workflow or agent input that is
    deliberately a message list (for example few-shot examples or a system prompt) is trimmed to the current turn too.
  • The LangChain middleware path has no working unit test in the repository: tests/test_langchain_middleware.py
    already errors at collection on origin/main. The live run above is the evidence for that path.
  • CHANGELOG.md updated under Unreleased.

etserend and others added 3 commits November 3, 2026 17:12
…put (SAO-19572)

When an app re-sends the whole conversation as graph state (for example a
LangGraph {"messages": [...]} state), every agent and workflow span exported
all earlier turns in gen_ai.input.messages. Each new trace's input therefore
repeated the previous inputs and outputs, and O11y Cloud rendered them as one
growing block.

Trim the exported input of full-history message containers to the messages
from the last user message onward, so tool-call and tool-response messages of
the current turn are kept. The untrimmed input is still used for the existing
output history dedup. Partial update containers, bare lists, inputs without a
user message, and LLM span inputs (the full prompt sent to the model) are
unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
…(SAO-19572)

The first change only trimmed inputs shaped as {"messages": [...]} graph
state. The LangChain agent middleware logs the agent input as a bare list of
messages, so create_agent apps still repeated every earlier turn in each new
trace's input (root input 1 / 5 / 9 messages over three turns).

Trim a bare list of role-tagged messages the same way: keep the messages from
the last user message onward. Partial update containers, single-turn lists,
lists with no user message, and LLM span inputs are unchanged. A workflow or
agent input that is deliberately a message list (for example few-shot
examples) is trimmed too, because it cannot be told apart from conversation
state.

Co-Authored-By: Claude <noreply@anthropic.com>
…re, OpenAI Agents, CrewAI (SAO-19572)

Add one end-to-end regression test per integration that re-sends the whole
conversation over three turns and asserts what the root span exports:

- LangGraph callback handler: root and node input stay at the current user
  question; the chat span keeps the full prompt (1, 3, 5 messages).
- LangChain agent middleware: same, for the bare message-list state it logs.
- OpenAI Agents: the workflow input is the latest user message (pins existing
  behaviour); the chat span keeps the full prompt.
- CrewAI: kickoff inputs that carry the conversation export only the current
  question; the chat span keeps every earlier question.

The LangGraph handler, middleware and CrewAI tests fail on origin/main without
the converter change; the OpenAI Agents test is a guard and passes on both.

Co-Authored-By: Claude <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant