Repository navigation
HTTPAppTarget: send each turn once and no transcript markers (fixes #124) - #125
Merged
Merged
Conversation
run_scenario passes the whole conversation as history, current turn
included, with SimpleAudit's markers beside the text. _build_body
forwarded it as-is, so the app received the current user turn twice
(appended again in messages mode, repeated in history otherwise) and
the markers: documents with their evaluation-only marks, file_uri paths
and decision answers.
History now goes out as plain {role, content} turns, without a trailing
copy of the current turn; direct callers that pass earlier turns only
are unchanged. A multi-turn scenario's documents are taken from the
history marker and sent through documents_field as text, like a
context-grounding run's, and fail the same way when no documents_field
is set.
Fixes #124
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to the history fix, from review: - Only the engine's markers (documents, file_uri, decision, tool_events) are removed from history; other fields stay, so direct callers sending OpenAI tool messages (tool_calls, tool_call_id, name) or turns without content get the body they built. - Documents go through parse_documents like the model and decision targets: a dict without a string text fails, empty documents keep their position, and an explicit documents=[] is not replaced by the history marker. The error no longer assumes a context-grounding run. - A scenario with file_uri now fails with an error; the app could never receive the file, so it was dropped silently. - Target.send documents the history contract, and history_documents() in targets/base.py replaces the copies in http.py and decision.py.
This was referenced Oct 24, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #124.
run_scenariopasses the whole conversation ashistory, current turn included. On the conversation's message dicts, the engine keeps markers besideroleandcontent:documentswith their marks;file_uri;decision.HTTPAppTarget._build_bodyforwarded this list unchanged. As a result:messagesmode it was appended again; in other modes it went out asmessageand again at the end ofhistory.examples/audit_openwebui_rag.pyusesmessagesmode, so it was affected.Changes
tool_calls,tool_call_id,name, …) stay, so direct callers that pass earlier turns get the body they built. Inmessagemode,historyis a new list of the earlier turns.documents_fieldas text, as in context-grounding runs.parse_documents, like the model and decision targets: a dict without a stringtextfails, and empty documents keep their position, matching the judge's numbering.documents=[]is respected.Target.sendnow documents it, andhistory_documents()intargets/base.pyreplaces the copies inhttp.pyanddecision.py.Behaviour changes:
documents_fieldnow ends in an ERROR that namesdocuments_field, the same rule context-grounding runs follow.file_urinow ends in an ERROR; an HTTP app can't receive the file, and it used to be dropped silently.Tests
Eight new tests in
tests/test_targets.py. Through real audits against a recording HTTP client, they cover:documents_fieldandfile_urierrors.Direct
_build_bodycalls cover:Live check against a real OpenAI-compatible chat endpoint (Anthropic's, with
claude-haiku-5-5), recording every request body:dev, the same run sent the current user turn twice in every request.documents_field, or with a file, stopped before any request was sent.Not covered here
CallableTargetalso passeshistoryto the wrapped function unchanged, markers included.