feat: add Netra.redteam.run_redteam() to trigger an existing red-team config - #386
Open
jithin23-kv wants to merge 5 commits into
Open
feat: add Netra.redteam.run_redteam() to trigger an existing red-team config#386jithin23-kv wants to merge 5 commits into
jithin23-kv wants to merge 5 commits into
Conversation
jithin23-kv
force-pushed
the
feat/redteam-sdk-client
branch
from
August 28, 2026 04:32
1fab2e5 to
2cb78ff
Compare
| error: Optional[str] = None | ||
| try: | ||
| message, session_id = await execute_handler(handler, prompt_text, session_id, turn_index) | ||
| output = self._truncate_output(message) |
Collaborator
There was a problem hiding this comment.
Why are we truncating the message here?
Author
There was a problem hiding this comment.
Raised it to match the backend's actual 100k limit
| ) | ||
|
|
||
| try: | ||
| result = self._client.submit_turn( |
Collaborator
There was a problem hiding this comment.
Will the BE need the trace_id for this particular turn? We are passing it within simulation and evaluation modules.
Author
There was a problem hiding this comment.
Currently our backend does not support this. will need some changes in backend including db change. should we do it now?
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Netra.redteam.run_redteam(config_id, handler, max_concurrency=None)— fetches the run's prompt list once, drives every session's turns locally against a plain handler function, submits results, returns an aggregatedRedteamResult(results/progress/risk_score/run_number)/redteam/sdk/*endpointsnetra/simulation/module conventions (synchttpxclient, dataclass models, thread-per-session concurrency)netra/shutdown_hooks.py, a shared SIGINT/SIGTERM hook registry so an in-flight run is cancelled server-side on Ctrl-C — new capability, Python had no signal handling before this625/625 tests passing (81 new). Verified against a real running backend + real Postgres (not just mocks) — full multi-turn attack loop across single-turn, multi-turn, and iterative-jailbreak evaluators, judge scoring, progress/risk_score aggregation, and cancellation via SIGINT (confirmed cancelled status + synthetic result backfill in the DB). Also cross-checked side-by-side against the JS SDK on the same config for both the completion and cancellation paths.