Repository navigation
perf(function_call): skip DeepSeek V3 regex scans until the JSON fence closes - #1716
Merged
Merged
Conversation
…e closes The streaming parser ran its DOTALL regex over the whole buffer on every delta. A JSON argument cannot match until a closing fence follows the json header, so the scan now waits for that. Output is unchanged. AI assistance (Cursor) was used for this change. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
deepseek_v3still selectsDeepSeekV3Detector. #1698 skips the V3.1, V3.2 and V4 scans until an end tag is in the buffer. The V3 pattern is different: it matches when a closing fence follows the json code block, and that fence arrives before the tool-call end token. The function name is emitted on the fence, so waiting for the end tag would change when the name and arguments are sent. This waits for the fence instead.Every delta ran that DOTALL regex over the whole accumulated text. A normal JSON argument contains no raw newline followed by a fence, because newlines inside strings are escaped, so the pattern cannot match until the format's own closing fence. The scan now runs only once that fence is present. The check is a
findthat starts after the json header, so a failed check means the regex would not have matched. Returned calls and parser state stay the same.If the argument text itself contains a newline followed by a fence, the pattern can match and the scan is not skipped. A 15 KB argument with that fence 2 KB in stays on the regex, about 0.74-0.92 s on both versions.
Parser time for one streamed call. Special tokens arrive whole and the argument is split into 4-character deltas, which is how a long
write_fileis streamed. Median of 5 fresh processes, one warmup in each,OMP_NUM_THREADS=1, CPython 3.12.16, Xeon Platinum 8352Y. The sha256 of the returned calls matches on every case.write_file, 4 KBwrite_file, 15 KBwrite_file, 32 KBget_weatherunit_tests/server/test_deepseek_tool_parser_stream.py: 7 passed on this change and on main. The three new V3 tests cover a long argument in 4-character deltas, a closing fence split across deltas, and a fence kept inside a JSON string.AI assistance (Cursor) was used for this change.