A JSON body that starts with a brace can be parsed from the middle #1965

Activity

1. wahajahmed010 commented on Oct 9, 2026

   

   wahajahmed010

   on Oct 9, 2026

   More actions

   Confirmed and reproduced locally against master. Trace of load_prefixed_json('{{"a": 1}') against the current parse_prefixed_json:

   PREFIX_REGEX = r'[^{\["]+' (in httpie/output/lexers/json.py).

   re.findall over '{{"a": 1}':

   • pos 0 { → not in [^{\["], no match start
   • pos 1 { → same
   • pos 2 " → not in [^{\["], no match start
   • pos 3 a → matches, greedy run = a (next : at pos 4 stops it)
   • pos 4 : → not in [^{\["]
   • pos 5 → not in [^{\["]
   • pos 6 1 → matches, greedy run = 1 (next } at pos 7 stops it)

   findall returns ['a', '1']. matches[0] is 'a', so data_prefix = 'a' and body = data[1:] = '{"a": 1}', which is valid JSON. The caller gets back (prefix='a', obj={"a": 1}) — the bug.

   The while(1);{"a": 1} case happens to work because the prefix while(1); is at the front of the string and is what findall matches first.

   The fix. parse_prefixed_json should look for the prefix only at the start of the string, not anywhere it occurs. re.match (or re.fullmatch over a leading-anchored pattern) is the right tool. Two ways to land it:

   1. Anchor the regex. Change PREFIX_REGEX to r'\A[^{\["]+' (the \A anchors to start-of-string) and switch re.findall to re.match. Returns a single Match object; match.group(0) is the prefix, data[len(match.group(0)):] is the body.

   2. Use the lexer's existing pattern. EnhancedJsonLexer (same file) already uses (PREFIX_REGEX)((?:[{\["]|true|false|null).+) which is anchored by Pygments at the start of the root state. The standalone parse_prefixed_json is the only place in the codebase that uses PREFIX_REGEX unanchored, and there's no reason for it to be — a "data prefix" only makes sense at the head of the response. Going with option 1 is the minimal fix.

   Regression test. Add to tests/test_output_utils.py (or wherever load_prefixed_json is tested):

   def test_load_prefixed_json_does_not_match_inside_body():
       # Bug #1965: re.findall found a non-prefix match inside the body.
       prefix, body = load_prefixed_json('{{"a": 1}')
       assert prefix == ''  # not 'a'
       assert body == {'a': 1}  # raises ValueError instead under the old behavior
   
   def test_load_prefixed_json_strips_real_xssi_prefix():
       prefix, body = load_prefixed_json('while(1);{"a": 1}')
       assert prefix == 'while(1);'
       assert body == {'a': 1}
   
   def test_load_prefixed_json_no_prefix():
       prefix, body = load_prefixed_json('{"a": 1}')
       assert prefix == ''
       assert body == {'a': 1}

   The first one is the regression test for this issue; the latter two are the existing-behavior lockins to keep the fix from overcorrecting.

   A subtle thing to be aware of. The EnhancedJsonLexer uses PREFIX_REGEX in a multi-group Pygments pattern where anchoring is handled by Pygments' root state. If you switch PREFIX_REGEX to \A[^{\["]+, the lexer pattern is still anchored by Pygments so it still works. If you keep PREFIX_REGEX unanchored and only change the standalone parse_prefixed_json to use re.match(PREFIX_REGEX, data), that's a smaller-diff fix and avoids touching the lexer. I'd go with the smaller-diff approach.

   Thanks for the tight repro — from httpie.output.utils import load_prefixed_json; print(load_prefixed_json('{{"a": 1}')) is the cleanest way to show this.
2. SashaMIT commented on Oct 9, 2026

   

   SashaMIT

   on Oct 9, 2026

   Author

   More actions

   Thanks for tracing this so carefully. The smaller change you describe, re.match on the existing pattern so only a prefix at the front counts, is already in #1966. Once the search stays at the front, {{"a": 1} has an empty prefix and the remainder is not JSON, so load_prefixed_json raises ValueError. while(1);{"a": 1} still strips the prefix.