Resolve string concatenation expressions in field values (#396)

[gaoflow]

Summary

Fixes #396 — string concatenation expressions in entry field values are now interpolated.

This is the second of the two PRs @MiWeiss outlined in the issue. The first (splitter capturing the full concatenation expression) is already merged; this one adapts ResolveStringReferencesMiddleware to actually resolve it.

The bug

ResolveStringReferencesMiddleware only handled a field whose entire value was a single string reference. For a concatenation it failed in two ways:

import bibtexparser
lib = bibtexparser.parse_string('@STRING{ jan = "Jan." }\n@INBOOK{x, month = 10 # "~" # jan,}')
lib.entries[0].fields_dict['month'].value
# before: '10 # "~" # jan'   (looked up as a single key → KeyError → left raw)
# after:  '10~Jan.'

A purely-quoted expression was even worse — it was skipped by the enclosed-value guard and then mangled by RemoveEnclosingMiddleware:

lib = bibtexparser.parse_string('@INBOOK{x, title = "hello" # " " # "world",}')
# before: 'hello" # " " # "world'   (outer quotes stripped off the whole expression)
# after:  'hello world'

The fix

In ResolveStringReferencesMiddleware.transform, before the existing single-reference logic, detect a top-level # concatenation (the # splitting respects "..." and {...} so e.g. {C# program} is not treated as a concatenation). Each token is resolved independently:

• a number → kept verbatim,
• a quoted/braced literal → its inner content,
• a string reference → its resolved value (with enclosing stripped),

then the parts are joined. The result is kept enclosed in braces so the downstream enclosing middlewares treat it as a plain string — this matters for writing: month = {10~Jan.} is valid BibTeX that round-trips, whereas a bare month = 10~Jan. would be re-parsed as a (broken) reference.

If any reference in the expression is unknown, the original expression is left untouched (no partial/incorrect resolution).

Scope

This handles concatenation in entry fields (the issue body). It does not yet resolve concatenation inside @string definitions (the asiacrypt91name = asiacryptname # "'91" case @kmccurley raised) — that needs string-to-string resolution and is worth a separate change. Happy to follow up on it.

Tests

Added to tests/middleware_tests/test_interpolate.py:

• test_string_interpolation_resolves_concatenation — number/quote/brace/reference mix at the middleware level
• test_string_interpolation_leaves_unresolvable_concatenation_untouched — unknown reference is a no-op
• test_parse_string_resolves_concatenation_end_to_end — exact issue Unexpected concatenation of field tokens #396 repro, plus a write + re-parse round-trip assertion

Full suite: 2579 passed, 12 skipped (3 new, no regressions). black / isort / flake8 / pyupgrade / docstr-coverage all clean (pinned versions).

---

This pull request was prepared with the assistance of AI, under my direction and review.

[claell]

I independently explored a broader version of this fix while auditing the parser's round-trip behavior. I have published the result as a comparison branch rather than opening a competing PR:

• Browse the branch
• Compare it with upstream main

In addition to resolving ordinary entry-field concatenations, the branch tests and handles:

• concatenations inside @string definitions;
• recursively chained string definitions;
• cyclic definitions without a recursion error;
• exact preservation of unresolved expressions instead of partial resolution; and
• preservation of unresolved quoted or braced multi-token expressions through RemoveEnclosingMiddleware.

The branch has two commits so the expression resolver and the enclosing-middleware safeguard can be reviewed independently. Validation is clean: 2584 passed, 12 skipped, plus the complete pre-commit hook suite. Please feel free to compare the implementation or cherry-pick/adapt any of its tests and cases for this PR.

For transparency, this comparison implementation and its tests were prepared with ChatGPT Codex using GPT-5.6 Sol with high reasoning effort. They should still receive normal maintainer review, especially because the work deliberately covers more cases than the original report.