Generated appearance for a list box widget does not indicate the selected option

[simneol]

Description of the bug

Setting a choice field's value and regenerating its appearance produces a stream that is byte-identical whether an option is selected or not. /V is written correctly, but the generated appearance lays out the background, the border and every option, with nothing marking the selected row.

Because the generated stream replaces the one the document arrived with, a form that already had a correct appearance loses it, and after pdf_bake_document() or a render to a pixmap there is no widget left to correct.

Check boxes and text fields do reflect their value in the same code path (/AS is set for a check box, /V is laid out for a text field), so this looks specific to choice fields. For comparison, another producer given the same widget and /I [0 2 4] emits the same stream plus one filled rectangle per selected row, drawn under the glyphs.

How to reproduce the bug

The script below writes its own one-page fixture with a multi-select list box, then:

before = appearance_bytes(annot)

pymupdf.mupdf.pdf_set_choice_field_value(annot, "Charlie")
pymupdf.mupdf.pdf_update_annot(annot)

after = appearance_bytes(annot)
assert before == after   # holds

Output:

/V after the call : 'Charlie'
/I after the call : absent

appearance before : 263 bytes
appearance after  : 263 bytes
identical         : True

The generated stream:

/Tx BMC
q
.94 .94 1 rg  0 0 160 100 re f
2 w  0 0 .5 RG  1 1 158 98 re s
2 2 156 96 re W n
BT
0 0 0 rg
4 96 Td
0 -13.392 Td /Helv 12 Tf (Alpha) Tj
0 -13.392 Td /Helv 12 Tf (Bravo) Tj
0 -13.392 Td /Helv 12 Tf (Charlie) Tj
0 -13.392 Td /Helv 12 Tf (Delta) Tj
ET
Q
EMC

I expected something in there to mark Charlie. The same happens through Widget.update(), and setting NeedAppearances does not change what gets drawn.

The example goes through pymupdf.mupdf.* rather than the Widget layer, in case that is useful for narrowing it down.

Minor, possibly related: pdf_set_choice_field_value() leaves /I absent. For a multi-select list box with repeated option strings, /I is the only unambiguous record of which rows are selected.

repro_listbox_appearance.py (needs only pip install pymupdf)

#!/usr/bin/env python3
"""Minimal reproduction: MuPDF's generated appearance for a choice (list box)
widget does not indicate which option is selected.

Everything under test goes through the MuPDF C API (`pymupdf.mupdf.*`), not
PyMuPDF's own Widget layer, so the result is not a binding-level artefact:

    pdf_set_choice_field_value(annot, "Charlie")   # MuPDF sets /V
    pdf_update_annot(annot)                        # MuPDF regenerates /AP

The appearance stream before and after are byte-identical.

    pip install pymupdf
    python repro_listbox_appearance.py
"""

import pymupdf

m = pymupdf.mupdf
FIXTURE = "listbox.pdf"
OUT = "listbox_after.pdf"


def build_fixture(path):
    """Not under test -- a one-page PDF holding one multi-select list box with
    a background and a border, so the generated appearance has something to
    draw besides the option text."""
    doc = pymupdf.open()
    page = doc.new_page(width=200, height=150)
    widget = pymupdf.Widget()
    widget.field_name = "listbox"
    widget.field_type = pymupdf.PDF_WIDGET_TYPE_LISTBOX
    widget.field_flags = pymupdf.PDF_CH_FIELD_IS_MULTI_SELECT
    widget.rect = pymupdf.Rect(20, 30, 180, 130)
    widget.choice_values = ["Alpha", "Bravo", "Charlie", "Delta"]
    widget.fill_color = (0.94, 0.94, 1.0)
    widget.border_color = (0, 0, 0.5)
    widget.border_width = 2
    page.add_widget(widget)
    doc.save(path)
    doc.close()


def appearance_bytes(annot):
    ap = m.pdf_annot_ap(annot)
    if not ap.m_internal:
        return b"<no appearance stream>"
    return bytes(m.fz_buffer_extract(m.pdf_load_stream(ap)))


def main():
    build_fixture(FIXTURE)

    doc = pymupdf.Document(FIXTURE)
    pdf = pymupdf._as_pdf_document(doc)
    page = m.pdf_load_page(pdf, 0)
    annot = m.pdf_first_widget(page)

    print(f"pymupdf {pymupdf.version[0]}  /  mupdf {pymupdf.version[1]}")
    print(f"widget type  : {m.pdf_widget_type(annot)} "
          f"(PDF_WIDGET_TYPE_LISTBOX = {pymupdf.PDF_WIDGET_TYPE_LISTBOX})")
    print(f"multi-select : {m.pdf_choice_widget_is_multiselect(annot)}")
    print()

    before = appearance_bytes(annot)

    # --- the two MuPDF calls under test ------------------------------------
    m.pdf_set_choice_field_value(annot, "Charlie")
    m.pdf_update_annot(annot)
    # -----------------------------------------------------------------------

    after = appearance_bytes(annot)
    doc.save(OUT)

    obj = m.pdf_annot_obj(annot)
    value = m.pdf_dict_get(obj, pymupdf.PDF_NAME("V"))
    indices = m.pdf_dict_get(obj, pymupdf.PDF_NAME("I"))
    print(f"/V after the call : {m.pdf_to_text_string(value)!r}")
    print(f"/I after the call : "
          f"{'absent' if not indices.m_internal else 'present'}")
    print()
    print(f"appearance before : {len(before)} bytes")
    print(f"appearance after  : {len(after)} bytes")
    print(f"identical         : {before == after}")
    print()
    print("appearance stream after the selection was set:")
    print(after.decode("latin-1"))

    if before == after:
        print("REPRODUCED: the selection is recorded in /V but nothing in the "
              "appearance stream marks it.")
    else:
        print("NOT reproduced on this build -- the streams differ.")


if __name__ == "__main__":
    main()

PyMuPDF version

1.28.2

Operating system

Linux

Python version

3.12

[JorjMcKie]

Thanks for the submission. Thanks for exclusively using mupdf.py functions demonstrating that the gap is not rooted in PyMuPDF. I will add a reproducer PDF here.

widgettest.pdf
Result after changing the selection for field "ListBox-1" to "Potsdam" show that only the /V value is changed:

After.pdf

MuPDF issue link: https://bugs.ghostscript.com/show_bug.cgi?id=709691

[simneol]

Thanks for picking this up so quickly.

One thing your widgettest.pdf shows that my fixture didn't: with /Opt on the parent field rather than on the widget, the regenerated stream loses all the option text as well (415 -> 86 bytes, 8 options -> 0), so the list box comes out blank rather than just unhighlighted. Copying only /Opt onto the widget restores it. Might be worth carrying upstream, since that parent/kid split is the usual Acrobat layout.