feat(text): Shape complex single-line UI text

[OmarAglan]

• Replaces feat(text): Shape complex single-line UI text #3231
• Relates to Game Text Compiler requires feature for reversing Arabic text #2
• Builds on chore(lib): Implement Usp10Loader class #3241
• Depends on chore(lib): Define Uniscribe itemization control #3287

Adds Windows Uniscribe shaping for eligible single-line UI text in Generals and Zero Hour. The existing renderer draws individual UTF-16 characters, preventing normally authored Arabic text from displaying its contextual letter forms.

The new path analyzes the paragraph before selecting font runs, preserving bidirectional ordering and directional overrides. Font boundaries keep combining sequences and surrogate pairs together. Paragraph-direction detection handles supplementary Unicode characters and excludes directional-isolate contents when finding the first strong character.

Accepted lines are rasterized once per sentence build and copied into the existing A4R4G4B4 sentence textures. The primary font is retained for Latin characters and the configured Unicode font is used for other runs. Identical fonts do not create unnecessary splits.

The change also:

• Keeps measured dimensions consistent when rendering falls back.
• Refreshes hotkey data when labels change and tracks resources created by size queries.
• Synchronizes GDI drawing before bitmap access and checks allocation dimensions.
• Shares layout and drawing helpers, removes duplicate change tracking, and reuses shaped dimensions during texture allocation.

Scope

Plain Latin-only strings, multiline text, lines exceeding the wrapping width or texture-height limits, monospaced text, and labels with active hotkey markers retain legacy rendering. Editable text, password text, and IME construction strings also retain that path.

Multiline shaping and RTL caret, selection, and IME behavior are outside this PR’s scope. Unavailable or unsuccessful shaping falls back to the legacy renderer.

Validation

• Clean Release builds of Generals and Zero Hour using the repository’s pinned VC6 toolchain.
• Affected translation units compiled for both games with MinGW GCC 16.2.
• Local native checks: 24 paragraph-direction cases and 20 layout/pixel comparisons against whole-paragraph Uniscribe output, with zero failures.
• Local checks for font and cluster boundaries, measurement/raster consistency, size-limit rejection, failure cleanup, repeated rendering, and loader unload/reload.
• Unicode table regeneration check and git diff --check.

Before

After

Implementation developed with AI assistance.

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Shape complex single-line UI text with Uniscribe

✨ Enhancement 🐞 Bug fix 🕐 40+ Minutes

AI Description

• Shape complex single-line UI text with contextual glyphs and bidirectional ordering.
• Measure and rasterize eligible runs through dynamically loaded Windows Uniscribe.
• Preserve legacy rendering for Latin, multiline, hotkey, monospaced, and editable text.

Diagram

graph TD
  A["Display String"] --> B["Sentence Renderer"] --> C{"Complex eligible?"}
  C -- Yes --> D["Windows Uniscribe"] --> E["GDI Raster"] --> F["Sentence Textures"]
  C -- No --> G["Legacy Glyph Path"] --> F

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Adopt DirectWrite rendering

• ➕ Provides a modern shaping and text-layout stack
• ➕ Could support multiline layout and caret metrics in one backend
• ➖ Requires a broader renderer redesign beyond existing GDI HFONT integration
• ➖ Raises migration, compatibility, and review risk substantially

2. Link usp10 statically

• ➕ Removes function-pointer wrappers and lazy-load branching
• ➕ Makes missing APIs fail at build or process load time
• ➖ Introduces a hard Windows import dependency
• ➖ Loses graceful fallback to legacy rendering when Uniscribe is unavailable

Recommendation: Keep the runtime-loaded Uniscribe approach for this scoped change. It reuses the existing GDI font pipeline, preserves legacy behavior when shaping is unavailable or unsupported, and avoids a disproportionate DirectWrite migration; editable and multiline shaping can be added once caret and layout metrics are designed.

Files changed (12) +600 / -5

Enhancement (7) +406 / -5

DisplayString.hExpose per-string complex text control +1/-0

Expose per-string complex text control

• Adds an abstract switch allowing display-string implementations and UI controls to enable or disable complex shaping.

Core/GameEngine/Include/GameClient/DisplayString.h

render2dsentence.cppShape and rasterize eligible complex text runs +355/-3

Shape and rasterize eligible complex text runs

• Detects complex single-line text, measures and renders it as one Uniscribe run, and copies bounded raster chunks into existing sentence textures. It preserves the legacy path for excluded modes, unsupported dimensions, non-Windows builds, or shaping failures.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp

render2dsentence.hDeclare complex shaping renderer APIs +18/-2

Declare complex shaping renderer APIs

• Adds font-level complexity, measurement, and rasterization methods plus renderer eligibility, sizing, texture-building, and enablement state.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.h

W3DDisplayString.hExpose complex shaping in Generals display strings +1/-0

Expose complex shaping in Generals display strings

• Adds the W3D display-string override for controlling complex text shaping.

Generals/Code/GameEngineDevice/Include/W3DDevice/GameClient/W3DDisplayString.h

W3DDisplayString.cppUse shaped widths and propagate shaping state +15/-0

Use shaped widths and propagate shaping state

• Returns whole-run Uniscribe width when applicable and applies shaping enablement to normal and hotkey renderers. State changes invalidate cached text geometry.

Generals/Code/GameEngineDevice/Source/W3DDevice/GameClient/W3DDisplayString.cpp

W3DDisplayString.hExpose complex shaping in Zero Hour display strings +1/-0

Expose complex shaping in Zero Hour display strings

• Adds the W3D display-string override for controlling complex text shaping.

GeneralsMD/Code/GameEngineDevice/Include/W3DDevice/GameClient/W3DDisplayString.h

W3DDisplayString.cppUse shaped widths and propagate shaping state +15/-0

Use shaped widths and propagate shaping state

• Returns whole-run Uniscribe width when applicable and applies shaping enablement to normal and hotkey renderers. State changes invalidate cached text geometry.

GeneralsMD/Code/GameEngineDevice/Source/W3DDevice/GameClient/W3DDisplayString.cpp

Bug fix (2) +10 / -0

GameWindowManager.cppKeep Generals text entries on legacy rendering +5/-0

Keep Generals text entries on legacy rendering

• Disables complex shaping for editable, selected, and composition display strings because caret and partial-character metrics remain per-character.

Generals/Code/GameEngine/Source/GameClient/GUI/GameWindowManager.cpp

GameWindowManager.cppKeep Zero Hour text entries on legacy rendering +5/-0

Keep Zero Hour text entries on legacy rendering

• Disables complex shaping for editable, selected, and composition display strings until shaped caret positioning is supported.

GeneralsMD/Code/GameEngine/Source/GameClient/GUI/GameWindowManager.cpp

Other (3) +184 / -0

CMakeLists.txtBuild the Uniscribe loader on Windows +2/-0

Build the Uniscribe loader on Windows

• Registers the new runtime loader sources in the Windows-only WWLib source list.

Core/Libraries/Source/WWVegas/WWLib/CMakeLists.txt

Usp10Loader.cppLoad Uniscribe APIs safely at runtime +111/-0

Load Uniscribe APIs safely at runtime

• Implements one-time, lock-protected loading of the system usp10.dll and resolves the shaping APIs used by the renderer. Wrappers return failures when the library or required exports are unavailable.

Core/Libraries/Source/WWVegas/WWLib/Usp10Loader.cpp

Usp10Loader.hDefine the runtime Uniscribe interface +71/-0

Define the runtime Uniscribe interface

• Declares the required Uniscribe flags, opaque analysis types, API wrappers, and resolved function-pointer storage without adding a static import dependency.

Core/Libraries/Source/WWVegas/WWLib/Usp10Loader.h

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Mixed text uses fallback font ✓ Resolved 🐞 Bug ≡ Correctness

Description

The complex path selects AlternateUnicodeFont for the entire string, so Latin characters in mixed
Arabic/Latin UI text no longer use the requested primary font. Existing font behavior delegates only
non-ASCII characters to the configured Unicode fallback, so this changes Latin styling and metrics
whenever the two fonts differ.

Code

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[R1418-1419]

+	FontCharsClass *render_font = AlternateUnicodeFont && this != AlternateUnicodeFont ?
+		AlternateUnicodeFont : this;

Evidence

The normal font lookup keeps characters below 256 in the primary font and delegates only non-ASCII
characters to the alternate font. The new code instead analyzes and outputs the whole string using
the alternate font's DC/HFONT, while game font loading identifies that font specifically as the
Unicode fallback.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1312-1319]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1418-1427]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1492-1539]
Core/GameEngineDevice/Source/W3DDevice/GameClient/GUI/W3DGameFont.cpp[80-99]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Complex mixed-script strings are measured and rendered entirely with `AlternateUnicodeFont`, replacing the requested primary font for Latin runs.
## Issue Context
The existing character path uses the primary font for ASCII and delegates only non-ASCII characters to `AlternateUnicodeFont`. Preserve that division while shaping the complete bidi string, using run-level font selection/fallback consistently for both measurement and rendering.
## Fix Focus Areas
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1418-1427]
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1492-1539]
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1312-1319]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Unsupported runs rasterize twice 🐞 Bug ➹ Performance

Description

Build_Complex_Sentence creates the complete GDI bitmap and A4R4G4B4 raster before checking whether
its width exceeds WrapWidth. Every single-line complex string requiring wrapping therefore pays
for a full-run shape, bitmap allocation, pixel conversion, and discard before Build_Sentence
renders it again through the legacy wrapped path, with particularly high cost for long text.

Code

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[R687-688]

+	if (!Font->Rasterize_Complex_Text(text, &raster, &text_width, &text_height) ||
+		!Is_Complex_Text_Size_Supported(text_width, text_height))

Evidence

The complex predicate does not exclude wrapped renderers, while the support check rejects runs whose
measured width reaches WrapWidth. Build_Complex_Sentence invokes the allocating rasterizer
before that support check; after it returns false, Build_Sentence immediately continues into the
existing centered/non-centered renderer, proving the discarded first rendering is repeated work.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[621-632]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[642-645]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[680-692]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1292-1302]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1632-1645]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Complex text is fully rasterized before checking whether its dimensions qualify for the complex single-line path. If its width reaches the configured wrapping width, that raster is discarded and the legacy path renders the text again.
## Issue Context
Use the lightweight Uniscribe extent query and `Is_Complex_Text_Size_Supported` before allocating/rasterizing the full run. Retain a post-rasterization dimension check for defensive consistency if needed.
## Fix Focus Areas
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[680-692]
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[642-645]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Run rerasterized per chunk ✓ Resolved 🐞 Bug ➹ Performance

Description

Every texture-width chunk calls Blit_Complex_Text, which remeasures, reshapes, allocates a
full-run bitmap, and rasterizes the entire string before copying one slice. Because chunks are
capped by the texture width, rendering cost and allocated pixel work grow quadratically with long
single-line strings and repeat even for ordinary runs wider than one texture.

Code

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[R679-680]

+		if (!Font->Blit_Complex_Text(text, LockedPtr, LockedStride, TextureOffset.I,
+			TextureOffset.J, source_x, chunk_width))

Evidence

The outer loop advances source_x by at most the available texture width, but each iteration
invokes a helper that recomputes full extents, allocates a text_width-wide bitmap, reruns
Uniscribe analysis, and outputs the complete string. Thus a run split into N chunks performs N
full-run rasterizations rather than one rasterization plus N slice copies.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[664-690]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1494-1539]
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1542-1552]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The complex string is fully analyzed and rasterized once for every texture chunk, making long-run construction scale quadratically.
## Issue Context
`Build_Complex_Sentence` iterates over texture-sized slices, while `Blit_Complex_Text` recreates a full-width DIB and repeats `ScriptStringAnalyse`/`ScriptStringOut` on every call. Produce the full raster once, then copy each slice into its destination surface.
## Fix Focus Areas
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[664-690]
- Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp[1494-1552]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[greptile-apps]

Greptile Summary

The PR adds Windows Uniscribe shaping for eligible single-line complex text while retaining legacy rendering for unsupported and editable cases.

• Adds paragraph itemization, bidirectional run ordering, font-run shaping, and rasterization into existing sentence textures.
• Reconciles shaped measurements with legacy fallback dimensions before layout and drawing.
• Refreshes hotkey state when labels change and tracks resources created by size queries.
• Adds Unicode bidirectional data generation and mirrored display-string integration for Generals and Zero Hour.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
Core/Libraries/Source/WWVegas/WW3D2/complextext.h	Implements bounded Uniscribe layout and visual-order rendering with clean failure propagation.
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp	Integrates complex-text measurement and raster copying while resolving the previously reported oversized-text and fallback issues.
Generals/Code/GameEngineDevice/Source/W3DDevice/GameClient/W3DDisplayString.cpp	Rebuilds sentences before positioning, reconciles fallback dimensions, and refreshes hotkeys when text changes.
GeneralsMD/Code/GameEngineDevice/Source/W3DDevice/GameClient/W3DDisplayString.cpp	Mirrors the corrected complex-text fallback and hotkey lifecycle for Zero Hour.
Core/Libraries/Source/WWVegas/WW3D2/unicodebidi.h	Determines paragraph direction from BMP and supplementary UTF-16 characters while excluding isolate contents.
scripts/generate_supplementary_bidi.py	Generates the supplementary Unicode bidirectional classification table consumed by the renderer.

_{Reviews (21): Last reviewed commit: "feat(text): Shape complex single-line UI..." | Re-trigger Greptile}

[tintinhamans]

@codex

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a852d41fbb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[stephanmeesters]

I can't tell from the text and images what the problems were and how this fixes it.

The main-menu test confirms correct contextual shaping, bidirectional ordering, digit ordering, centering, and clipping.

Can you give before and afters of each of these individually?

[OmarAglan]

I can't tell from the text and images what the problems were and how this fixes it.

The main-menu test confirms correct contextual shaping, bidirectional ordering, digit ordering, centering, and clipping.

Can you give before and afters of each of these individually?

will provide examples of it as soon as possible

[OmarAglan]

I can't tell from the text and images what the problems were and how this fixes it.

The main-menu test confirms correct contextual shaping, bidirectional ordering, digit ordering, centering, and clipping.

Can you give before and afters of each of these individually?

will provide examples of it as soon as possible

the Arabic text as for now!

before

after

[OmarAglan]

draft to fix the vc6 issue

[bobtista]

We can handle VC6 in a small prerequisite PR by runtime-loading usp10.dll, using DbgHelpLoader as an example. Then this PR can drop the usp10 link and guard the Uniscribe code with _WIN32.

[qodo-free-for-open-source-projects]

Code review by qodo was updated up to the latest commit 74c1e27

[OmarAglan]

We can handle VC6 in a small prerequisite PR by runtime-loading usp10.dll, using DbgHelpLoader as an example. Then this PR can drop the usp10 link and guard the Uniscribe code with _WIN32.

yes im working on it!

[xezon]

In what shape is the arabic text you tested with? As far as I am aware the old translations had the words reversed to accomodate the game implementation. Can arabic text now be supplied normally?

[OmarAglan]

In what shape is the arabic text you tested with? As far as I am aware the old translations had the words reversed to accomodate the game implementation. Can arabic text now be supplied normally?

i used noraml text arabic, i didnt reverse the text, this fixes the hack that is to reverse the arabic text!
still i need to extand upon this on adding maybe support for arabic text in chat and multi line, this can be addressed in follow up pr.

[OmarAglan]

needs rebase and conflict fix, working on it.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[OmarAglan]

Rebased!

[OmarAglan]

@codex

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c986dedf67

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[xezon]

This change adds a lot of new code for the text rendering. Is all of it absolutely required? And is there no code duplication, aka new code doing the same as nearby other code, in part?

[OmarAglan]

This change adds a lot of new code for the text rendering. Is all of it absolutely required? And is there no code duplication, aka new code doing the same as nearby other code, in part?

right now, i can't simplify more than that!
so, the code as it is neccery for this to work, off course i maybe missed some things, waiting for your review!

[OmarAglan]

rebased and fixed all the apparent issues, ready for review!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ec473e0955

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[xezon]

There are left-over bot comments.

[OmarAglan]

rebased and fixed and ready for review

[OmarAglan]

this is getting so big! any advice?

[OmarAglan]

@codex

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c30ed4ae60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Synchronize extents when shaping recovers

If the measurement pass in computeExtents() temporarily fails to create/analyze a complex-text layout, it caches legacy extents and leaves m_hasComplexTextExtents false; a later draw can retry successfully and set usedComplexText true, but this one-way condition never replaces the cached legacy size. Consequently, callers can align or position the successfully shaped sentence using stale legacy dimensions. Update the cached size in both directions, returning shaped extents from the successful build or remeasuring them here; the mirrored Generals implementation has the same issue.

Useful? React with 👍 / 👎.