[stobrien89]

Issue #, if available:
#797

Description

Partitions large standalone types packages to reduce generation memory and static analysis time.

Schema partitioning now follows the strategy merged for the Java generator in #880, adapted so standalone types packages can still use operation reachability even though they do not emit operation schemas.

Changes

• Partition schemas above 100 shapes using weighted source-size boundaries.
• Keep recursive schema components together.
• Keep each operation's selected input and output together.
• Use operation reachability to move shared shape groups together when it reduces the number of partitions touched by related operations.
• Bound affinity moves to keep partition source size and shape counts reasonably balanced, and ignore very widely shared groups so they do not pull unrelated operations together.
• Partition model stubs above 250 declarations.
• Preserve stable schemas.py and models.pyi facades.
• Preserve single-file output for packages at or below each threshold.

Earlier EC2 benchmark

Measured on x86 with 4,065 selected shapes, five runs, and one warmup. These results used the earlier 250-shape layout, before operation affinity was added, and are retained to show the original partitioning results.

Metric	Unpartitioned	Partitioned	Change
Generation time	3.557 s	3.555 s	-0.1%
Peak RSS	264.5 MiB	122.4 MiB	-53.7%
Peak Python allocation	222.6 MiB	84.7 MiB	-61.9%
Generated bytes	4.78 MiB	5.33 MiB	+11.6%
Model import	50.6 ms	40.4 ms	-20.2%
First model access	12.3 ms	22.3 ms	+10.0 ms
Strict Pyright	81.98 s	15.25 s	-81.4%

Validation

• 567 smithy-python tests passed and 1 skipped after rebasing onto develop.
• Tests cover schema and stub output below, at, and above their thresholds.
• Tests verify recursive and operation I/O grouping, affinity improvements, deterministic operation ordering, cross-partition references, facade caching, and stable public exports.
• Ruff and strict Pyright pass for the updated implementation and tests.

Compatibility

Public model imports, lazy model construction, schema identity, serde behavior, and typing remain unchanged. Partition files live under _private.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[Copilot]

🟡 Changes recommended

Generated-output compatibility and lint failures remain, alongside schema-planning and partition-weighting issues that undermine scalability.

4 open issues

• Re-export unused private deserializer helpers · New
• Add planned schema constants to lazy facade all · New
• Avoid quadratic recursive component ordering · New
• Include interned string declarations in partition weight estimates · New

What changed in this PR

This PR extends the native standalone types generator with private schema and model-stub partitions, reducing memory use and static analysis time while retaining stable facade modules.

Changes:

• Adds configurable partition targets, defaulting to 250 items.
• Balances schema partitions by estimated source size and keeps recursive components together.
• Adds partitioning tests and documents output layout, compatibility, and benchmark results.

File	Description
packages/​smithy-python/​tests/​unit/​test_schemas.py	Tests thresholds, lazy loading, recursion, and size balancing.
packages/​smithy-python/​tests/​unit/​test_models.py	Tests partitioned stubs, public models, and strict Pyright.
packages/​smithy-python/​tests/​unit/​test_generation.py	Tests partition defaults and positive-value validation.
packages/​smithy-python/​src/​smithy_python/​schemas.py	Adds weighted schema partitions and a lazy facade.
packages/​smithy-python/​src/​smithy_python/​models.py	Adds private stub partitions and public re-exports.
packages/​smithy-python/​src/​smithy_python/​generation.py	Defines and validates partition settings.
designs/​codegen/​types.md	Updates settings and large-model design guidance.
designs/​codegen/​partitioning.md	Documents partitioning behavior and performance measurements.
designs/​codegen/​index.md	Links the partitioning design.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

Static review found no blocking issues in partitioning, facade behavior, or compatibility, and the supporting tests cover the key boundaries.

No open issues 🎉

4 resolved since last review

• Include interned string declarations in partition weight estimates
• Avoid quadratic recursive component ordering
• Add planned schema constants to lazy facade all
• Re-export unused private deserializer helpers

_{Review effort: Balanced}

[kuhe]

is first model access added time paid every partition that's touched for the first time?

also, what is happening in the additional 10ms?

[stobrien89]

@kuhe

is first model access added time paid every partition that's touched for the first time?

also, what is happening in the additional 10ms?

yeah, there’s a one-time cost when each schema partition is first touched, but it’s not 10 ms per partition

That 10 ms is mostly work moved out of the initial import. Import plus first access stayed basically the same at 62.9 ms unpartitioned versus 62.7 ms partitioned. After that, the loaded partitions and schemas are cached.