diff --git a/AGENTS.md b/AGENTS.md index e70be4d8a..2744026da 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -137,6 +137,7 @@ compilation should use the focused owners under `tests/fortran/infrastructure/building/compiling/` as applicable. Include the relevant end-to-end feature tests whenever a generated or compiled mechanism changes; run a broader suite when behavior spans multiple stages. +Run pytest with at most `-n 2`. Never `-n 4`, `-n 8`, or `-n auto`. The development machine has 12 cores but only about 7 GB of RAM, and every xdist worker loads NumPy while the Fortran end-to-end tests fork gfortran and cc per test on top of `pytest-monitor` profiling each one. Higher parallelism exhausts memory and thrashes swap, which has hard-frozen the machine and forced a reboot. Prefer the narrowest owning test path over a full suite run, and commit verified work promptly rather than batching it behind a long run. Do not run LAPACK wrapper tests locally unless the user explicitly asks for them. Local verification may run everything else, including BLAS-only real-library tests; leave LAPACK coverage to GitHub Actions by default. Do not run the full coverage workflow for routine changes. Run focused tests plus the required static-analysis suite. Reserve the complete CI-style coverage workflow for explicit pre-merge or pull-request verification, or when the user specifically requests it. When investigating coverage failures, mirror the GitHub Actions workflow before deciding the fix: run coverage with `COVERAGE_PROCESS_START=pyproject.toml`, combine parallel data with `python3 -m coverage combine`, then run `python3 -m coverage report`. Do not assume a plain local coverage run matches CI, especially when subprocess tests are involved. diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ccb2696c..7ded88fd9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,65 @@ release tags add a leading `v` to the package version. ## Unreleased +- Fixed Fortran allocatable and pointer descriptor arguments to use the + compiler's live descriptor. This works on Intel ifx as well as GNU Fortran, + preserves lower bounds, allocation and association changes, and covers + required and optional dummies, multiple descriptor dummies per call, + `intent(out)` and status-bearing calls, and handles from module variables, + fields, results, and caller-created storage. + +- Descriptor-backed native calls honor `@nogil` while the native procedure is + running, including calls with multiple allocatable or pointer arguments. + +- Deferred-length character pointer-array arguments are rejected before code + generation; module and field handles expose only operations that do not + require an unsupported `bind(C)` descriptor interface. + +- Generated array handles share descriptors between extensions through the + versioned `prik.native_array_backend.v1.` capsule. It runs a + consumer while the handle's descriptor is live and refuses incompatible ABI + versions or record layouts before reading the backend. Rebuild generated + extensions together after upgrading PRIK. + +- Fixed-width character module and field handles expose their storage through + an ordinary descriptor. They support inquiries, NumPy views, and ordinary + array arguments. Fixed-width character allocatable and pointer array + arguments are rejected because no interoperable descriptor interface carries + their allocation or association semantics. + +- Generated Fortran allocatable and pointer handles can be passed directly to + matching ordinary array arguments. Numeric assumed-shape and assumed-rank + arguments accept representable forward or reversed Fortran sections from + either handles or NumPy arrays, including direct `bind(C)` procedures. + Optional arrays apply the same layout rules when present and accept omission + or `None` as absence. Explicit-shape, assumed-size, and fixed- or + assumed-width character arrays retain their declared layout. C array + arguments accept NumPy arrays. + +- Array-valued functions whose result extents depend on descriptor arguments + return initialized NumPy arrays on Intel ifx and GNU Fortran. + +- Descriptor-backed NumPy views preserve native byte strides, including + negative strides, non-contiguous pointer targets, and zero-sized dimensions. + +- Writable module and derived-field allocatable arrays support allocation and + reassignment. PRIK rejects `protected` module arrays when the generated API + would require writable access. + +- Module array views cover fixed, target, allocatable, pointer, shifted-bound, + character, and logical storage. Ordinary fixed-shape module arrays expose + live NumPy views with or without `target`. + +- PRIK now selects interoperable Fortran logical storage on Intel and + PGI/NVIDIA compilers. Use `--no-standard-logicals` or + `standard_logicals=False` only when linking Intel objects compiled without + that option. Wider logical arrays are exposed with their matching integer + dtype, including caller-created allocatable and pointer handles; + `logical(c_bool)` remains `numpy.bool_`. A wider logical array argument is + now aliased rather than converted, so it takes that integer dtype where a + one-byte `numpy.bool_` buffer was previously copied in and out. Logical + scalars are unaffected and stay Python `bool` in every kind. + ## 0.4.3 — 2026-08-31 - Republishes 0.4.2. That tag carried the previous package version, so the diff --git a/docs/developer/packages/runtime.md b/docs/developer/packages/runtime.md index bb6269727..6e22dcd40 100644 --- a/docs/developer/packages/runtime.md +++ b/docs/developer/packages/runtime.md @@ -23,16 +23,114 @@ select a different view behavior from local descriptor facts. ## A Native Array Handle At Runtime ```text -generated operation dictionary + dtype, rank, ownership, and view policy +generated dispatcher + completed capability set + native backend capsule + + dtype, rank, ownership, and view policy -> NativeArrayHandleBase validation and owner retention -> AllocatableArray or PointerArray -> state, lifecycle, association, and to_numpy() operations ``` -The operation dictionary is the boundary between generated extension code and -the stable Python handle API. An operation exists only when the completed plan -allows the generator to expose it. Missing operations fail explicitly rather -than being inferred from `allocatable` or `pointer` alone. +The dispatcher is the single Python call boundary between generated extension +code and the stable handle API. Its immutable capability set comes from the +completed plan and states which operation names the dispatcher accepts. The +runtime validates that set when it creates the handle and before dispatching an +operation. + +### The Backend Capsule + +Every generated handle publishes one versioned capsule, +`prik.native_array_backend.v1`, on `_native_backend`. It is the whole +cross-extension ABI for an array handle: + +```c +typedef struct { + uint32_t descriptor_kind; + uint32_t descriptor_attribute; + uint32_t rank; + uint32_t descriptor_size; + int32_t cfi_type; + size_t element_size; + void *context; + prik_native_array_with_descriptor_fn with_descriptor; + prik_native_array_release_fn release; +} prik_native_array_backend; +``` + +`with_descriptor(context, consumer, consumer_context)` supplies a live +descriptor and runs the consumer on it: + +- **Borrowed** — a module variable or a derived-type field. The entry point + enters Fortran and supplies the plan-selected descriptor for that call. The + descriptor is gone when the consumer returns and must never be retained, + copied, or serialized. An ordinary projection does not invoke the consumer + while its allocatable or pointer entity has no storage; inquiries return the + corresponding absent value. +- **Owned** — a native result, or a contract handle that has been given + storage. The binding allocated a descriptor and keeps it for the handle's + life, so the entry point hands that storage straight to the consumer. + +Consumers use the same contract for both forms. `context` is the parent's +address for a field, descriptor storage for an owned handle, and `NULL` for a +module variable. `release` is non-`NULL` when the extension owns `context`. +Clearing `context` after release makes `close()` and finalization idempotent. + +The capsule name is `prik.native_array_backend.v1.`. The version identifies +the callback contract and field meanings. Change it when either changes without +changing the C record layout. The tag folds the record size and each field's +name, offset, and width, so a layout mismatch also changes the name. +`PyCapsule_GetPointer` compares that name before returning the record pointer; +an incompatible producer is therefore refused before its fields are read. + +`descriptor_size` stays in the record because it attests the producer's +`CFI_CDESC_T(rank)` layout, which is the compiler's, not this header's, and so +is not folded into the tag. `descriptor_kind` identifies the native entity; +`descriptor_attribute` identifies the descriptor supplied to a consumer. A +descriptor-dummy call requires both to match, while ordinary-array consumers +can use a descriptor with the `other` attribute. Readers also validate `rank`, +`cfi_type` and `element_size`. `element_size` is `0` for widths determined at +run time, such as deferred-length character arrays. + +### Inquiries Read The Descriptor + +`shape`, `allocated`, `associated`, `contiguous`, `element_length` and +`to_numpy` are all answered by small shared C consumers run through +`with_descriptor`, for borrowed and owned handles alike. Each consumer returns +the completed Python value. The bridge provides the descriptor entry point and +the mutations that act on the entity itself: `allocate`, `resize`, +`deallocate`, `nullify`, `associate`, and `destroy`. + +A pointer additionally reports `descriptor` as a flat fact tuple: base address, +element width, rank, then a lower bound, extent, and byte stride per axis. A +pointer assignment uses this snapshot. A handle created from a `.pyi` contract +retains the snapshot until a call attaches native storage and replays the +association. + +A call with more than one allocatable or pointer dummy enters each argument's +backend in turn. Each consumer records its descriptor and enters the next, and +the call runs inside the last consumer while every descriptor is live. Borrowed +and owned handles use the same placement; an owned backend hands the consumer +its persistent storage. An absent optional argument contributes an unallocated +placeholder to the chain. Each descriptor remains scoped to the consumer that +supplied it. + +Ordinary numeric assumed-shape and assumed-rank arguments use the same C +descriptor entrypoint for both direct and adapted calls. The binding describes +a NumPy array with call-local descriptor storage, or enters a handle's live +descriptor through the consumer chain. A direct `bind(C)` procedure receives +that descriptor itself; a non-`bind(C)` procedure has an interoperable bridge +dummy that passes the array through unchanged. Explicit-shape, assumed-size, +raw C-pointer, and character-array entrypoints keep their planned address ABI. + +### Views And Ownership + +`to_numpy()` builds the view in C while the descriptor is live, over the +storage the descriptor names, with the descriptor's own byte strides — so +negative strides, non-contiguous pointer targets and zero-sized dimensions all +come through unchanged. The view's base is what keeps that storage valid: the +parent object for a derived-type field, the backend capsule for an owned +handle, and nothing for a module variable, whose storage outlives every view of +it. An owned handle's view additionally retains the handle, because closing the +handle is what releases the storage. ## Local Structure @@ -45,11 +143,13 @@ prik/runtime/ ``` - [`handles.py`](../../../prik/runtime/handles.py) contains the Python runtime. - `NativeArrayHandleBase` validates common metadata and operations. + `NativeArrayHandleBase` validates common metadata, the dispatcher, and its + completed capabilities. `AllocatableArray` adds allocation state, resize, and deallocation; `PointerArray` adds association, nullification, allocation, resize, and - deallocation when supplied. Internal adapters translate generated call - signatures and descriptor handoffs. + deallocation when supplied. A handle created from a `.pyi` contract answers + from a fact tuple of its own until a call attaches generated storage; every + other handle answers from its descriptor. - `native_support/prik_binding.h` contains header-only CPython/NumPy conversion, descriptor, validation, capsule, and release support. Change it only with its generated C users and `prik/compiler/native_support.py`. @@ -57,9 +157,11 @@ prik/runtime/ `to_numpy()` returns `None` for an absent allocatable or pointer and otherwise validates the completed view policy, dtype, rank, and any required contiguity. -Native argument handoff performs the additional expected shape, layout, -alignment, byte-order, and writeability checks. A returned NumPy array is a -view of native storage; a caller that needs independent storage must copy it. +Native argument handoff is performed in the binding, against the live +descriptor, and refuses a mismatched dtype, rank, fixed shape, character width, +layout, byte order, alignment, writeability, or contiguity there. A returned +NumPy array is a view of native storage; a caller that needs independent +storage must copy it. ## Run The Handle Demonstration @@ -75,10 +177,10 @@ Resized shape: (4,) Generated resize received NumPy extents: True ``` -The example supplies the same operation-dictionary shape as generated code. -It creates an allocatable handle, reads its live NumPy view, and routes a -resize through the adapter. The native header has no standalone Python route; -the compiler installs it into a generated `binding_support/` directory. +The example supplies the same dispatcher and capability set as generated code. +It creates an allocatable handle, reads its live NumPy view, and routes a resize +through the dispatcher. The compiler installs the native header into the +generated `binding_support/` directory. ## Change Routes And Evidence diff --git a/docs/user/guide/allocatables.md b/docs/user/guide/allocatables.md index f9dbeb843..5c2d72bba 100644 --- a/docs/user/guide/allocatables.md +++ b/docs/user/guide/allocatables.md @@ -50,10 +50,21 @@ Use ordinary `T[...]` when the callable needs only array data: def sum_values(values: Float64[:]) -> Float64: ... ``` +An allocated handle may be passed to that ordinary Fortran array argument; +PRIK reads the live storage from its descriptor and applies the ordinary array +contract. This also works for optional, strided, flattened, and assumed-rank +ordinary arguments when their declared constraints match the handle. + A plain NumPy array cannot satisfy an `Allocatable[T[...]]` parameter because it does not carry native allocation state. Use `to_numpy()` when Python needs the current array data held by an allocatable handle. +PRIK does not wrap `Allocatable[String[N][...]]` parameters: a fixed-width +character allocatable array has no interoperable allocatable descriptor +interface that preserves its allocation semantics. A fixed-width character +module or field handle can still be viewed with `to_numpy()` and passed to an +ordinary `String[N][...]` array parameter. + --- ## Allocatable Array Handle API diff --git a/docs/user/guide/arrays.md b/docs/user/guide/arrays.md index cca59e2fc..dcac58c76 100644 --- a/docs/user/guide/arrays.md +++ b/docs/user/guide/arrays.md @@ -481,7 +481,7 @@ This checks the final Python axis and flattens the leading axes. ## Strided Views Use `::` for an assumed-shape axis that accepts F-contiguous arrays and -positive-stride views without copying: +Fortran-ordered strided views without copying: ```python from prik.contracts import Float64 @@ -511,9 +511,16 @@ print(out) # [21. 45. 69.]] ``` -PRIK passes the base address, extents, and positive element strides. Reversed -slices, broadcasted views, and C-order strided matrices are rejected for this -Fortran-oriented contract. Strides are not an order workaround. +Numeric assumed-shape and assumed-rank arguments receive a Fortran descriptor, +so an axis may run forward or backward. For example, +`scale_visible_rows(visible_rows[::-1, :], out[::-1, :])` updates the same +selected rows in reverse order. + +The view must still be a non-overlapping Fortran-ordered array section. +Broadcasted views, overlapping views, and C-order strided matrices are +rejected. Strides are not an order workaround. Explicit-shape, assumed-size, +and character arrays use an address-based entrypoint and therefore require a +forward layout that their contract can express. --- @@ -548,7 +555,18 @@ Use this list when reading or editing a generated `.pyi` contract: - `T[rows, Flat]`: Fortran-contiguous; checked prefix, remaining axes flattened - `Annotated[T[Flat, columns], ORDER_C]`: C-contiguous; checked suffix, leading axes flattened -- `T[...]`: assumed-rank, currently rank 1-15 +- `T[...]`: assumed-rank, currently rank 1-15, including supported reversed + sections + +An allocated `Allocatable[T[...]]` handle or associated `Pointer[T[...]]` +handle can also satisfy a matching ordinary Fortran array argument. The same +element type, rank, shape, layout, contiguity, and writeability requirements +apply as for a NumPy array. This includes explicit-shape, assumed-shape, +signed-strided, assumed-size/`Flat`, and assumed-rank arguments, plus +fixed-width and assumed-width character arrays. Reversed targets are accepted +by matching numeric assumed-shape and assumed-rank arguments. An absent handle +is rejected; pass `None` only when the ordinary argument itself is optional. +C array arguments accept NumPy arrays, not Fortran descriptor handles. Generated contracts may describe a shape with visible arguments, such as `T[rows, columns]`. Most users should keep those generated relationships diff --git a/docs/user/guide/building-shared-library.md b/docs/user/guide/building-shared-library.md index 78d22f7db..697cb475b 100644 --- a/docs/user/guide/building-shared-library.md +++ b/docs/user/guide/building-shared-library.md @@ -59,6 +59,13 @@ GNU, IFX, and Flang are tested on Linux. See [Compiler Toolchains](../getting-started/installation.md#compiler-toolchains) for versions and other recognized options. +On Intel PRIK adds `-standard-semantics` (and `-Munixlogical` on PGI/NVIDIA) so +a Fortran `logical` has the representation C expects. Pass +`--no-standard-logicals` to leave it out — needed only when you link prebuilt +Intel objects that were themselves compiled without it, since the option also +changes module symbol mangling. See +[Compiler options for interoperable logicals](data-types.md#compiler-options-for-interoperable-logicals). + ## Build a primitive C API directly PRIK supports C source as well. Start with the [C User Guide](c/index.md) for diff --git a/docs/user/guide/data-types.md b/docs/user/guide/data-types.md index 1be2a3851..c778036fe 100644 --- a/docs/user/guide/data-types.md +++ b/docs/user/guide/data-types.md @@ -206,19 +206,75 @@ double` is IEEE quad, `real(16)` maps to it instead. PRIK decides from the mantissa width the compiler reports, never from storage size — see [Unsupported Widths And Forms](#unsupported-widths-and-forms). -Boolean contract names describe native storage, not different Python dtypes: +Boolean contract names describe native storage. Scalars cross as Python +`bool`. Arrays are aliased element by element: one-byte logical arrays use +`numpy.bool_`, and wider kinds use the integer dtype of matching width. | Semantic Contract | Native Logical Storage Represented | Scalar Input | Direct Result | Array Storage | | --- | --- | --- | --- | --- | | `Bool` | 8 bits; portable default, equivalent to `Bool8` | `bool` or `np.bool_` | `bool` | `dtype=np.bool_` | | `Bool8` | 8 bits | `bool` or `np.bool_` | `bool` | `dtype=np.bool_` | -| `Bool16` | 16 bits | `bool` or `np.bool_` | `bool` | `dtype=np.bool_` | -| `Bool32` | 32 bits | `bool` or `np.bool_` | `bool` | `dtype=np.bool_` | -| `Bool64` | 64 bits | `bool` or `np.bool_` | `bool` | `dtype=np.bool_` | +| `Bool16` | 16 bits | `bool` or `np.bool_` | `bool` | `dtype=np.int16` | +| `Bool32` | 32 bits | `bool` or `np.bool_` | `bool` | `dtype=np.int32` | +| `Bool64` | 64 bits | `bool` or `np.bool_` | `bool` | `dtype=np.int64` | Generated contracts select a numbered name after probing the chosen compiler. -Callers never pass integer arrays for wider logical storage: the wrapper adapts -the one-byte NumPy Boolean representation at the native boundary. + +A `logical(c_bool)` array uses `numpy.bool_`. Wider logical kinds use the +integer dtype of matching width and can be read as Boolean values with +`.astype(bool)`: + +```python +flags = mod.flags # dtype bool for logical(c_bool) +wide = mod.wide # dtype int32 for a default `logical` +wide.astype(bool) # array([True, False, True]) +wide[0] = 0 # visible to Fortran +``` + +Write only `0` or `1` into an integer-typed logical array. Other integer values +are not portable Fortran logical representations. + +### Compiler options for interoperable logicals + +PRIK requests each compiler's interoperable representation for Fortran +`logical` values: + +| Compiler | Option PRIK passes | +| --- | --- | +| gfortran, Cray, IBM XL | none needed | +| Intel `ifx` / `ifort` | `-standard-semantics` | +| PGI / NVIDIA | `-Munixlogical` | + +Keep the listed option when overriding PRIK's compiler flags. + +#### Turning it off for prebuilt Intel objects + +On Intel, `-standard-semantics` also changes how module symbols are mangled. +A variable `flag` in module `logtest` is emitted as `logtest_MP_flag_` with the +option and as `logtest_mp_flag_` without it. Objects compiled with the option +and objects compiled without it therefore cannot be linked together, and mixing +them fails with `undefined reference to lib_MP_name_` rather than with anything +about logicals. + +So if you link PRIK against a **prebuilt** Intel library or object file that +was compiled without `-standard-semantics`, either rebuild that library with +the option — the better fix — or tell PRIK to leave the option out: + +```bash +python3 -m prik lib.f90 --compiler ifx --no-standard-logicals +``` + +```python +build_fortran_extension( + "lib.f90", + preprocessing=PreprocessingConfig(compiler="ifx"), + standard_logicals=False, +) +``` + +Disabling standard logicals also disables the interoperable representation +guarantee. Prefer rebuilding the dependency with `-standard-semantics` when +possible. --- diff --git a/docs/user/guide/pointers.md b/docs/user/guide/pointers.md index 91c5b7d24..b9dee8aca 100644 --- a/docs/user/guide/pointers.md +++ b/docs/user/guide/pointers.md @@ -49,10 +49,14 @@ Use ordinary `T[...]` when the callable needs only array data: def sum_values(values: Float64[:]) -> Float64: ... ``` -An associated pointer handle may satisfy an ordinary array parameter when its -dtype, rank, shape, layout, and contiguity meet that parameter's contract. A -plain NumPy array cannot satisfy a `Pointer[T[...]]` parameter because it does -not carry a native pointer descriptor. +An associated pointer handle may satisfy an ordinary Fortran array parameter +when its dtype, rank, shape, layout, contiguity, and writeability meet that +parameter's contract. Forward- and reverse-strided numeric targets are +accepted by matching assumed-shape and assumed-rank arguments; address-only +and contiguous arguments retain their declared layout requirements. Optional +and flattened ordinary arguments use the same rules. A plain NumPy array +cannot satisfy a `Pointer[T[...]]` parameter because it does not carry a native +pointer descriptor. --- @@ -97,6 +101,17 @@ boundary as values rather than array handles. support allocation, target deallocation, resizing, and NumPy extraction. An unavailable operation raises `NotImplementedError`. +A deferred-length character pointer array can report `associated`, `shape`, +and its current element width, and can be nullified or deallocated. It cannot +be passed as a pointer-descriptor argument or exposed with `to_numpy()`, because +Fortran does not permit its descriptor form in a `bind(C)` interface. + +PRIK does not wrap `Pointer[String[N][...]]` parameters: a fixed-width +character pointer array has no interoperable pointer descriptor interface that +preserves its association semantics. A fixed-width character module or field +pointer can still expose a NumPy view and satisfy an ordinary +`String[N][...]` array parameter. + --- ## Associate Two Pointers diff --git a/docs/user/guide/wrapping-modules.md b/docs/user/guide/wrapping-modules.md index e82449295..974d63de7 100644 --- a/docs/user/guide/wrapping-modules.md +++ b/docs/user/guide/wrapping-modules.md @@ -114,12 +114,46 @@ print(mod.nmax) # 12 (read-only parameter) ## Module Arrays & Saved State +A fixed-shape module array is exposed as a live NumPy view over the real +Fortran storage. Reads see whatever the native code last wrote, and writes +through the view are visible to Fortran: + +```fortran +module state + use iso_fortran_env, only: real64 + implicit none + real(real64) :: grid(2, 3) +end module state +``` + +```python +grid = mod.grid # a view, not a copy +grid[0, 0] = 10.0 # Fortran sees this +mod.bump() # and this is visible through `grid` +``` + +The whole variable cannot be reassigned (`mod.grid = ...` raises +`AttributeError`); its shape belongs to the Fortran declaration. Write into the +view instead, with `mod.grid[:] = ...`. + +- Fixed-shape module arrays and derived-type array fields expose live views + whether or not their declarations include `target`. +- A `logical(c_bool)` array uses `numpy.bool_`; wider logical kinds use the + matching NumPy integer dtype. See the + [logical type mapping](data-types.md#scalar-type-mapping) for reading and + writing those values. - Allocatable module arrays use the `Allocatable[T[...]]` API. - Allocation, lifetime, NumPy views, and mutation rules are covered in the storage and objects section. - `save` attributes (including procedure-local `save` variables) persist across calls. - Multiple Python imports of the same extension share the same native module state. +!!! warning "A borrowed view assumes module storage stays put" + + A module view borrows the variable's current address. Copy it with + `np.array(mod.grid)` before native work that may relocate that storage. + Declaring the variable `target` gives its address Fortran-defined stability. + --- ## Shape the Module API With the Contract diff --git a/docs/user/language-support/feature-matrix.md b/docs/user/language-support/feature-matrix.md index 4e423566a..23e26f806 100644 --- a/docs/user/language-support/feature-matrix.md +++ b/docs/user/language-support/feature-matrix.md @@ -59,8 +59,8 @@ where they apply. | Defined operators and assignment overloads | Supported | [Defined operators](../guide/wrapping-derived-types.md#defined-operators) | [Defined operator tests](../../../tests/fortran/generic_interfaces/end_to_end/test_defined_operators.py) | Supported operators are those covered by the wrapper guide and runtime tests. | | Output arguments and multiple results | Supported | [Subroutine projection](../guide/wrapping-subroutines.md) | [Calls and results tests](../../../tests/fortran/infrastructure/semantic_pyi/contracts/calls_and_results/end_to_end/test_edited_call_surfaces.py), [function result tests](../../../tests/fortran/functions/end_to_end/test_documented_function_journeys.py) | Tuple ordering and caller-provided array behavior follow the wrapper guide. | | Optional arguments | Supported | [Optional arguments](../guide/optional-arguments.md) | [Optional argument tests](../../../tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py) | Unsupported optional combinations fail during wrapper planning. | -| Allocatable array handles, descriptor arguments, and owned results | Supported | [Allocatables](../guide/allocatables.md) | [Allocatable runtime tests](../../../tests/fortran/allocatables/end_to_end/test_allocatable_handles.py), [scalar-derived matrix tests](../../../tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py) | Array module/field handles borrow their owner; result handles own persistent descriptor storage. Wrapper-owned scalar-derived allocatables use typed holders; module scalar allocatables use reversible `move_alloc` transactions for compatible dummies. | -| Pointer scalar projections and array handles | Partially supported | [Pointers](../guide/pointers.md) | [Pointer handle tests](../../../tests/fortran/pointers/end_to_end/test_pointer_handles.py), [pointer policy tests](../../../tests/fortran/pointers/policy/test_pointer_ownership_policy.py), [scalar-derived matrix tests](../../../tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py) | Descriptor arguments, module/field handles, strided views, wrapper-owned pointer-array results and outputs, scalar-derived pointer holders, and module pointer reassociation transactions are supported. Target deallocation and writable reassociation remain policy-gated. | +| Allocatable array handles, descriptor arguments, and owned results | Supported | [Allocatables](../guide/allocatables.md) | [Allocatable runtime tests](../../../tests/fortran/allocatables/end_to_end/test_allocatable_handles.py), [scalar-derived matrix tests](../../../tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py) | Array module/field handles borrow their owner; result handles own persistent descriptor storage. Fixed-width character descriptor arguments are unsupported; their module/field handles can still satisfy ordinary array arguments. Wrapper-owned scalar-derived allocatables use typed holders; module scalar allocatables use reversible `move_alloc` transactions for compatible dummies. | +| Pointer scalar projections and array handles | Partially supported | [Pointers](../guide/pointers.md) | [Pointer handle tests](../../../tests/fortran/pointers/end_to_end/test_pointer_handles.py), [pointer policy tests](../../../tests/fortran/pointers/policy/test_pointer_ownership_policy.py), [scalar-derived matrix tests](../../../tests/fortran/derived_types/end_to_end/test_scalar_actual_dummy_matrix.py) | Descriptor arguments, module/field handles, strided views, wrapper-owned pointer-array results and outputs, scalar-derived pointer holders, and module pointer reassociation transactions are supported. Fixed- and deferred-width character descriptor arguments are unsupported. Target deallocation and writable reassociation remain policy-gated. | | Array-valued function results | Supported | [Array results](../guide/arrays.md#mutation-and-results) | [Array result tests](../../../tests/fortran/arrays/end_to_end/test_array_results.py) | Ownership and dtype/shape behavior are limited to documented array result forms. | | NumPy array argument contracts | Supported | [Arrays](../guide/arrays.md) | [Array contract tests](../../../tests/fortran/arrays/end_to_end/test_array_contract_validation.py), [multidimensional tests](../../../tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py) | Wrong dtype, rank, shape, contiguity, alignment, or mutability is rejected. | | Derived-type scalar boundaries and methods | Supported | [Derived types](../guide/wrapping-derived-types.md) | [Derived boundary tests](../../../tests/fortran/derived_types/end_to_end/test_derived_boundaries.py), [method tests](../../../tests/fortran/derived_types/end_to_end/test_type_bound_methods.py) | Derived-type arrays and some polymorphic forms are not included. | diff --git a/docs/user/reference/cli-commands.md b/docs/user/reference/cli-commands.md index 1aa466746..2de9be866 100644 --- a/docs/user/reference/cli-commands.md +++ b/docs/user/reference/cli-commands.md @@ -90,6 +90,7 @@ least one explicit native input: `--native-fortran-sources`, `--native-c-sources | `--strict-wrapper-names` | Rejects Python names that would need escaping or a collision suffix. | | `--assume-intent-in-scalars` | Treats a primitive or non-descriptor character scalar dummy that declares no `intent` as `intent(in)`, so its value is not returned. A declared `intent` always wins; arrays, derived-type objects, and descriptor character scalars are unaffected. Also accepted by `generate --pyi`, where it removes the same results from the generated contract, and by `semantics`. | | `--no-compile-input-sources` | Treats positional sources as semantic inputs only. Requires an explicit native input. | +| `--no-standard-logicals` | Omits the compiler option that gives a Fortran `logical` the representation C expects (`-standard-semantics` on Intel, `-Munixlogical` on PGI/NVIDIA). Use only to link prebuilt Intel objects compiled without it, since that option also changes module symbol mangling. | | `--native-fortran-sources PATH ...` | Compiles extra native sources without exposing them as public API. | | `--native-c-sources PATH ...` | Compiles extra C sources without exposing them as public API. | | `--native-compile-flags FLAG ...` | Flags for native implementation compilation. | diff --git a/docs/user/reference/pyi-format.md b/docs/user/reference/pyi-format.md index 956acc089..071f0e403 100644 --- a/docs/user/reference/pyi-format.md +++ b/docs/user/reference/pyi-format.md @@ -684,7 +684,7 @@ contract is generated. | Family | Available names | Normal Python boundary | | --- | --- | --- | -| Boolean | `Bool`, `Bool8`, `Bool16`, `Bool32`, `Bool64` | `bool` or `numpy.bool_`; arrays use `numpy.bool_`. | +| Boolean | `Bool`, `Bool8`, `Bool16`, `Bool32`, `Bool64` | Scalar inputs accept `bool` or `numpy.bool_`; results are `bool`. `Bool` and `Bool8` arrays use `numpy.bool_`; wider arrays use `int16`, `int32`, or `int64`. | | Signed integer | `Int`, `Int8`, `Int16`, `Int32`, `Int64` | Matching NumPy integer scalar or array dtype. `Int` retains target-dependent C `int` identity. | | Unsigned integer | `UInt`, `UInt8`, `UInt16`, `UInt32`, `UInt64`, `SizeT` | Matching NumPy unsigned scalar or array dtype; `UInt` and `SizeT` are target-dependent. | | Real | `Float16`, `Float32`, `Float64`, `Float128` | Matching NumPy real dtype when the selected target supports it. | diff --git a/examples/fortran/lapack/routine_inventory.py b/examples/fortran/lapack/routine_inventory.py index fa5278c1d..4cd99e2a0 100644 --- a/examples/fortran/lapack/routine_inventory.py +++ b/examples/fortran/lapack/routine_inventory.py @@ -443,10 +443,10 @@ "dgges": "NumPy f2py 2.5.1 generates an incomplete selctg callback declaration from the unannotated source", } -PRIK_ABI_ADAPTERS = { - "dtgsen": "GFortran default-LOGICAL selection storage is four bytes per element while PRIK accepts a NumPy bool buffer", - "dtrsen": "GFortran default-LOGICAL selection storage is four bytes per element while PRIK accepts a NumPy bool buffer", -} +# PRIK aliases every array argument at the width its elements really occupy, so +# a default-LOGICAL selection or workspace array crosses as the matching integer +# dtype and no routine here needs a generated representation adapter. +PRIK_ABI_ADAPTERS: dict[str, str] = {} @dataclass(frozen=True) diff --git a/examples/fortran/lapack/tests/test_eigen_generalized.py b/examples/fortran/lapack/tests/test_eigen_generalized.py index 295a6b286..8237bcec3 100644 --- a/examples/fortran/lapack/tests/test_eigen_generalized.py +++ b/examples/fortran/lapack/tests/test_eigen_generalized.py @@ -51,7 +51,8 @@ def test_dgges_computes_generalized_real_schur_form(prik_lapack, scipy_lapack): np.int32(2), np.empty(64), np.int32(64), - np.zeros(2, dtype=np.bool_), + # BWORK is a default-kind LOGICAL array, four bytes to an element. + np.zeros(2, dtype=np.int32), np.int32(0), ) scipy_a, scipy_b, scipy_sdim, scipy_ar, scipy_ai, scipy_beta, scipy_vsl, scipy_vsr, _work, scipy_info = ( @@ -342,8 +343,11 @@ def test_dtgexc_reorders_generalized_schur_blocks(prik_lapack, scipy_lapack, f2p def test_dtgsen_reorders_selected_generalized_eigenvalue(prik_lapack, scipy_lapack, f2py_lapack): a, b = _generalized_problem() identity = np.eye(2, dtype=np.float64, order="F") + # A default-kind Fortran LOGICAL is four bytes wide under gfortran, and an + # array of them is aliased element for element, so every binding here takes + # the matching integer dtype rather than a one-byte numpy.bool_ buffer. selection = np.array([False, True], dtype=np.bool_) - prik_selection = selection.copy() + prik_selection = selection.astype(np.int32) prik_a, f2py_a = a.copy(order="F"), a.copy(order="F") prik_b, f2py_b = b.copy(order="F"), b.copy(order="F") prik_q, f2py_q = identity.copy(order="F"), identity.copy(order="F") diff --git a/examples/fortran/lapack/tests/test_eigen_nonsymmetric.py b/examples/fortran/lapack/tests/test_eigen_nonsymmetric.py index 27f6b44ef..2b12e30df 100644 --- a/examples/fortran/lapack/tests/test_eigen_nonsymmetric.py +++ b/examples/fortran/lapack/tests/test_eigen_nonsymmetric.py @@ -72,7 +72,8 @@ def test_dgees_computes_real_schur_decomposition(prik_lapack, scipy_lapack): np.int32(2), np.empty(64), np.int32(64), - np.zeros(2, dtype=np.bool_), + # BWORK is a default-kind LOGICAL array, four bytes to an element. + np.zeros(2, dtype=np.int32), np.int32(0), ) scipy_t, scipy_sdim, scipy_wr, scipy_wi, scipy_vs, _work, scipy_info = scipy_lapack.dgees( @@ -202,8 +203,11 @@ def test_dtrsen_reorders_selected_schur_eigenvalue(prik_lapack, scipy_lapack, f2 prik_q, f2py_q = identity.copy(order="F"), identity.copy(order="F") prik_wr, prik_wi = np.empty(2), np.empty(2) f2py_wr, f2py_wi = np.empty(2), np.empty(2) + # A default-kind Fortran LOGICAL is four bytes wide under gfortran, and an + # array of them is aliased element for element, so every binding here takes + # the matching integer dtype rather than a one-byte numpy.bool_ buffer. selection = np.array([False, True], dtype=np.bool_) - prik_selection = selection.copy() + prik_selection = selection.astype(np.int32) prik_scalars = prik_lapack.dtrsen( "N", diff --git a/prik/cli.py b/prik/cli.py index beb99cf3a..b93d03b17 100644 --- a/prik/cli.py +++ b/prik/cli.py @@ -1468,6 +1468,7 @@ def record_total_build_time(elapsed: float) -> None: makefile=getattr(args, "makefile", False), generate_sources=getattr(args, "generate_sources", False), jobs=getattr(args, "jobs", None), + standard_logicals=getattr(args, "standard_logicals", True), verbose=1 if getattr(args, "verbose", False) else 0, wrapper_compiler_debug=getattr(args, "wrapper_compiler_debug", False), wrapper_fortran_flags=_with_link_time_optimization( @@ -1517,6 +1518,7 @@ def record_total_build_time(elapsed: float) -> None: wrapper_c_flags=_with_link_time_optimization( _cli_wrapper_c_flags(getattr(args, "wrapper_c_flags", None)), args ), + standard_logicals=getattr(args, "standard_logicals", True), _on_total_build_time=total_build_time_reporter, ) return _copy_wrapper_shared_library_alias(args, result) @@ -1532,6 +1534,7 @@ def record_total_build_time(elapsed: float) -> None: positional_only=getattr(args, "positional_only", False), assume_intent_in_scalars=getattr(args, "assume_intent_in_scalars", False), compile_input_sources=not getattr(args, "no_compile_input_sources", False), + standard_logicals=getattr(args, "standard_logicals", True), native_fortran_sources=getattr(args, "native_fortran_sources", None), native_fortran_flags=_with_link_time_optimization( _cli_native_compile_flags(getattr(args, "native_compile_flags", None)), args @@ -2225,6 +2228,16 @@ def _add_native_compilation_options(group: argparse._ArgumentGroup) -> None: action="store_true", help="Read positional sources without compiling them; require an explicit native implementation", ) + group.add_argument( + "--no-standard-logicals", + dest="standard_logicals", + action="store_false", + help=( + "Omit the compiler option that makes a Fortran logical interoperable with C " + "(-standard-semantics on Intel, -Munixlogical on PGI/NVIDIA); use only to match " + "prebuilt objects already compiled without it" + ), + ) group.add_argument( "--native-fortran-sources", dest="native_fortran_sources", diff --git a/prik/codegen/c/binding.py b/prik/codegen/c/binding.py index 55c01ac5f..98594bc25 100644 --- a/prik/codegen/c/binding.py +++ b/prik/codegen/c/binding.py @@ -23,6 +23,7 @@ ) from prik.policy.models import ( ArgumentHandoffMode, + ArrayEntrypointABI, ArrayPythonLayout, CallbackABIKind, CallbackResultAction, @@ -35,11 +36,15 @@ DerivedWriteback, DirectResultABI, ModuleObjectAccessMechanism, + ModuleArrayAddressMechanism, ModuleGetterAction, + NativeArrayDescriptorAttribute, NativeArrayDescriptorKind, NativeArrayDescriptorInterop, NativeArrayDefaultConstruction, NativeArrayOperation, + NativeArrayOutputProjection, + NativeArraySourceKind, NativeDescriptorHandoffABI, EntrypointProjectionAction, EntrypointPassingConvention, @@ -101,7 +106,6 @@ ModulePlan, ModuleVariablePlan, NamespacePlan, - NativeArrayActualPlan, NativeArrayHandlePlan, NativeEntrypointABIValueKind, NativeEntrypointABIValuePlan, @@ -116,6 +120,14 @@ ) from prik.codegen.primitive_scalar_types import NativeCArrayStorageRegistry, PrimitiveScalarTypeRegistry from prik.codegen.visitor import ClassVisitor +from prik.policy.native_array_handles import NATIVE_ARRAY_POINTER_C_DESCRIPTOR_HEADER + + +def _descriptor_binding_noun(handle: NativeArrayHandlePlan) -> str: + """Name what a callee can change about this descriptor's entity.""" + if handle.descriptor_kind is NativeArrayDescriptorKind.POINTER: + return "association" + return "allocation" @dataclass @@ -155,6 +167,13 @@ class _CFunctionContext: python_result_name: str | None python_results: dict[str, str] role_values: dict[str, str] + # Every argument reached through its descriptor entry point, in call order. + # The call is made inside the innermost consumer, so each one is entered in + # turn and they are all live together by the time it happens. + inverted_descriptors: tuple[str, ...] = () + # The function whose lowering this context serves, so an argument's nodes + # can name helpers emitted once per function at module scope. + function: FunctionPlan | None = None @dataclass(frozen=True) @@ -166,6 +185,20 @@ class _COverloadDispatch: public: bool +# Every inquiry a handle answers is read from the live descriptor its entry +# point supplies, so none of these needs a Fortran procedure of its own. +_DESCRIPTOR_ANSWERED_OPERATIONS = frozenset( + { + NativeArrayOperation.ALLOCATED, + NativeArrayOperation.ASSOCIATED, + NativeArrayOperation.CONTIGUOUS, + NativeArrayOperation.DESCRIPTOR, + NativeArrayOperation.ELEMENT_LENGTH, + NativeArrayOperation.SHAPE, + NativeArrayOperation.TO_NUMPY, + } +) + _BINDING_GETTER_SUMMARIES = { ModuleGetterAction.CONSTANT_VALUE: "The value is a constant placed in the module dictionary at import.", ModuleGetterAction.NATIVE_CONSTANT_VALUE: "Builds a Python object from the compiler-evaluated constant.", @@ -257,6 +290,10 @@ def _require_argument_supported(self, argument: ArgumentTransferPlan) -> None: return self._require_backend_type_supported(argument.semantic_type_name, argument.datatype_family) + # Reading a descriptor needs the Fortran interop header, which a module + # only includes when its plan calls for one. A C-only wrapper never does. + _reads_native_descriptors = False + def _visit_ModulePlan(self, plan: ModulePlan) -> tuple[CModule, CHeader]: """Build the matching C implementation module and public header. @@ -272,6 +309,9 @@ def binding_module(self, plan: ModulePlan) -> CModule: then assembles module support, runtime helpers, wrappers, and module initialization in emitted dependency order. """ + # Reading a descriptor needs the Fortran interop header, and a module + # only includes it when its plan calls for one; a C-only wrapper does not. + self._reads_native_descriptors = NATIVE_ARRAY_POINTER_C_DESCRIPTOR_HEADER in plan.required_headers # Stage 1: index planner-owned cross-language operations before any lowering. self._generated_support_procedure_entrypoints = { (procedure.owner_path, procedure.role): procedure for procedure in plan.entrypoint.support_procedures @@ -300,6 +340,11 @@ def binding_module(self, plan: ModulePlan) -> CModule: declarations=self._module_declarations(plan), functions=( *self._module_allocator_functions(needs_free), + # Every handle inquiry runs through these, so they precede the + # first handle operation that names one. + *self._numpy_descriptor_builder_function(plan), + *self._array_extents_reader_function(plan), + *self._native_array_projection_functions(plan), *self._extent_expression_support_functions(plan), *self._callback_runtime_functions(plan), *self._derived_call_runtime_functions(plan), @@ -640,8 +685,30 @@ def _module_defines(self, plan: ModulePlan, needs_native_support: bool) -> tuple for argument in function.arguments ): definitions.append(CMacroDefinition("PRIK_BINDING_NATIVE_ARRAY_ACTUAL", "1")) + # The bundled address-capture primitive needs external linkage for the + # Fortran bridge to call it, so the header defines it only where this + # macro opts in. Selecting it here keeps it in one translation unit. + if self._requires_address_capture(plan): + definitions.append(CMacroDefinition("PRIK_BINDING_CAPTURE_ADDRESS", "1")) return tuple(definitions) + def _requires_address_capture(self, plan: ModulePlan) -> bool: + """Report whether any borrowed view in this module takes its address in C. + + Both cases name their storage directly rather than reaching it through a + pointer, so neither has a Fortran route to its own address: a module + array whose declaration withheld ``target``, and an array member of a + plain module object, which is likewise not a target. + """ + return any( + variable.array_address is ModuleArrayAddressMechanism.CAPTURED_ADDRESS for variable in self._variables(plan) + ) or any( + member.field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR + for variable in self._variables(plan) + if variable.derived is not None and variable.derived.access is ModuleObjectAccessMechanism.MEMBER_PROXY + for member in variable.derived.member_paths + ) + def _module_includes( self, plan: ModulePlan, @@ -2701,12 +2768,7 @@ def _direct_handle_field_functions(self, derived, field) -> tuple[CFunction, ... def _direct_array_field_functions(self, derived, field) -> tuple[CFunction, ...]: """Build direct array field functions from the supplied completed binding records; emitted nodes only project completed binding actions.""" - callback = self._ordinary_array_field_descriptor_callback( - field, - self._derived_field_descriptor_callback_name(derived, field), - ) return ( - callback, self._direct_ordinary_array_field_getter(derived, field), *self._present_field_function(self._direct_ordinary_array_field_setter(derived, field)), ) @@ -2738,12 +2800,7 @@ def _module_handle_member_functions(self, variable, member) -> tuple[CFunction, def _module_array_member_functions(self, variable, member) -> tuple[CFunction, ...]: """Build module array member functions from the supplied completed binding records; emitted nodes only project completed binding actions.""" - callback = self._ordinary_array_field_descriptor_callback( - member.field, - self._module_member_descriptor_callback_name(variable, member), - ) return ( - callback, self._module_ordinary_array_member_getter(variable, member), *self._present_field_function(self._module_ordinary_array_member_setter(variable, member)), ) @@ -2780,17 +2837,70 @@ def _direct_ordinary_array_field_getter( """Create a live NumPy view over one fixed address-backed field.""" body = ( *self._derived_owner_address_nodes(derived), - CDeclaration("field_view", "PyObject *", CodeExpression("NULL")), - CExpressionStatement( - CodeExpression( - f"{self._derived_field_bridge_name(derived, field, 'get')}(owner_address, " - f"{self._derived_field_descriptor_callback_name(derived, field)}, &field_view)" - ) + *self._borrowed_array_view_nodes( + field, + self._derived_field_bridge_name(derived, field, "get"), + owner="owner_obj", + leading_arguments=("owner_address",), ), - *self._ordinary_array_field_owner_nodes("field_view", "owner_obj"), ) return self._derived_private_method(self._derived_field_method_name(derived, field, "get"), body) + def _borrowed_array_view_nodes( + self, + field: DerivedFieldPlan, + bridge_name: str, + *, + owner: str, + leading_arguments: tuple[str, ...] = (), + ) -> tuple: + """Build one live Fortran-ordered NumPy alias from a base pointer and extents. + + Both field families share this construction: the bridge reports where + the member lives plus one extent per axis, and the view is formed here. + ``leading_arguments`` carries the owner address where one is passed. + """ + array = field.array + if array is None or array.rank is None: + raise ValueError(f"Ordinary array field {field.owner_path!r} has no fixed rank") + scalar = PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name) + extents = tuple(f"extent_{axis}" for axis in range(array.rank)) + arguments = ", ".join((*leading_arguments, *(f"&{name}" for name in extents))) + return ( + *(CDeclaration(name, "int64_t", CodeExpression("0")) for name in extents), + CDeclaration("field_data", "void *", CodeExpression(f"{bridge_name}({arguments})")), + CIf( + CodeExpression("field_data == NULL"), + body=( + CExpressionStatement( + CodeExpression('PyErr_SetString(PyExc_ReferenceError, "array field storage is unavailable")') + ), + CReturn(CodeExpression("NULL")), + ), + ), + CDeclaration( + f"dimensions[{array.rank}]", + "npy_intp", + CodeExpression("{" + ", ".join(extents) + "}"), + ), + CDeclaration(f"strides[{array.rank}]", "npy_intp"), + CExpressionStatement(CodeExpression(f"strides[0] = (npy_intp)sizeof({scalar.array_c_spelling})")), + *( + CExpressionStatement(CodeExpression(f"strides[{axis}] = strides[{axis - 1}] * dimensions[{axis - 1}]")) + for axis in range(1, array.rank) + ), + CDeclaration( + "field_view", + "PyObject *", + CodeExpression( + f"PyArray_New(&PyArray_Type, {array.rank}, dimensions, {scalar.array_numpy_type}, " + "strides, field_data, 0, NPY_ARRAY_F_CONTIGUOUS | NPY_ARRAY_ALIGNED | " + "NPY_ARRAY_WRITEABLE, NULL)" + ), + ), + *self._ordinary_array_field_owner_nodes("field_view", owner), + ) + def _direct_native_handle_field_getter( self, derived: DerivedTypePlan, @@ -2870,14 +2980,11 @@ def _module_ordinary_array_member_getter( body = ( CDeclaration("owner_obj", "PyObject *"), CExpressionStatement(CodeExpression('if (!PyArg_ParseTuple(args, "O", &owner_obj)) return NULL')), - CDeclaration("field_view", "PyObject *", CodeExpression("NULL")), - CExpressionStatement( - CodeExpression( - f"{self._module_member_bridge_name(variable, member, 'get')}(" - f"{self._module_member_descriptor_callback_name(variable, member)}, &field_view)" - ) + *self._borrowed_array_view_nodes( + member.field, + self._module_member_bridge_name(variable, member, "get"), + owner="owner_obj", ), - *self._ordinary_array_field_owner_nodes("field_view", "owner_obj"), ) return self._derived_private_method(self._module_member_method_name(variable, member, "get"), body) @@ -2974,50 +3081,103 @@ def _fixed_string_field_input_nodes(self, field: DerivedFieldPlan, object_name: ), ) + def _field_handle_backend_release_nodes(self, field: DerivedFieldPlan, prefix: str) -> tuple: + """Release the reference the published backend capsule was created with.""" + if self._field_handle_backend_capsule_name(field, prefix) == "Py_None": + return () + return (CExpressionStatement(CodeExpression(f"Py_XDECREF({prefix}_native_backend)")),) + + def _field_handle_backend_capsule_name(self, field: DerivedFieldPlan, prefix: str) -> str: + """Return the local holding this field handle's published backend.""" + handle = field.native_array_handle + if ( + handle is None + or not handle.descriptor_inquiries + or handle.handoff.abi is not NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + ): + return "Py_None" + return f"{prefix}_native_backend" + + def _field_handle_backend_capsule_nodes( + self, owner, field: DerivedFieldPlan, prefix: str, owner_name: str + ) -> tuple: + """Build the descriptor backend this field handle publishes. + + A derived-type field reaches its entity through the parent's address, + so that address is resolved once here rather than on every operation. + A module member needs none. + """ + handle = field.native_array_handle + if handle is None or handle.array.rank is None: + return () + if ( + not handle.descriptor_inquiries + or handle.handoff.abi is not NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + ): + return () + cfi_type = self._field_native_array_cfi_type(field) + if cfi_type is None: + return () + parent = f"{prefix}_parent" + address = f"{parent}_address" if isinstance(owner, DerivedTypePlan) else "NULL" + forward = self._field_handle_with_descriptor_name(self._field_handle_descriptor_callback(owner, field)) + capsule = f"{prefix}_native_backend" + return ( + *( + self._derived_address_from_object_nodes(owner.backend_symbol, owner_name, parent) + if isinstance(owner, DerivedTypePlan) + else () + ), + CDeclaration( + capsule, + "PyObject *", + CodeExpression( + "prik_native_array_backend_capsule_new(" + f"{self._native_array_handle_kind_constant(handle)}, " + f"{self._native_array_descriptor_attribute_constant(handle)}, {handle.array.rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({handle.array.rank})), {cfi_type}, " + f"{self._field_native_array_element_size(field)}, {address}, {forward}, NULL)" + ), + ), + CIf(CodeExpression(f"{capsule} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + ) + def _field_handle_factory_nodes(self, owner, field: DerivedFieldPlan, owner_name: str) -> tuple: - """Build a fresh borrowed handle whose operations are bound to its parent.""" + """Build a fresh borrowed handle whose dispatcher is bound to its parent.""" handle = field.native_array_handle if handle is None or handle.array.rank is None: raise ValueError(f"Native handle field {field.owner_path!r} has no factory plan") prefix = re.sub(r"\W", "_", field.owner_path).casefold() - ops = f"{prefix}_ops" - operation_object = f"{prefix}_operation" + capabilities = f"{prefix}_capabilities" + invoke = f"{prefix}_invoke" runtime = f"{prefix}_runtime" helper = f"{prefix}_helper" result = f"{prefix}_handle" + dispatch = self._field_handle_dispatch_name(owner, field) nodes = [ - CDeclaration(ops, "PyObject *", CodeExpression("PyDict_New()")), - CDeclaration(operation_object, "PyObject *", CodeExpression("NULL")), + *self._field_handle_backend_capsule_nodes(owner, field, prefix, owner_name), + CDeclaration(capabilities, "PyObject *", CodeExpression("NULL")), + CDeclaration(invoke, "PyObject *", CodeExpression("NULL")), CDeclaration(runtime, "PyObject *", CodeExpression("NULL")), CDeclaration(helper, "PyObject *", CodeExpression("NULL")), CDeclaration(result, "PyObject *", CodeExpression("NULL")), - CIf(CodeExpression(f"{ops} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CExpressionStatement( + CodeExpression(f"{capabilities} = {self._native_array_capabilities_expression(handle.operations)}") + ), + CIf( + CodeExpression(f"{capabilities} == NULL"), + body=(*self._field_handle_backend_release_nodes(field, prefix), CReturn(CodeExpression("NULL"))), + ), + CExpressionStatement(CodeExpression(f"{invoke} = PyCFunction_NewEx(&{dispatch}_def, {owner_name}, NULL)")), + CIf( + CodeExpression(f"{invoke} == NULL"), + body=( + CExpressionStatement(CodeExpression(f"Py_DECREF({capabilities})")), + *self._field_handle_backend_release_nodes(field, prefix), + CReturn(CodeExpression("NULL")), + ), + ), ] - for operation in handle.operations: - name = self._field_handle_operation_name(owner, field, operation) - nodes.extend( - ( - CExpressionStatement( - CodeExpression(f"{operation_object} = PyCFunction_NewEx(&{name}_def, {owner_name}, NULL)") - ), - CIf( - CodeExpression(f"{operation_object} == NULL"), - body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), - CReturn(CodeExpression("NULL")), - ), - ), - CIf( - CodeExpression(f'PyDict_SetItemString({ops}, "{operation.value}", {operation_object}) < 0'), - body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({operation_object})")), - CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), - CReturn(CodeExpression("NULL")), - ), - ), - CExpressionStatement(CodeExpression(f"Py_DECREF({operation_object})")), - ) - ) family = DatatypeFamily.STRING if field.string_element else DatatypeFamily.REAL nodes.extend( ( @@ -3025,20 +3185,24 @@ def _field_handle_factory_nodes(self, owner, field: DerivedFieldPlan, owner_name CIf( CodeExpression(f"{runtime} == NULL"), body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({invoke})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({capabilities})")), + *self._field_handle_backend_release_nodes(field, prefix), CReturn(CodeExpression("NULL")), ), ), CExpressionStatement( CodeExpression( - f'{helper} = PyObject_GetAttrString({runtime}, "_native_array_handle_from_generated_ops")' + f'{helper} = PyObject_GetAttrString({runtime}, "_native_array_handle_from_generated_dispatch")' ) ), CExpressionStatement(CodeExpression(f"Py_DECREF({runtime})")), CIf( CodeExpression(f"{helper} == NULL"), body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({invoke})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({capabilities})")), + *self._field_handle_backend_release_nodes(field, prefix), CReturn(CodeExpression("NULL")), ), ), @@ -3051,15 +3215,19 @@ def _field_handle_factory_nodes(self, owner, field: DerivedFieldPlan, owner_name semantic_type_name=field.semantic_type_name, datatype_family=family, rank=handle.array.rank, - ops=ops, + invoke=invoke, + capabilities=capabilities, owner=owner_name, descriptor_ownership="borrowed", + native_backend=self._field_handle_backend_capsule_name(field, prefix), extraction_action=handle.extraction_action.value, ) ) ), CExpressionStatement(CodeExpression(f"Py_DECREF({helper})")), - CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({invoke})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({capabilities})")), + *self._field_handle_backend_release_nodes(field, prefix), CReturn(CodeExpression(result)), ) ) @@ -3077,6 +3245,10 @@ def _field_handle_operation_name( variable, member = owner return self._module_member_handle_operation_name(variable, member, operation) + def _field_handle_dispatch_name(self, owner, field: DerivedFieldPlan) -> str: + """Return the single Python dispatcher name for one field handle.""" + return f"{self._field_handle_operation_name(owner, field, NativeArrayOperation.SHAPE)}_dispatch" + def _module_ordinary_array_member_setter( self, variable: ModuleVariablePlan, @@ -3100,56 +3272,6 @@ def _module_ordinary_array_member_setter( ) return self._derived_private_method(self._module_member_method_name(variable, member, "set"), body) - def _ordinary_array_field_descriptor_callback( - self, - field: DerivedFieldPlan, - callback_name: str, - ) -> CFunction: - """Construct a NumPy view from one standard field descriptor.""" - array = field.array - if array is None or array.rank is None or not array.shape: - raise ValueError(f"Ordinary array field {field.owner_path!r} has no fixed shape") - scalar = PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name) - dims = ", ".join(f"(npy_intp)descriptor->dim[{axis}].extent" for axis in range(array.rank)) - strides = ", ".join(f"(npy_intp)descriptor->dim[{axis}].sm" for axis in range(array.rank)) - return CFunction( - callback_name, - "void", - parameters=(CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), - storage="static", - body=( - CExpressionStatement(CodeExpression("*(PyObject **)context = NULL")), - CIf( - CodeExpression("descriptor == NULL || descriptor->base_addr == NULL"), - body=( - CExpressionStatement( - CodeExpression( - 'PyErr_SetString(PyExc_ReferenceError, "array field descriptor is unavailable")' - ) - ), - CReturn(), - ), - ), - CDeclaration( - f"field_dims[{array.rank}]", - "npy_intp", - CodeExpression("{" + dims + "}"), - ), - CDeclaration( - f"field_strides[{array.rank}]", - "npy_intp", - CodeExpression("{" + strides + "}"), - ), - CExpressionStatement( - CodeExpression( - f"*(PyObject **)context = PyArray_New(&PyArray_Type, {array.rank}, field_dims, " - f"{scalar.numpy_type_macro}, field_strides, descriptor->base_addr, 0, " - "NPY_ARRAY_F_CONTIGUOUS | NPY_ARRAY_ALIGNED | NPY_ARRAY_WRITEABLE, NULL)" - ) - ), - ), - ) - @staticmethod def _ordinary_array_field_owner_nodes(field_view: str, owner_name: str) -> tuple: """Retain the live parent as the NumPy view base after descriptor use.""" @@ -3550,64 +3672,97 @@ def _derived_field_c_type(self, field: DerivedFieldPlan) -> str: return PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).c_spelling # Derived native-array-handle fields reuse the Phase 7 runtime protocol. + def _field_handle_with_descriptor_prototypes( + self, + field: DerivedFieldPlan, + descriptor_callback: str, + ) -> tuple[CFunctionPrototype, ...]: + """Declare the forwarder driving one field's descriptor bridge.""" + handle = field.native_array_handle + if ( + handle is None + or not handle.descriptor_inquiries + or handle.handoff.abi is not NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + ): + return () + return ( + CFunctionPrototype( + self._field_handle_with_descriptor_name(descriptor_callback), + "void", + ( + CParameter("context", "void *"), + CParameter("consumer", "prik_native_array_descriptor_fn"), + CParameter("consumer_context", "void *"), + ), + storage="static", + ), + ) + def _derived_handle_operation_declarations( self, plan: ModulePlan, ) -> tuple[CFunctionPrototype | CDeclaration, ...]: - """Declare every parent-bound field-handle callable and method record.""" + """Declare one parent-bound dispatcher per field handle.""" declarations = [] - for _owner, field, operation_name, callback_names in self._derived_handle_targets(plan): - descriptor_callback, actual_callback = callback_names + for _owner, field, operation_name, descriptor_callback in self._derived_handle_targets(plan): + # The getter that publishes this field's backend is emitted before the + # forwarder it names, so the forwarder is declared here. + declarations.extend(self._field_handle_with_descriptor_prototypes(field, descriptor_callback)) + handle = field.native_array_handle + if handle is None: + continue + name = self._native_array_dispatch_name(operation_name) declarations.extend( ( CFunctionPrototype( - descriptor_callback, - "void", - (CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), + name, + "PyObject *", + (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), storage="static", ), - CFunctionPrototype( - actual_callback, - "void", - (CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), - storage="static", + CDeclaration( + f"{name}_def", + "static PyMethodDef", + CodeExpression(f'{{"{name}", (PyCFunction){name}, METH_VARARGS, ""}}'), ), ) ) - handle = field.native_array_handle - if handle is None: - continue - for operation in handle.operations: - name = operation_name(operation) - declarations.extend( - ( - CFunctionPrototype( - name, - "PyObject *", - (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), - storage="static", - ), - CDeclaration( - f"{name}_def", - "static PyMethodDef", - CodeExpression(f'{{"{name}", (PyCFunction){name}, METH_VARARGS, ""}}'), - ), - ) - ) return tuple(declarations) def _derived_handle_operation_functions(self, plan: ModulePlan) -> tuple[CFunction, ...]: """Lower descriptor callbacks and parent-bound runtime operations.""" functions = [] - for owner, field, operation_name, callback_names in self._derived_handle_targets(plan): - descriptor_callback, actual_callback = callback_names - functions.extend(self._field_handle_descriptor_callbacks(field, descriptor_callback, actual_callback)) + if self._emits_native_array_backend(plan): + # The shared consumer is defined once with the module-array + # section, which follows these forwarders in the emitted file. + functions.append( + CFunctionPrototype( + "prik_native_array_forward_descriptor", + "void", + (CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), + storage="static", + ) + ) + for owner, field, operation_name, descriptor_callback in self._derived_handle_targets(plan): handle = field.native_array_handle + if handle is not None and handle.descriptor_inquiries: + functions.extend( + self._field_handle_backend_nodes( + field, + self._field_handle_bridge_name(owner, field, NativeArrayOperation.DESCRIPTOR), + self._field_handle_with_descriptor_name(descriptor_callback), + takes_owner=isinstance(owner, DerivedTypePlan), + ) + ) if handle is None: continue - functions.extend( - self._field_handle_operation_function(owner, field, operation, operation_name(operation)) - for operation in handle.operations + functions.append( + self._field_handle_dispatch_function( + owner, + field, + handle.operations, + self._native_array_dispatch_name(operation_name), + ) ) return tuple(functions) @@ -3620,10 +3775,7 @@ def _derived_handle_targets(self, plan: ModulePlan) -> tuple[tuple, ...]: lambda operation, derived=derived, field=field: self._derived_handle_operation_name( derived, field, operation ), - ( - self._derived_handle_descriptor_callback_name(derived, field), - self._derived_handle_actual_callback_name(derived, field), - ), + self._derived_handle_descriptor_callback_name(derived, field), ) for derived in self._derived_types(plan) for field in derived.fields @@ -3636,10 +3788,7 @@ def _derived_handle_targets(self, plan: ModulePlan) -> tuple[tuple, ...]: lambda operation, variable=variable, member=member: self._module_member_handle_operation_name( variable, member, operation ), - ( - self._module_member_handle_descriptor_callback_name(variable, member), - self._module_member_handle_actual_callback_name(variable, member), - ), + self._module_member_handle_descriptor_callback_name(variable, member), ) for variable in self._derived_member_proxy_variables(plan) for member in variable.derived.member_paths @@ -3647,124 +3796,231 @@ def _derived_handle_targets(self, plan: ModulePlan) -> tuple[tuple, ...]: ) return tuple(targets) - def _field_handle_descriptor_callbacks( + @staticmethod + def _field_handle_with_descriptor_name(descriptor_callback: str) -> str: + """Return the forwarder name that drives one field's descriptor bridge.""" + return f"{descriptor_callback}_with_descriptor" + + def _field_handle_backend_nodes( self, field: DerivedFieldPlan, - descriptor_name: str, - actual_name: str, - ) -> tuple[CFunction, CFunction]: - """Decode one current field descriptor without copying its payload.""" + descriptor_bridge: str, + forward_name: str, + *, + takes_owner: bool, + ) -> tuple[CFunction, ...]: + """Emit the forwarder driving one field's descriptor bridge. + + A field reaches its entity through its parent's address, so unlike a + module variable the backend cannot be a file-scope constant: the owner + differs per handle and is filled in when the handle is built. + """ handle = field.native_array_handle if handle is None or handle.array.rank is None: - raise ValueError(f"Native handle field {field.owner_path!r} has no descriptor rank") - descriptor = CFunction( - descriptor_name, - "void", - parameters=(CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), - storage="static", - body=( - CExpressionStatement(CodeExpression("*(PyObject **)context = NULL")), - *self._native_array_descriptor_record_nodes( - handle.array.rank, - "descriptor", - return_target="*(PyObject **)context", + return () + if ( + not handle.descriptor_inquiries + or handle.handoff.abi is not NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + ): + return () + return ( + CFunction( + forward_name, + "void", + parameters=( + CParameter("context", "void *"), + CParameter("consumer", "prik_native_array_descriptor_fn"), + CParameter("consumer_context", "void *"), + ), + storage="static", + body=( + CDeclaration( + "forwarded", + "prik_native_array_descriptor_forward", + CodeExpression("{consumer, consumer_context}"), + ), + # A module member reaches its field without a parent address. + *(() if takes_owner else (CExpressionStatement(CodeExpression("(void)context")),)), + CExpressionStatement( + CodeExpression( + f"{descriptor_bridge}({'context, ' if takes_owner else ''}" + "prik_native_array_forward_descriptor, &forwarded)" + ) + ), + CReturn(), ), ), ) - actual = CFunction( - actual_name, - "void", - parameters=(CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), - storage="static", - body=( - CExpressionStatement(CodeExpression("*(void **)context = descriptor->base_addr")), - CReturn(), - ), - ) - return descriptor, actual - def _field_handle_operation_function( + def _field_handle_dispatch_function( self, owner, field: DerivedFieldPlan, - operation: NativeArrayOperation, + operations: tuple[NativeArrayOperation, ...], name: str, ) -> CFunction: - """Lower one live field-handle operation selected by completed policy.""" + """Lower one live field-handle dispatcher selected by completed policy.""" return CFunction( name, "PyObject *", parameters=(CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), storage="static", - body=self._field_handle_operation_body(owner, field, operation), + body=self._native_array_dispatch_body( + operations, + lambda operation: self._field_handle_operation_body(owner, field, operation), + ), ) - def _field_handle_operation_body(self, owner, field: DerivedFieldPlan, operation: NativeArrayOperation) -> tuple: - """Dispatch one operation without inferring descriptor ownership.""" - if operation in { - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, - }: - return self._module_native_array_metadata_body(operation) - prefix = self._field_handle_owner_nodes(owner) - owner_args = self._field_handle_owner_arguments(owner) - if operation in {NativeArrayOperation.DESCRIPTOR, NativeArrayOperation.TO_NUMPY}: - callback = self._field_handle_descriptor_callback(owner, field) - descriptor_bridge = self._field_handle_bridge_name( - owner, - field, - NativeArrayOperation.DESCRIPTOR, - ) - return (*prefix, *self._field_handle_descriptor_nodes(descriptor_bridge, owner_args, callback)) - if operation is NativeArrayOperation.ARRAY_ACTUAL: - callback = self._field_handle_actual_callback(owner, field) - descriptor_bridge = self._field_handle_bridge_name( - owner, - field, - NativeArrayOperation.DESCRIPTOR, - ) - return (*prefix, *self._field_handle_actual_nodes(descriptor_bridge, owner_args, callback)) - bridge = self._field_handle_bridge_name(owner, field, operation) - if operation in { - NativeArrayOperation.ALLOCATED, - NativeArrayOperation.ASSOCIATED, - NativeArrayOperation.CONTIGUOUS, - }: - return (*prefix, CReturn(CodeExpression(f"PyBool_FromLong({bridge}({owner_args}))"))) - if operation is NativeArrayOperation.ELEMENT_LENGTH: - return (*prefix, CReturn(CodeExpression(f"PyLong_FromLongLong((long long){bridge}({owner_args}))"))) - if operation is NativeArrayOperation.SHAPE: - return (*prefix, *self._field_handle_shape_nodes(field, bridge, owner_args)) - if operation is NativeArrayOperation.ASSOCIATE: - return self._field_handle_associate_body(field, prefix, bridge, owner_args) - if operation in {NativeArrayOperation.ALLOCATE, NativeArrayOperation.RESIZE}: - return (*prefix, *self._field_handle_shape_mutation_nodes(field, bridge, owner_args)) - if operation in {NativeArrayOperation.DEALLOCATE, NativeArrayOperation.NULLIFY}: - arguments = owner_args - return ( - *prefix, - CExpressionStatement(CodeExpression(f"{bridge}({arguments})")), - CExpressionStatement(CodeExpression("Py_RETURN_NONE")), + def _native_array_dispatch_body(self, operations, operation_body) -> tuple: + """Dispatch one generated callable through its completed capability set.""" + return ( + CDeclaration("operation_object", "PyObject *", CodeExpression("NULL")), + CDeclaration("operation_args", "PyObject *", CodeExpression("NULL")), + CDeclaration("operation", "const char *", CodeExpression("NULL")), + CExpressionStatement( + CodeExpression( + 'if (!PyArg_ParseTuple(args, "OO!:native array dispatcher", &operation_object, ' + "&PyTuple_Type, &operation_args)) return NULL" + ) + ), + CIf( + CodeExpression("!PyUnicode_Check(operation_object)"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_TypeError, "native array operation name must be a string")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + CExpressionStatement(CodeExpression("operation = PyUnicode_AsUTF8(operation_object)")), + CIf(CodeExpression("operation == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CExpressionStatement(CodeExpression("args = operation_args")), + *( + CIf( + CodeExpression(f'strcmp(operation, "{item.value}") == 0'), + body=operation_body(item), + ) + for item in operations + ), + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_NotImplementedError, "native array operation %R is not available", ' + "operation_object)" + ) + ), + CReturn(CodeExpression("NULL")), + ) + + @staticmethod + def _native_array_dispatch_name(operation_name) -> str: + """Name the single dispatcher replacing one handle's operation methods.""" + return f"{operation_name(NativeArrayOperation.SHAPE)}_dispatch" + + def _field_handle_operation_body(self, owner, field: DerivedFieldPlan, operation: NativeArrayOperation) -> tuple: + """Dispatch one operation without inferring descriptor ownership.""" + prefix = self._field_handle_owner_nodes(owner) + owner_args = self._field_handle_owner_arguments(owner) + handle = field.native_array_handle + if handle is not None and operation in _DESCRIPTOR_ANSWERED_OPERATIONS: + if handle.descriptor_inquiries: + return (*prefix, *self._field_handle_inquiry_nodes(owner, field, operation, owner_args)) + return ( + *prefix, + *self._native_array_bridge_inquiry_nodes( + operation, + f"{self._field_handle_bridge_name(owner, field, operation)}({owner_args})", + handle.array.rank, + self._field_handle_bridge_name(owner, field, NativeArrayOperation.SHAPE), + owner_args, + ), + ) + bridge = self._field_handle_bridge_name(owner, field, operation) + if operation is NativeArrayOperation.ASSOCIATE: + return self._field_handle_associate_body( + field, + prefix, + bridge, + self._field_handle_bridge_name(owner, field, NativeArrayOperation.NULLIFY), + owner_args, + ) + if operation in {NativeArrayOperation.ALLOCATE, NativeArrayOperation.RESIZE}: + return (*prefix, *self._field_handle_shape_mutation_nodes(field, bridge, owner_args)) + if operation in {NativeArrayOperation.DEALLOCATE, NativeArrayOperation.NULLIFY}: + arguments = owner_args + return ( + *prefix, + CExpressionStatement(CodeExpression(f"{bridge}({arguments})")), + CExpressionStatement(CodeExpression("Py_RETURN_NONE")), ) raise ValueError(f"Unsupported field handle operation for {field.owner_path!r}: {operation!r}") + def _field_handle_inquiry_nodes( + self, + owner, + field: DerivedFieldPlan, + operation: NativeArrayOperation, + owner_args: str, + ) -> tuple: + """Answer one inquiry from the descriptor this field exposes. + + A view over a field's storage must keep the parent object alive, and + the parent is exactly the object this operation was bound to, so it + becomes the view's base. + """ + handle = field.native_array_handle + if handle is None or handle.array.rank is None: + raise ValueError(f"Native handle field {field.owner_path!r} has no descriptor rank") + return self._native_array_projection_call_nodes( + operation, + rank=handle.array.rank, + numpy_type=self._field_native_array_numpy_type(field), + element_size=self._field_native_array_element_size(field), + descriptor_attribute=handle.descriptor_attribute, + base="self", + entry_point=self._field_handle_with_descriptor_name(self._field_handle_descriptor_callback(owner, field)), + context=owner_args or "NULL", + ) + + def _field_native_array_numpy_type(self, field: DerivedFieldPlan) -> str: + """Return the NumPy element type one field handle's view is built with.""" + if field.string_element: + return "NPY_STRING" + return PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).array_numpy_type + def _field_handle_associate_body( self, field: DerivedFieldPlan, prefix: tuple, bridge: str, + nullify_bridge: str, owner_args: str, ) -> tuple: """Associate one field pointer through its selected bridge operation.""" arguments = f"{owner_args}, source_descriptor" if owner_args else "source_descriptor" + nullify_arguments = owner_args + handle = field.native_array_handle + if handle is None: + raise ValueError(f"Pointer field {field.owner_path!r} has no descriptor policy") + association = ( + CIf( + CodeExpression("source_base_addr == NULL"), + body=(CExpressionStatement(CodeExpression(f"{nullify_bridge}({nullify_arguments})")),), + else_body=(CExpressionStatement(CodeExpression(f"{bridge}({arguments})")),), + ) + if handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER + else CExpressionStatement(CodeExpression(f"{bridge}({arguments})")) + ) return ( *prefix, CDeclaration("source_packed", "PyObject *"), CExpressionStatement(CodeExpression('if (!PyArg_ParseTuple(args, "O", &source_packed)) return NULL')), - *self._pointer_association_source_nodes(field), - CExpressionStatement(CodeExpression(f"{bridge}({arguments})")), + *self._pointer_association_source_nodes( + field, + establish_if_present_only=handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER, + ), + association, CExpressionStatement(CodeExpression("Py_RETURN_NONE")), ) @@ -3792,64 +4048,12 @@ def _field_handle_bridge_name( return self._module_member_handle_bridge_name(variable, member, operation) def _field_handle_descriptor_callback(self, owner, field: DerivedFieldPlan) -> str: - """Build field handle descriptor callback from the supplied completed binding records; emitted nodes only project completed binding actions.""" + """Return the consumer decoding one field descriptor into facts.""" if isinstance(owner, DerivedTypePlan): return self._derived_handle_descriptor_callback_name(owner, field) variable, member = owner return self._module_member_handle_descriptor_callback_name(variable, member) - def _field_handle_actual_callback(self, owner, field: DerivedFieldPlan) -> str: - """Build field handle actual callback from the supplied completed binding records; emitted nodes only project completed binding actions.""" - if isinstance(owner, DerivedTypePlan): - return self._derived_handle_actual_callback_name(owner, field) - variable, member = owner - return self._module_member_handle_actual_callback_name(variable, member) - - def _field_handle_shape_nodes(self, field: DerivedFieldPlan, bridge: str, owner_args: str) -> tuple: - """Build field handle shape nodes from the supplied completed binding records; emitted nodes only project completed binding actions.""" - handle = field.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Native handle field {field.owner_path!r} has no shape rank") - extents = tuple(f"extent_{axis}" for axis in range(handle.array.rank)) - call_args = ", ".join((*((owner_args,) if owner_args else ()), *(f"&{item}" for item in extents))) - return ( - *(CDeclaration(item, "int64_t", CodeExpression("0")) for item in extents), - CExpressionStatement(CodeExpression(f"{bridge}({call_args})")), - CDeclaration("shape", "PyObject *", CodeExpression(f"PyTuple_New({handle.array.rank})")), - CIf(CodeExpression("shape == NULL"), body=(CReturn(CodeExpression("NULL")),)), - *( - CExpressionStatement( - CodeExpression(f"PyTuple_SET_ITEM(shape, {axis}, PyLong_FromLongLong((long long){extent}))") - ) - for axis, extent in enumerate(extents) - ), - CIf( - CodeExpression("PyErr_Occurred()"), - body=(CExpressionStatement(CodeExpression("Py_DECREF(shape)")), CReturn(CodeExpression("NULL"))), - ), - CReturn(CodeExpression("shape")), - ) - - @staticmethod - def _field_handle_descriptor_nodes(bridge: str, owner_args: str, callback: str) -> tuple: - """Build field handle descriptor nodes from the supplied local lowering values; emitted nodes only project completed binding actions.""" - arguments = ", ".join((*((owner_args,) if owner_args else ()), callback, "&descriptor_record")) - return ( - CDeclaration("descriptor_record", "PyObject *", CodeExpression("NULL")), - CExpressionStatement(CodeExpression(f"{bridge}({arguments})")), - CReturn(CodeExpression("descriptor_record")), - ) - - @staticmethod - def _field_handle_actual_nodes(bridge: str, owner_args: str, callback: str) -> tuple: - """Build field handle actual nodes from the supplied local lowering values; emitted nodes only project completed binding actions.""" - arguments = ", ".join((*((owner_args,) if owner_args else ()), callback, "&base_addr")) - return ( - CDeclaration("base_addr", "void *", CodeExpression("NULL")), - CExpressionStatement(CodeExpression(f"{bridge}({arguments})")), - CReturn(CodeExpression("PyLong_FromVoidPtr(base_addr)")), - ) - def _field_handle_shape_mutation_nodes(self, field: DerivedFieldPlan, bridge: str, owner_args: str) -> tuple: """Build field handle shape mutation nodes from the supplied completed binding records; emitted nodes only project completed binding actions.""" handle = field.native_array_handle @@ -3901,12 +4105,12 @@ def _module_allocator_functions(self, required: bool) -> tuple[CFunction, ...]: ), ) - # Owned native-array-handle operation tables. + # Native-array-handle Python dispatch. def _native_array_operation_declarations( self, plan: ModulePlan, ) -> tuple[CFunctionPrototype | CDeclaration, ...]: - """Declare private operation wrappers and their callable definitions.""" + """Declare one private dispatcher per generated handle source.""" declarations = [] for variable in self._module_array_owner_variables(plan): if variable.binding.getter_action is ModuleGetterAction.NATIVE_ARRAY_HANDLE: @@ -3926,41 +4130,13 @@ def _native_array_operation_declarations( ) if variable.native_array_handle is None: continue - for operation in variable.native_array_handle.operations: - name = self._module_native_array_operation_name(variable, operation) - declarations.extend( - ( - CFunctionPrototype( - name, - "PyObject *", - (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), - storage="static", - ), - CDeclaration( - self._module_native_array_operation_def_name(variable, operation), - "static PyMethodDef", - CodeExpression(f'{{"{name}", (PyCFunction){name}, METH_VARARGS, ""}}'), - ), - ) - ) + declarations.extend( + self._native_array_dispatch_declarations(self._module_native_array_dispatch_name(variable)) + ) for function, result in self._owned_native_array_results(plan): - for operation in result.native_array_handle.operations: - name = self._owned_native_array_operation_name(function, result, operation) - declarations.extend( - ( - CFunctionPrototype( - name, - "PyObject *", - (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), - storage="static", - ), - CDeclaration( - self._owned_native_array_operation_def_name(function, result, operation), - "static PyMethodDef", - CodeExpression(f'{{"{name}", (PyCFunction){name}, METH_VARARGS, ""}}'), - ), - ) - ) + declarations.extend( + self._native_array_dispatch_declarations(self._owned_native_array_dispatch_name(function, result)) + ) for function, argument in self._default_native_array_arguments(plan): binder_name = self._default_native_array_binder_name(argument) declarations.extend( @@ -3978,27 +4154,30 @@ def _native_array_operation_declarations( ), ) ) - for operation in argument.native_array_handle.default_handle.operations: - name = self._owned_native_array_operation_name(function, argument, operation) - declarations.extend( - ( - CFunctionPrototype( - name, - "PyObject *", - (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), - storage="static", - ), - CDeclaration( - self._owned_native_array_operation_def_name(function, argument, operation), - "static PyMethodDef", - CodeExpression(f'{{"{name}", (PyCFunction){name}, METH_VARARGS, ""}}'), - ), - ) - ) + declarations.extend( + self._native_array_dispatch_declarations(self._owned_native_array_dispatch_name(function, argument)) + ) return tuple(declarations) + @staticmethod + def _native_array_dispatch_declarations(name: str) -> tuple[CFunctionPrototype | CDeclaration, ...]: + """Declare one dispatcher function and its Python callable record.""" + return ( + CFunctionPrototype( + name, + "PyObject *", + (CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), + storage="static", + ), + CDeclaration( + f"{name}_def", + "static PyMethodDef", + CodeExpression(f'{{"{name}", (PyCFunction){name}, METH_VARARGS, ""}}'), + ), + ) + def _native_array_operation_functions(self, plan: ModulePlan) -> tuple[CFunction, ...]: - """Lower every planned owned-descriptor operation into a named C method.""" + """Lower each generated handle source into one named dispatcher.""" return ( *( self._native_array_capsule_release_function(result) @@ -4008,21 +4187,22 @@ def _native_array_operation_functions(self, plan: ModulePlan) -> tuple[CFunction self._native_array_capsule_release_function(argument) for _function, argument in self._default_native_array_arguments(plan) ), + *((self._native_array_forward_descriptor_function(),) if self._emits_native_array_backend(plan) else ()), + *self._array_actual_reader_functions(plan), + *self._inverted_descriptor_consumer_functions(plan), *( callback for variable in self._module_native_array_variables(plan) - for callback in self._module_allocatable_descriptor_callbacks(variable) + for callback in self._module_native_array_backend_functions(variable) ), *( - self._module_native_array_operation_function(variable, operation) + self._module_native_array_dispatch_function(variable) for variable in self._module_native_array_variables(plan) if variable.native_array_handle is not None - for operation in variable.native_array_handle.operations ), *( - self._owned_native_array_operation_function(function, result, operation) + self._owned_native_array_dispatch_function(function, result) for function, result in self._owned_native_array_results(plan) - for operation in result.native_array_handle.operations ), *self._default_native_array_operation_functions(plan), ) @@ -4034,7 +4214,7 @@ def _native_array_capsule_release_function( """Release descriptor payload through the module that created its record.""" descriptor = "owner_descriptor" body: tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...] = ( - CDeclaration(descriptor, "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)storage")), + CDeclaration(descriptor, "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)context")), CIf(CodeExpression(f"{descriptor} == NULL"), body=(CReturn(),)), ) handle = plan.native_array_handle @@ -4062,23 +4242,21 @@ def _native_array_capsule_release_function( return CFunction( self._native_array_capsule_release_name(plan), "void", - parameters=(CParameter("storage", "void *"),), + parameters=(CParameter("context", "void *"),), storage="static", body=body, ) def _default_native_array_operation_functions(self, plan: ModulePlan) -> tuple[CFunction, ...]: - """Lower lazy caller-handle binders and their owned operation methods.""" + """Lower lazy caller-handle binders and their owned dispatchers.""" arguments = self._default_native_array_arguments(plan) - operations = tuple( - self._owned_native_array_operation_function(function, argument, operation) - for function, argument in arguments - for operation in argument.native_array_handle.default_handle.operations + dispatchers = tuple( + self._owned_native_array_dispatch_function(function, argument) for function, argument in arguments ) binders = tuple( self._default_native_array_binder_function(function, argument) for function, argument in arguments ) - return (*operations, *binders) + return (*dispatchers, *binders) def _module_native_array_variables(self, plan: ModulePlan) -> tuple[ModuleVariablePlan, ...]: """Return borrowed module-handle plans in stable namespace order.""" @@ -4098,18 +4276,20 @@ def _module_array_owner_variables(self, plan: ModulePlan) -> tuple[ModuleVariabl ) # Borrowed module native-array-handle operations. - def _module_native_array_operation_function( - self, - variable: ModuleVariablePlan, - operation: NativeArrayOperation, - ) -> CFunction: - """Lower one planned borrowed-module operation into a private callable.""" + def _module_native_array_dispatch_function(self, variable: ModuleVariablePlan) -> CFunction: + """Lower one planned borrowed-module dispatcher.""" + handle = variable.native_array_handle + if handle is None: + raise ValueError(f"Module native array handle {variable.owner_path!r} is incomplete") return CFunction( - self._module_native_array_operation_name(variable, operation), + self._module_native_array_dispatch_name(variable), "PyObject *", parameters=(CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), storage="static", - body=self._module_native_array_operation_body(variable, operation), + body=self._native_array_dispatch_body( + handle.operations, + lambda operation: self._module_native_array_operation_body(variable, operation), + ), ) def _module_native_array_operation_body( @@ -4118,75 +4298,131 @@ def _module_native_array_operation_body( operation: NativeArrayOperation, ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: """Dispatch one module operation without rediscovering semantic policy.""" - if operation in { - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, - }: - return self._module_native_array_metadata_body(operation) - if operation in { - NativeArrayOperation.ALLOCATED, - NativeArrayOperation.ASSOCIATED, - NativeArrayOperation.CONTIGUOUS, - NativeArrayOperation.ELEMENT_LENGTH, - NativeArrayOperation.ARRAY_ACTUAL, - }: - return self._module_native_array_query_body(variable, operation) + handle = variable.native_array_handle + if handle is not None and operation in _DESCRIPTOR_ANSWERED_OPERATIONS: + if handle.descriptor_inquiries: + return self._module_native_array_inquiry_body(variable, operation) + return self._native_array_bridge_inquiry_nodes( + operation, + f"{self._module_native_array_bridge_operation_name(variable, operation)}()", + handle.array.rank, + self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.SHAPE), + "", + ) return self._module_native_array_data_operation_body(variable, operation) - @staticmethod - def _module_native_array_metadata_body( - operation: NativeArrayOperation, - ) -> tuple[CReturn, ...]: - """Return binding-known metadata that requires no bridge call.""" - if operation is NativeArrayOperation.LAYOUT: - return (CReturn(CodeExpression('PyUnicode_FromString("F")')),) - return (CReturn(CodeExpression("PyBool_FromLong(1)")),) - - def _module_native_array_query_body( + def _native_array_bridge_inquiry_nodes( self, - variable: ModuleVariablePlan, operation: NativeArrayOperation, - ) -> tuple[CReturn, ...]: - """Return one scalar fact queried from the native bridge.""" - if operation is NativeArrayOperation.ARRAY_ACTUAL and self._uses_module_allocatable_descriptor(variable): - return self._module_allocatable_array_actual_body(variable) - call = f"{self._module_native_array_bridge_operation_name(variable, operation)}()" + call: str, + rank: int, + shape_bridge: str, + owner_args: str, + ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: + """Answer one inquiry through a generated Fortran procedure. + + Only a declaration with no legal bind(C) descriptor interface takes this + route; completed policy says which, and it is the compiler's own + inquiry that answers rather than a descriptor nothing may build. + """ if operation in { NativeArrayOperation.ALLOCATED, NativeArrayOperation.ASSOCIATED, NativeArrayOperation.CONTIGUOUS, }: - expression = f"PyBool_FromLong({call})" - elif operation is NativeArrayOperation.ELEMENT_LENGTH: - expression = f"PyLong_FromLongLong((long long){call})" - else: - expression = f"PyLong_FromVoidPtr({call})" - return (CReturn(CodeExpression(expression)),) + return (CReturn(CodeExpression(f"PyBool_FromLong({call})")),) + if operation is NativeArrayOperation.ELEMENT_LENGTH: + return (CReturn(CodeExpression(f"PyLong_FromLongLong((long long){call})")),) + if operation is not NativeArrayOperation.SHAPE: + raise ValueError(f"Native array inquiry {operation.value!r} has no generated Fortran lowering") + extents = tuple(f"extent_{axis}" for axis in range(rank)) + arguments = ", ".join((*((owner_args,) if owner_args else ()), *(f"&{name}" for name in extents))) + return ( + *(CDeclaration(name, "int64_t", CodeExpression("0")) for name in extents), + CDeclaration("present", "bool", CodeExpression(f"{shape_bridge}({arguments})")), + CIf( + CodeExpression("!present"), + body=(CReturn(CodeExpression("Py_NewRef(Py_None)")),), + ), + CDeclaration("shape", "PyObject *", CodeExpression(f"PyTuple_New({rank})")), + CIf(CodeExpression("shape == NULL"), body=(CReturn(CodeExpression("NULL")),)), + *( + CExpressionStatement( + CodeExpression(f"PyTuple_SET_ITEM(shape, {axis}, PyLong_FromLongLong((long long){name}))") + ) + for axis, name in enumerate(extents) + ), + CExpressionStatement(CodeExpression("if (PyErr_Occurred()) { Py_DECREF(shape); return NULL; }")), + CReturn(CodeExpression("shape")), + ) + + def _module_native_array_inquiry_body( + self, + variable: ModuleVariablePlan, + operation: NativeArrayOperation, + ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: + """Answer one inquiry from the descriptor this module array exposes.""" + handle = variable.native_array_handle + if handle is None or handle.array.rank is None: + raise ValueError(f"Module handle {variable.owner_path!r} has no descriptor rank") + return self._native_array_projection_call_nodes( + operation, + rank=handle.array.rank, + numpy_type=self._module_native_array_numpy_type(variable), + element_size=(str(variable.character_length) if variable.character_length is not None else "0"), + descriptor_attribute=handle.descriptor_attribute, + # Module storage outlives any view of it, so a view retains nothing. + base="NULL", + entry_point=self._module_with_descriptor_name(variable), + context="NULL", + ) + + def _module_native_array_numpy_type(self, variable: ModuleVariablePlan) -> str: + """Return the NumPy element type one module array's view is built with.""" + if variable.datatype_family is DatatypeFamily.STRING: + return "NPY_STRING" + # A wider logical is exposed with the integer dtype its elements + # occupy, which is what the array projection already selected. + return PrimitiveScalarTypeRegistry.type_for(variable.semantic_type_name).array_numpy_type def _module_native_array_data_operation_body( self, variable: ModuleVariablePlan, operation: NativeArrayOperation, ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Lower shape, extraction, descriptor, and mutation operations.""" - if operation is NativeArrayOperation.SHAPE: - return self._module_native_array_shape_body(variable) - if operation is NativeArrayOperation.TO_NUMPY: - return self._module_native_array_descriptor_body(variable) - if operation is NativeArrayOperation.DESCRIPTOR: - return self._module_native_array_descriptor_body(variable) + """Lower the mutations that must reach the module variable itself.""" if operation is NativeArrayOperation.ASSOCIATE: + handle = variable.native_array_handle + if handle is None: + raise ValueError(f"Module pointer handle {variable.owner_path!r} has no descriptor policy") + associate = CExpressionStatement( + CodeExpression( + f"{self._module_native_array_bridge_operation_name(variable, operation)}(source_descriptor)" + ) + ) + association = ( + CIf( + CodeExpression("source_base_addr == NULL"), + body=( + CExpressionStatement( + CodeExpression( + f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.NULLIFY)}()" + ) + ), + ), + else_body=(associate,), + ) + if handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER + else associate + ) return ( CDeclaration("source_packed", "PyObject *"), CExpressionStatement(CodeExpression('if (!PyArg_ParseTuple(args, "O", &source_packed)) return NULL')), - *self._pointer_association_source_nodes(variable), - CExpressionStatement( - CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, operation)}(source_descriptor)" - ) + *self._pointer_association_source_nodes( + variable, + establish_if_present_only=handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER, ), + association, CExpressionStatement(CodeExpression("Py_RETURN_NONE")), ) if operation in {NativeArrayOperation.ALLOCATE, NativeArrayOperation.RESIZE}: @@ -4203,265 +4439,176 @@ def _module_native_array_data_operation_body( ) raise ValueError(f"Unsupported module native array operation for {variable.owner_path!r}: {operation!r}") - def _module_native_array_shape_body( - self, - variable: ModuleVariablePlan, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Return current module-array extents as one Python tuple.""" + @staticmethod + def _uses_module_descriptor_backend(variable: ModuleVariablePlan) -> bool: + """Return whether a handle reaches its descriptor through a consumer. + + A module array hands a plan-selected descriptor projection to a + consumer rather than filling a record supplied from C. + """ handle = variable.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Module handle {variable.owner_path!r} has no rank") - rank = handle.array.rank - extents = tuple(f"extent_{axis}" for axis in range(rank)) - return ( - *(CDeclaration(name, "int64_t", CodeExpression("0")) for name in extents), - CExpressionStatement( - CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.SHAPE)}(" - f"{', '.join(f'&{name}' for name in extents)})" - ) - ), - CDeclaration("shape", "PyObject *", CodeExpression(f"PyTuple_New({rank})")), - CIf(CodeExpression("shape == NULL"), body=(CReturn(CodeExpression("NULL")),)), - *( - CExpressionStatement( - CodeExpression(f"PyTuple_SET_ITEM(shape, {axis}, PyLong_FromLongLong((long long){name}))") - ) - for axis, name in enumerate(extents) - ), - CExpressionStatement(CodeExpression("if (PyErr_Occurred()) { Py_DECREF(shape); return NULL; }")), - CReturn(CodeExpression("shape")), + return bool( + handle is not None + and handle.descriptor_interop + in { + NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR, + NativeArrayDescriptorInterop.POINTER_C_DESCRIPTOR, + } ) - def _module_native_array_descriptor_body( + def _module_native_array_backend_functions( self, variable: ModuleVariablePlan, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Return standard descriptor facts for module extraction and handoff.""" + ) -> tuple[CFunction, ...]: + """Return the entry point one module array publishes, and its record.""" + if not self._uses_module_descriptor_backend(variable): + return () handle = variable.native_array_handle if handle is None or handle.array.rank is None: raise ValueError(f"Module handle {variable.owner_path!r} has no descriptor rank") - if self._uses_module_allocatable_descriptor(variable): - return self._module_allocatable_descriptor_body(variable) - if handle.descriptor_kind is NativeArrayDescriptorKind.POINTER: - return self._module_pointer_descriptor_body(variable) - return self._module_contiguous_descriptor_body(variable) - - @staticmethod - def _uses_module_allocatable_descriptor(variable: ModuleVariablePlan) -> bool: - """Return whether completed policy selected callback-based descriptor access.""" - handle = variable.native_array_handle - return bool( - handle is not None - and handle.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR - ) + return self._module_native_array_backend_nodes(variable, handle) - def _module_allocatable_descriptor_body( + def _module_native_array_backend_nodes( self, variable: ModuleVariablePlan, - ) -> tuple[CDeclaration | CExpressionStatement | CReturn, ...]: - """Request the current standard descriptor and return its decoded facts.""" - return ( - CDeclaration("descriptor_record", "PyObject *", CodeExpression("NULL")), - CExpressionStatement( - CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.DESCRIPTOR)}(" - f"{self._module_descriptor_callback_name(variable)}, &descriptor_record)" - ) - ), - CReturn(CodeExpression("descriptor_record")), + handle: NativeArrayHandlePlan, + ) -> tuple[CFunction | CDeclaration, ...]: + """Emit the native descriptor backend one module array handle publishes. + + The backend names the bridge symbols for this variable so a consumer can + reach it with one indirect call instead of a Python operation lookup. + A module variable needs no owner, so the backend is a file-scope + constant rather than per-handle storage. + """ + cfi_type = self._module_native_array_cfi_type(variable) + if cfi_type is None: + return () + bridge = self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.DESCRIPTOR) + forward = self._module_with_descriptor_name(variable) + element_size = ( + "0" + if variable.datatype_family is DatatypeFamily.STRING + else f"sizeof({PrimitiveScalarTypeRegistry.type_for(variable.semantic_type_name).array_c_spelling})" ) - - def _module_allocatable_array_actual_body( - self, - variable: ModuleVariablePlan, - ) -> tuple[CDeclaration | CExpressionStatement | CReturn, ...]: - """Request the current standard descriptor and expose only its data address.""" return ( - CDeclaration("base_addr", "void *", CodeExpression("NULL")), - CExpressionStatement( + CFunction( + forward, + "void", + parameters=( + CParameter("context", "void *"), + CParameter("consumer", "prik_native_array_descriptor_fn"), + CParameter("consumer_context", "void *"), + ), + storage="static", + body=( + CDeclaration( + "forwarded", + "prik_native_array_descriptor_forward", + CodeExpression("{consumer, consumer_context}"), + ), + CExpressionStatement(CodeExpression("(void)context")), + CExpressionStatement(CodeExpression(f"{bridge}(prik_native_array_forward_descriptor, &forwarded)")), + CReturn(), + ), + ), + CDeclaration( + self._module_native_array_backend_name(variable), + "static prik_native_array_backend", CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.ARRAY_ACTUAL)}(" - f"{self._module_array_actual_callback_name(variable)}, &base_addr)" - ) + f"{{{self._native_array_handle_kind_constant(handle)}, " + f"{self._native_array_descriptor_attribute_constant(handle)}, {handle.array.rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({handle.array.rank})), {cfi_type}, {element_size}, " + f"NULL, {forward}, NULL}}" + ), ), - CReturn(CodeExpression("PyLong_FromVoidPtr(base_addr)")), ) - def _module_allocatable_descriptor_callbacks( - self, - variable: ModuleVariablePlan, - ) -> tuple[CFunction, ...]: - """Return C consumers for descriptor-record and data-address operations.""" - if not self._uses_module_allocatable_descriptor(variable): - return () - handle = variable.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Module handle {variable.owner_path!r} has no descriptor rank") - descriptor_callback = CFunction( - self._module_descriptor_callback_name(variable), - "void", - parameters=(CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), - storage="static", - body=( - CExpressionStatement(CodeExpression("*(PyObject **)context = NULL")), - *self._native_array_descriptor_record_nodes( - handle.array.rank, - "descriptor", - return_target="*(PyObject **)context", - ), - ), + def _emits_native_array_backend(self, plan: ModulePlan) -> bool: + """Report whether any handle in this module publishes a native backend.""" + if any( + self._uses_module_descriptor_backend(variable) for variable in self._module_native_array_variables(plan) + ): + return True + return any( + field.native_array_handle is not None + and field.native_array_handle.descriptor_inquiries + and field.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + for _owner, field, _operation_name, _callbacks in self._derived_handle_targets(plan) ) - array_actual_callback = CFunction( - self._module_array_actual_callback_name(variable), + + @staticmethod + def _native_array_forward_descriptor_function() -> CFunction: + """Emit the consumer that hands a runtime descriptor to a backend consumer.""" + return CFunction( + "prik_native_array_forward_descriptor", "void", parameters=(CParameter("descriptor", "CFI_cdesc_t *"), CParameter("context", "void *")), storage="static", body=( - CExpressionStatement(CodeExpression("*(void **)context = descriptor->base_addr")), + CDeclaration( + "forwarded", + "prik_native_array_descriptor_forward *", + CodeExpression("(prik_native_array_descriptor_forward *)context"), + ), + CExpressionStatement(CodeExpression("forwarded->consumer(descriptor, forwarded->context)")), CReturn(), ), ) - return descriptor_callback, array_actual_callback - - def _module_descriptor_callback_name(self, variable: ModuleVariablePlan) -> str: - """Return the binding-local module descriptor callback name derived from the supplied completed binding records; this helper preserves completed policy.""" - owner = re.sub(r"\W", "_", variable.owner_path).casefold() - return f"prik_module_{owner}_descriptor_callback" - def _module_array_actual_callback_name(self, variable: ModuleVariablePlan) -> str: - """Return the binding-local module array actual callback name derived from the supplied completed binding records; this helper preserves completed policy.""" - owner = re.sub(r"\W", "_", variable.owner_path).casefold() - return f"prik_module_{owner}_array_actual_callback" + def _module_native_array_backend_capsule_name(self, variable: ModuleVariablePlan, prefix: str) -> str: + """Return the local holding this variable's published backend.""" + if not self._uses_module_descriptor_backend(variable): + return "Py_None" + return f"{prefix}_native_backend" - def _module_contiguous_descriptor_body( + def _module_native_array_backend_declaration_nodes( self, variable: ModuleVariablePlan, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Build allocatable descriptor facts from native data and extents.""" - handle = variable.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Module handle {variable.owner_path!r} has no descriptor rank") - rank = handle.array.rank - scalar_type = ( - None - if variable.datatype_family is DatatypeFamily.STRING - else PrimitiveScalarTypeRegistry.type_for(variable.semantic_type_name) - ) - extents = tuple(f"extent_{axis}" for axis in range(rank)) - elem_len = ( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.ELEMENT_LENGTH)}()" - if variable.datatype_family is DatatypeFamily.STRING - else f"sizeof({scalar_type.c_spelling})" - ) - nodes: list[CDeclaration | CExpressionStatement | CIf | CReturn] = [ + prefix: str, + ) -> tuple[CDeclaration, ...]: + """Declare and build the capsule publishing one variable's backend.""" + if not self._uses_module_descriptor_backend(variable): + return () + return ( CDeclaration( - "base_addr", - "void *", - CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.ARRAY_ACTUAL)}()" - ), - ), - *(CDeclaration(name, "int64_t", CodeExpression("0")) for name in extents), - CExpressionStatement( - CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.SHAPE)}(" - f"{', '.join(f'&{name}' for name in extents)})" - ) + f"{prefix}_native_backend", + "PyObject *", + CodeExpression("NULL"), ), - CDeclaration("dimensions", "PyObject *", CodeExpression(f"PyList_New({rank})")), - CIf(CodeExpression("dimensions == NULL"), body=(CReturn(CodeExpression("NULL")),)), - CDeclaration("stride", "int64_t", CodeExpression(elem_len)), - ] - for axis, extent in enumerate(extents): - nodes.extend( - ( - CDeclaration( - f"dimension_{axis}", - "PyObject *", - CodeExpression( - f'Py_BuildValue("{{sL,sL,sL}}", "lower_bound", (long long)0, ' - f'"extent", (long long){extent}, "sm", (long long)stride)' - ), - ), - CIf( - CodeExpression(f"dimension_{axis} == NULL"), - body=( - CExpressionStatement(CodeExpression("Py_DECREF(dimensions)")), - CReturn(CodeExpression("NULL")), - ), - ), - CExpressionStatement(CodeExpression(f"PyList_SET_ITEM(dimensions, {axis}, dimension_{axis})")), - CExpressionStatement(CodeExpression(f"stride *= ({extent} > 0 ? {extent} : 1)")), - ) - ) - nodes.extend(self._descriptor_record_return_nodes("base_addr", elem_len, rank)) - return tuple(nodes) + ) - def _module_pointer_descriptor_body( + def _module_native_array_backend_release_nodes( self, variable: ModuleVariablePlan, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Decode a call-local standard pointer descriptor without copying data.""" - handle = variable.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Pointer module handle {variable.owner_path!r} has no descriptor rank") - cfi_type = self._module_native_array_cfi_type(variable) - if cfi_type is None: - raise ValueError(f"Pointer module handle {variable.owner_path!r} has no CFI type") - rank = handle.array.rank - elem_len = self._module_native_array_elem_size(variable) - return ( - CDeclaration("descriptor_storage", f"CFI_CDESC_T({rank})"), - CDeclaration("descriptor", "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)&descriptor_storage")), - CDeclaration("status", "int", CodeExpression("CFI_SUCCESS")), - CExpressionStatement( - CodeExpression( - f"status = CFI_establish(descriptor, NULL, CFI_attribute_pointer, " - f"{cfi_type}, {elem_len}, {rank}, NULL)" - ) - ), - CIf( - CodeExpression("status != CFI_SUCCESS"), - body=( - CExpressionStatement( - CodeExpression( - 'PyErr_SetString(PyExc_RuntimeError, "failed to establish pointer descriptor reader")' - ) - ), - CReturn(CodeExpression("NULL")), - ), - ), - CExpressionStatement( - CodeExpression( - f"{self._module_native_array_bridge_operation_name(variable, NativeArrayOperation.DESCRIPTOR)}" - "(descriptor)" - ) - ), - *self._native_array_descriptor_record_nodes(rank, "descriptor"), - ) + prefix: str, + ) -> tuple[CExpressionStatement, ...]: + """Release the reference the published backend capsule was created with.""" + if not self._uses_module_descriptor_backend(variable): + return () + return (CExpressionStatement(CodeExpression(f"Py_XDECREF({prefix}_native_backend)")),) - def _descriptor_record_return_nodes( - self, - base_addr: str, - elem_len: str, - rank: int, - ) -> tuple[CDeclaration | CExpressionStatement | CReturn, ...]: - """Finish one standard descriptor mapping from existing dimensions.""" + def _module_native_array_backend_capsule(self, variable: ModuleVariablePlan) -> str: + """Return the expression publishing this variable's native backend.""" + if not self._uses_module_descriptor_backend(variable): + return "Py_None" return ( - CDeclaration( - "descriptor_record", - "PyObject *", - CodeExpression( - f'Py_BuildValue("{{sK,sK,si,sO}}", "base_addr", ' - f'(unsigned long long)(uintptr_t){base_addr}, "elem_len", ' - f'(unsigned long long)({elem_len}), "rank", {rank}, "dim", dimensions)' - ), - ), - CExpressionStatement(CodeExpression("Py_DECREF(dimensions)")), - CReturn(CodeExpression("descriptor_record")), + f"PyCapsule_New(&{self._module_native_array_backend_name(variable)}, " + "prik_native_array_backend_capsule_name(), NULL)" ) + def _module_with_descriptor_name(self, variable: ModuleVariablePlan) -> str: + """Return the forwarder name that drives this variable's descriptor bridge.""" + return f"{self._module_descriptor_callback_name(variable)}_with_descriptor" + + def _module_native_array_backend_name(self, variable: ModuleVariablePlan) -> str: + """Return the file-scope native backend name for one module array.""" + return f"{self._module_descriptor_callback_name(variable)}_backend" + + def _module_descriptor_callback_name(self, variable: ModuleVariablePlan) -> str: + """Return the binding-local module descriptor callback name derived from the supplied completed binding records; this helper preserves completed policy.""" + owner = re.sub(r"\W", "_", variable.owner_path).casefold() + return f"prik_module_{owner}_descriptor_callback" + def _module_native_array_shape_mutation_body( self, variable: ModuleVariablePlan, @@ -4497,22 +4644,10 @@ def _module_native_array_shape_mutation_body( CExpressionStatement(CodeExpression("Py_RETURN_NONE")), ) - def _module_native_array_operation_name( - self, - variable: ModuleVariablePlan, - operation: NativeArrayOperation, - ) -> str: - """Return the binding-local module native array operation name derived from the supplied completed binding records; this helper preserves completed policy.""" + def _module_native_array_dispatch_name(self, variable: ModuleVariablePlan) -> str: + """Return the binding-local module native array dispatcher name.""" owner = re.sub(r"\W", "_", variable.owner_path).casefold() - return f"prik_module_{owner}_{operation.value}" - - def _module_native_array_operation_def_name( - self, - variable: ModuleVariablePlan, - operation: NativeArrayOperation, - ) -> str: - """Return the binding-local module native array operation def name derived from the supplied completed binding records; this helper preserves completed policy.""" - return f"{self._module_native_array_operation_name(variable, operation)}_def" + return f"prik_module_{owner}_dispatch" def _module_native_array_bridge_operation_name( self, @@ -4558,21 +4693,26 @@ def _default_native_array_arguments( is NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR ) - def _owned_native_array_operation_function( + def _owned_native_array_dispatch_function( self, function: FunctionPlan, - result: ResultPlan, - operation: NativeArrayOperation, + result: ArgumentTransferPlan | ResultPlan, ) -> CFunction: - """Dispatch one planned owned-descriptor runtime operation.""" - name = self._owned_native_array_operation_name(function, result, operation) - body = self._owned_native_array_operation_body(result, operation) + """Lower one planned owned-descriptor dispatcher.""" + operations = ( + result.native_array_handle.default_handle.operations + if isinstance(result, ArgumentTransferPlan) + else result.native_array_handle.operations + ) return CFunction( - name, + self._owned_native_array_dispatch_name(function, result), "PyObject *", parameters=(CParameter("self", "PyObject *"), CParameter("args", "PyObject *")), storage="static", - body=body, + body=self._native_array_dispatch_body( + operations, + lambda operation: self._owned_native_array_operation_body(result, operation), + ), ) def _default_native_array_binder_function( @@ -4580,21 +4720,33 @@ def _default_native_array_binder_function( function: FunctionPlan, argument: ArgumentTransferPlan, ) -> CFunction: - """Attach one compiler-compatible owned descriptor to a fresh handle.""" + """Attach one compiler-compatible owned descriptor to a fresh handle. + + An argument that projects a result decides what the handle exposes + afterwards, because the handle stands for what the call produced. One + that does not is only borrowing the handle for a descriptor, so it + passes NULL and leaves the handle's own exposure alone. + """ handle = argument.native_array_handle default = handle.default_handle + exposure = ( + f'"{handle.extraction_action.value}"' + if handle.output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE + else "NULL" + ) dtype = self._native_array_dtype_for_semantic_type( argument.semantic_type_name, argument.datatype_family, ) cfi_type = self._native_array_cfi_type(argument) - elem_len = f"sizeof({PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).c_spelling})" + elem_len = f"sizeof({PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).array_c_spelling})" + dispatch = self._owned_native_array_dispatch_name(function, argument) nodes: list[CDeclaration | CExpressionStatement | CIf | CReturn] = [ CDeclaration("handle_obj", "PyObject *"), CDeclaration("owner_descriptor", "CFI_cdesc_t *", CodeExpression("NULL")), CDeclaration("owner_status", "int", CodeExpression("CFI_SUCCESS")), - CDeclaration("ops", "PyObject *", CodeExpression("NULL")), - CDeclaration("operation", "PyObject *", CodeExpression("NULL")), + CDeclaration("capabilities", "PyObject *", CodeExpression("NULL")), + CDeclaration("invoke", "PyObject *", CodeExpression("NULL")), CDeclaration("owner_obj", "PyObject *", CodeExpression("NULL")), CDeclaration("runtime", "PyObject *", CodeExpression("NULL")), CDeclaration("helper", "PyObject *", CodeExpression("NULL")), @@ -4630,40 +4782,26 @@ def _default_native_array_binder_function( CReturn(CodeExpression("NULL")), ), ), - CExpressionStatement(CodeExpression("ops = PyDict_New()")), + CExpressionStatement( + CodeExpression(f"capabilities = {self._native_array_capabilities_expression(default.operations)}") + ), + CIf( + CodeExpression("capabilities == NULL"), + body=( + CExpressionStatement(CodeExpression("free(owner_descriptor)")), + CReturn(CodeExpression("NULL")), + ), + ), + CExpressionStatement(CodeExpression(f"invoke = PyCFunction_NewEx(&{dispatch}_def, NULL, NULL)")), CIf( - CodeExpression("ops == NULL"), + CodeExpression("invoke == NULL"), body=( + CExpressionStatement(CodeExpression("Py_DECREF(capabilities)")), CExpressionStatement(CodeExpression("free(owner_descriptor)")), CReturn(CodeExpression("NULL")), ), ), ] - for operation in default.operations: - definition = self._owned_native_array_operation_def_name(function, argument, operation) - nodes.extend( - ( - CExpressionStatement(CodeExpression(f"operation = PyCFunction_NewEx(&{definition}, NULL, NULL)")), - CIf( - CodeExpression("operation == NULL"), - body=( - CExpressionStatement(CodeExpression("Py_DECREF(ops)")), - CExpressionStatement(CodeExpression("free(owner_descriptor)")), - CReturn(CodeExpression("NULL")), - ), - ), - CIf( - CodeExpression(f'PyDict_SetItemString(ops, "{operation.value}", operation) < 0'), - body=( - CExpressionStatement(CodeExpression("Py_DECREF(operation)")), - CExpressionStatement(CodeExpression("Py_DECREF(ops)")), - CExpressionStatement(CodeExpression("free(owner_descriptor)")), - CReturn(CodeExpression("NULL")), - ), - ), - CExpressionStatement(CodeExpression("Py_DECREF(operation)")), - ) - ) nodes.extend( ( CExpressionStatement( @@ -4674,7 +4812,8 @@ def _default_native_array_binder_function( CIf( CodeExpression("owner_obj == NULL"), body=( - CExpressionStatement(CodeExpression("Py_DECREF(ops)")), + CExpressionStatement(CodeExpression("Py_DECREF(invoke)")), + CExpressionStatement(CodeExpression("Py_DECREF(capabilities)")), CExpressionStatement(CodeExpression("free(owner_descriptor)")), CReturn(CodeExpression("NULL")), ), @@ -4685,7 +4824,8 @@ def _default_native_array_binder_function( CodeExpression("runtime == NULL"), body=( CExpressionStatement(CodeExpression("Py_DECREF(owner_obj)")), - CExpressionStatement(CodeExpression("Py_DECREF(ops)")), + CExpressionStatement(CodeExpression("Py_DECREF(invoke)")), + CExpressionStatement(CodeExpression("Py_DECREF(capabilities)")), CExpressionStatement(CodeExpression("free(owner_descriptor)")), CReturn(CodeExpression("NULL")), ), @@ -4698,21 +4838,27 @@ def _default_native_array_binder_function( CodeExpression("helper == NULL"), body=( CExpressionStatement(CodeExpression("Py_DECREF(owner_obj)")), - CExpressionStatement(CodeExpression("Py_DECREF(ops)")), + CExpressionStatement(CodeExpression("Py_DECREF(invoke)")), + CExpressionStatement(CodeExpression("Py_DECREF(capabilities)")), CExpressionStatement(CodeExpression("free(owner_descriptor)")), CReturn(CodeExpression("NULL")), ), ), + # The backend over the attached storage is what the handle + # carries as its owner and what a later call reads directly, so + # it is published once and handed over under both names. CExpressionStatement( CodeExpression( - f'result = PyObject_CallFunction(helper, "OssiOOssO", handle_obj, ' - f'"{handle.descriptor_kind.value}", "{dtype}", {handle.array.rank}, ops, owner_obj, ' - f'"{default.descriptor_ownership.value}", "{handle.extraction_action.value}", Py_None)' + f'result = PyObject_CallFunction(helper, "OssiOOOszOO", handle_obj, ' + f'"{handle.descriptor_kind.value}", "{dtype}", {handle.array.rank}, ' + "invoke, capabilities, owner_obj, " + f'"{default.descriptor_ownership.value}", {exposure}, Py_None, owner_obj)' ) ), CExpressionStatement(CodeExpression("Py_DECREF(helper)")), CExpressionStatement(CodeExpression("Py_DECREF(owner_obj)")), - CExpressionStatement(CodeExpression("Py_DECREF(ops)")), + CExpressionStatement(CodeExpression("Py_DECREF(invoke)")), + CExpressionStatement(CodeExpression("Py_DECREF(capabilities)")), CReturn(CodeExpression("result")), ) ) @@ -4730,6 +4876,8 @@ def _owned_native_array_operation_body( operation: NativeArrayOperation, ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: """Return one operation body over persistent CFI owner storage.""" + if operation in _DESCRIPTOR_ANSWERED_OPERATIONS: + return self._owned_native_array_inquiry_body(result, operation) if operation is NativeArrayOperation.ASSOCIATE: return self._owned_native_array_associate_body(result) if operation in {NativeArrayOperation.ALLOCATE, NativeArrayOperation.RESIZE}: @@ -4738,38 +4886,52 @@ def _owned_native_array_operation_body( release_existing=operation is NativeArrayOperation.RESIZE, ) handler = self._owned_native_array_operation_handler(operation) - materialize_descriptor = operation not in { - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, - NativeArrayOperation.DESCRIPTOR, - NativeArrayOperation.DESTROY, - } return ( *self._owned_native_array_owner_nodes( result, "owner", - materialize_descriptor=materialize_descriptor, + materialize_descriptor=operation is not NativeArrayOperation.DESTROY, ), *handler(result), ) + def _owned_native_array_inquiry_body( + self, + result: ArgumentTransferPlan | ResultPlan, + operation: NativeArrayOperation, + ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: + """Answer one inquiry from the descriptor this handle owns. + + The storage is already here, so the entry point hands it straight to the + consumer. A view over it must keep the capsule that owns the storage + alive, so that capsule becomes the view's base. + """ + handle = result.native_array_handle + if handle is None or handle.array.rank is None: + raise ValueError(f"Owned native array handle {result.owner_path!r} has no descriptor rank") + return ( + *self._owned_native_array_owner_nodes(result, "owner", materialize_descriptor=False), + *self._native_array_projection_call_nodes( + operation, + rank=handle.array.rank, + numpy_type=self._owned_native_array_numpy_type(result), + element_size=self._native_array_expected_element_size(result), + descriptor_attribute=handle.descriptor_attribute, + base="owner_obj", + entry_point="owner_backend->with_descriptor", + context="owner_backend->context", + ), + ) + + def _owned_native_array_numpy_type(self, plan: ArgumentTransferPlan | ResultPlan) -> str: + """Return the NumPy element type one owned handle's view is built with.""" + if plan.datatype_family is DatatypeFamily.STRING: + return "NPY_STRING" + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_numpy_type + def _owned_native_array_operation_handler(self, operation: NativeArrayOperation): """Return one directly named operation lowerer.""" handlers = { - NativeArrayOperation.SHAPE: self._owned_native_array_shape_body, - NativeArrayOperation.TO_NUMPY: self._owned_native_array_descriptor_record_body, - NativeArrayOperation.ELEMENT_LENGTH: self._owned_native_array_element_length_body, - NativeArrayOperation.ARRAY_ACTUAL: self._owned_native_array_actual_body, - NativeArrayOperation.DESCRIPTOR: self._owned_native_array_descriptor_body, - NativeArrayOperation.ALLOCATED: self._owned_native_array_allocated_body, - NativeArrayOperation.ASSOCIATED: self._owned_native_array_associated_body, - NativeArrayOperation.NATIVE_BYTE_ORDER: self._owned_native_array_true_body, - NativeArrayOperation.ALIGNED: self._owned_native_array_true_body, - NativeArrayOperation.WRITEABLE: self._owned_native_array_true_body, - NativeArrayOperation.LAYOUT: self._owned_native_array_layout_body, - NativeArrayOperation.CONTIGUOUS: self._owned_native_array_contiguous_body, NativeArrayOperation.DEALLOCATE: self._owned_native_array_deallocate_body, NativeArrayOperation.NULLIFY: self._owned_native_array_nullify_body, NativeArrayOperation.DESTROY: self._owned_native_array_destroy_body, @@ -4796,99 +4958,137 @@ def _owned_native_array_associate_body( CExpressionStatement(CodeExpression("Py_RETURN_NONE")), ) - def _owned_native_array_descriptor_record_body( - self, - result: ResultPlan, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Expose one owned descriptor record for shape or NumPy extraction.""" - handle = result.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Owned result {result.owner_path!r} has no descriptor rank") - return self._native_array_descriptor_record_nodes(handle.array.rank, "owner_descriptor") - - def _owned_native_array_shape_body( + def _pointer_association_source_nodes( self, - result: ResultPlan, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Expose extents using the typed compiler descriptor inquiry.""" - if self._is_owned_deferred_character_result(result): - return self._owned_native_array_descriptor_record_body(result) - handle = result.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Owned result {result.owner_path!r} has no shape rank") - dimensions = tuple(f"extent_{axis}" for axis in range(handle.array.rank)) - return ( - *(CDeclaration(name, "int64_t", CodeExpression("0")) for name in dimensions), + plan: ArgumentTransferPlan | ResultPlan | ModuleVariablePlan | DerivedFieldPlan, + *, + establish_if_present_only: bool = False, + ) -> tuple[CDeclaration | CExpressionStatement | CIf, ...]: + """Establish one call-local pointer descriptor from validated source facts.""" + handle = plan.native_array_handle + rank = handle.array.rank + cfi_type = self._pointer_association_cfi_type(plan) + expected_fields = 3 + 3 * rank + nodes: list[CDeclaration | CExpressionStatement | CIf] = [ + CDeclaration("source_item", "PyObject *", CodeExpression("NULL")), + CDeclaration("source_storage", f"CFI_CDESC_T({rank})"), + CDeclaration("source_descriptor", "CFI_cdesc_t *", CodeExpression("NULL")), + CDeclaration("source_base_addr", "void *", CodeExpression("NULL")), + CDeclaration("source_elem_len", "size_t", CodeExpression("0")), + CDeclaration("source_descriptor_rank", "CFI_rank_t", CodeExpression("0")), + CDeclaration(f"source_extents[{rank}]", "CFI_index_t"), + *( + CDeclaration(f"source_{label}_{axis}", "CFI_index_t", CodeExpression("0")) + for axis in range(rank) + for label in ("lower_bound", "extent", "stride_multiplier") + ), + CDeclaration("source_establish_status", "int", CodeExpression("CFI_SUCCESS")), CExpressionStatement( CodeExpression( - f"{self._owned_native_array_bridge_operation_name(result, NativeArrayOperation.SHAPE)}" - f"(owner_descriptor, {', '.join(f'&{name}' for name in dimensions)})" + f"if (!PyTuple_Check(source_packed) || PyTuple_GET_SIZE(source_packed) != {expected_fields}) {{ " + f'PyErr_SetString(PyExc_TypeError, "pointer association requires {expected_fields} ' + 'descriptor facts"); return NULL; }' ) ), - CReturn(CodeExpression(f'Py_BuildValue("({",".join("L" for _ in dimensions)})", {", ".join(dimensions)})')), - ) - - def _owned_native_array_actual_body(self, _result: ResultPlan) -> tuple[CReturn, ...]: - """Expose the current owned allocation data address.""" - return (CReturn(CodeExpression("PyLong_FromVoidPtr(owner_descriptor->base_addr)")),) - - def _owned_native_array_element_length_body(self, _result: ResultPlan) -> tuple[CReturn, ...]: - """Expose the current deferred character element width.""" - return (CReturn(CodeExpression("PyLong_FromSize_t(owner_descriptor->elem_len)")),) - - def _owned_native_array_descriptor_body( - self, - _result: ResultPlan, - ) -> tuple[CExpressionStatement | CReturn, ...]: - """Expose the versioned owner capsule for cross-extension handoff.""" - return ( - CExpressionStatement(CodeExpression("Py_INCREF(owner_obj)")), - CReturn(CodeExpression("owner_obj")), + *self._pointer_association_fact_nodes("source_base_addr", 0, pointer=True), + *self._pointer_association_fact_nodes("source_elem_len", 1, unsigned=True), + *self._pointer_association_fact_nodes("source_descriptor_rank", 2), + ] + for axis in range(rank): + offset = 3 + 3 * axis + nodes.extend( + ( + *self._pointer_association_fact_nodes(f"source_lower_bound_{axis}", offset), + *self._pointer_association_fact_nodes(f"source_extent_{axis}", offset + 1), + *self._pointer_association_fact_nodes(f"source_stride_multiplier_{axis}", offset + 2), + CExpressionStatement(CodeExpression(f"source_extents[{axis}] = source_extent_{axis}")), + ) + ) + nodes.append( + CExpressionStatement( + CodeExpression( + f"if (source_descriptor_rank != {rank}) {{ PyErr_Format(PyExc_ValueError, " + f'"pointer association source rank %d does not match destination rank {rank}", ' + "(int)source_descriptor_rank); return NULL; }" + ) + ) ) - - def _owned_native_array_allocated_body(self, _result: ResultPlan) -> tuple[CReturn, ...]: - """Report the current allocation state.""" - if self._is_owned_deferred_character_result(_result): - return (CReturn(CodeExpression("PyBool_FromLong(owner_descriptor->base_addr != NULL)")),) - return ( - CReturn( + establish = [ + CExpressionStatement( CodeExpression( - f"PyBool_FromLong({self._owned_native_array_bridge_operation_name(_result, NativeArrayOperation.ALLOCATED)}" - "(owner_descriptor))" + "source_establish_status = CFI_establish((CFI_cdesc_t *)&source_storage, " + f"source_base_addr, CFI_attribute_pointer, {cfi_type}, source_elem_len, " + f"{rank}, source_extents)" ) ), - ) - - def _owned_native_array_associated_body(self, result: ResultPlan) -> tuple[CReturn, ...]: - """Report the current pointer association state.""" - return self._owned_native_array_bridge_state_body(result, NativeArrayOperation.ASSOCIATED) - - def _owned_native_array_contiguous_body(self, result: ResultPlan) -> tuple[CReturn, ...]: - """Report whether the current pointer target is contiguous.""" - return self._owned_native_array_bridge_state_body(result, NativeArrayOperation.CONTIGUOUS) - - def _owned_native_array_bridge_state_body( - self, - result: ResultPlan, - operation: NativeArrayOperation, - ) -> tuple[CReturn, ...]: - """Call one typed compiler descriptor inquiry.""" - return ( - CReturn( + CExpressionStatement( CodeExpression( - f"PyBool_FromLong({self._owned_native_array_bridge_operation_name(result, operation)}" - "(owner_descriptor))" + "if (source_establish_status != CFI_SUCCESS) { " + 'PyErr_SetString(PyExc_RuntimeError, "failed to establish pointer association source"); ' + "return NULL; }" ) ), - ) + ] + for axis in range(rank): + establish.extend( + ( + CExpressionStatement( + CodeExpression( + f"((CFI_cdesc_t *)&source_storage)->dim[{axis}].lower_bound = source_lower_bound_{axis}" + ) + ), + CExpressionStatement( + CodeExpression(f"((CFI_cdesc_t *)&source_storage)->dim[{axis}].extent = source_extent_{axis}") + ), + CExpressionStatement( + CodeExpression( + f"((CFI_cdesc_t *)&source_storage)->dim[{axis}].sm = source_stride_multiplier_{axis}" + ) + ), + ) + ) + establish.append(CExpressionStatement(CodeExpression("source_descriptor = (CFI_cdesc_t *)&source_storage"))) + if establish_if_present_only: + nodes.append(CIf(CodeExpression("source_base_addr != NULL"), body=tuple(establish))) + else: + nodes.extend(establish) + return tuple(nodes) - def _owned_native_array_true_body(self, _result: ResultPlan) -> tuple[CReturn, ...]: - """Return one invariant true array capability.""" - return (CReturn(CodeExpression("PyBool_FromLong(1)")),) + @staticmethod + def _pointer_association_fact_nodes( + target: str, + index: int, + *, + pointer: bool = False, + unsigned: bool = False, + ) -> tuple[CExpressionStatement, ...]: + """Decode one pointer-association descriptor fact.""" + if pointer: + conversion = "(void *)PyLong_AsVoidPtr(source_item)" + error = f"{target} == NULL && PyErr_Occurred()" + elif unsigned: + conversion = "(size_t)PyLong_AsUnsignedLongLong(source_item)" + error = "PyErr_Occurred()" + else: + conversion = "PyLong_AsLongLong(source_item)" + error = "PyErr_Occurred()" + return ( + CExpressionStatement(CodeExpression(f"source_item = PyTuple_GET_ITEM(source_packed, {index})")), + CExpressionStatement(CodeExpression(f"{target} = {conversion}")), + CExpressionStatement(CodeExpression(f"if ({error}) return NULL")), + ) - def _owned_native_array_layout_body(self, _result: ResultPlan) -> tuple[CReturn, ...]: - """Return the planned Fortran layout marker.""" - return (CReturn(CodeExpression('PyUnicode_FromString("F")')),) + @staticmethod + def _pointer_association_cfi_type( + plan: ArgumentTransferPlan | ResultPlan | ModuleVariablePlan | DerivedFieldPlan, + ) -> str: + """Return the completed standard-descriptor type for pointer assignment.""" + if isinstance(plan, DerivedFieldPlan): + if plan.string_element: + return "CFI_type_char" + elif plan.datatype_family is DatatypeFamily.STRING: + return "CFI_type_char" + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_cfi_type def _owned_native_array_deallocate_body( self, @@ -4910,7 +5110,7 @@ def _owned_native_array_destroy_body( ) -> tuple[CExpressionStatement, ...]: """Destroy payload and persistent owner storage.""" return ( - CExpressionStatement(CodeExpression("prik_native_array_handle_release(owner_handle)")), + CExpressionStatement(CodeExpression("prik_native_array_backend_release(owner_backend)")), CExpressionStatement(CodeExpression("Py_RETURN_NONE")), ) @@ -4928,7 +5128,7 @@ def _owned_native_array_owner_nodes( return ( CDeclaration(f"{prefix}_obj", "PyObject *"), *(CDeclaration(name, "PyObject *") for name in trailing_objects), - CDeclaration(f"{prefix}_handle", "prik_native_array_handle *", CodeExpression("NULL")), + CDeclaration(f"{prefix}_backend", "prik_native_array_backend *", CodeExpression("NULL")), *( (CDeclaration(f"{prefix}_descriptor", "CFI_cdesc_t *", CodeExpression("NULL")),) if materialize_descriptor @@ -4943,210 +5143,28 @@ def _owned_native_array_owner_nodes( ), CExpressionStatement( CodeExpression( - f"{prefix}_handle = prik_native_array_handle_from_capsule({prefix}_obj, " - f"{self._native_array_handle_kind_constant(handle)}, {handle.array.rank}, {cfi_type}, " - f"{self._native_array_expected_element_size(plan)}, " - f"sizeof(CFI_CDESC_T({handle.array.rank})))" + f"{prefix}_backend = prik_native_array_backend_for_descriptor({prefix}_obj, " + f"{self._native_array_handle_kind_constant(handle)}, {handle.array.rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({handle.array.rank})), {cfi_type}, " + f"{self._native_array_expected_element_size(plan)})" ) ), - CExpressionStatement(CodeExpression(f"if ({prefix}_handle == NULL) return NULL")), + CExpressionStatement(CodeExpression(f"if ({prefix}_backend == NULL) return NULL")), *( ( CExpressionStatement( - CodeExpression(f"{prefix}_descriptor = (CFI_cdesc_t *){prefix}_handle->descriptor") + CodeExpression( + f"{prefix}_descriptor = (CFI_cdesc_t *)prik_native_array_backend_owned_descriptor(" + f"{prefix}_backend)" + ) ), + CExpressionStatement(CodeExpression(f"if ({prefix}_descriptor == NULL) return NULL")), ) if materialize_descriptor else () ), ) - def _pointer_association_source_nodes( - self, - plan: ArgumentTransferPlan | ResultPlan | ModuleVariablePlan | DerivedFieldPlan, - ) -> tuple[CDeclaration | CExpressionStatement, ...]: - """Establish one call-local pointer descriptor from validated source facts.""" - handle = plan.native_array_handle - rank = handle.array.rank - cfi_type = self._pointer_association_cfi_type(plan) - expected_fields = 3 + 3 * rank - nodes: list[CDeclaration | CExpressionStatement] = [ - CDeclaration("source_item", "PyObject *", CodeExpression("NULL")), - CDeclaration("source_storage", f"CFI_CDESC_T({rank})"), - CDeclaration("source_descriptor", "CFI_cdesc_t *", CodeExpression("NULL")), - CDeclaration("source_base_addr", "void *", CodeExpression("NULL")), - CDeclaration("source_elem_len", "size_t", CodeExpression("0")), - CDeclaration("source_descriptor_rank", "CFI_rank_t", CodeExpression("0")), - CDeclaration(f"source_extents[{rank}]", "CFI_index_t"), - *( - CDeclaration(f"source_{label}_{axis}", "CFI_index_t", CodeExpression("0")) - for axis in range(rank) - for label in ("lower_bound", "extent", "stride_multiplier") - ), - CDeclaration("source_establish_status", "int", CodeExpression("CFI_SUCCESS")), - CExpressionStatement( - CodeExpression( - f"if (!PyTuple_Check(source_packed) || PyTuple_GET_SIZE(source_packed) != {expected_fields}) {{ " - f'PyErr_SetString(PyExc_TypeError, "pointer association requires {expected_fields} ' - 'descriptor facts"); return NULL; }' - ) - ), - *self._pointer_association_fact_nodes("source_base_addr", 0, pointer=True), - *self._pointer_association_fact_nodes("source_elem_len", 1, unsigned=True), - *self._pointer_association_fact_nodes("source_descriptor_rank", 2), - ] - for axis in range(rank): - offset = 3 + 3 * axis - nodes.extend( - ( - *self._pointer_association_fact_nodes(f"source_lower_bound_{axis}", offset), - *self._pointer_association_fact_nodes(f"source_extent_{axis}", offset + 1), - *self._pointer_association_fact_nodes(f"source_stride_multiplier_{axis}", offset + 2), - CExpressionStatement(CodeExpression(f"source_extents[{axis}] = source_extent_{axis}")), - ) - ) - nodes.extend( - ( - CExpressionStatement( - CodeExpression( - f"if (source_descriptor_rank != {rank}) {{ PyErr_Format(PyExc_ValueError, " - f'"pointer association source rank %d does not match destination rank {rank}", ' - "(int)source_descriptor_rank); return NULL; }" - ) - ), - CExpressionStatement( - CodeExpression( - "source_establish_status = CFI_establish((CFI_cdesc_t *)&source_storage, " - f"source_base_addr, CFI_attribute_pointer, {cfi_type}, source_elem_len, " - f"{rank}, source_extents)" - ) - ), - CExpressionStatement( - CodeExpression( - "if (source_establish_status != CFI_SUCCESS) { " - 'PyErr_SetString(PyExc_RuntimeError, "failed to establish pointer association source"); ' - "return NULL; }" - ) - ), - ) - ) - for axis in range(rank): - nodes.extend( - ( - CExpressionStatement( - CodeExpression( - f"((CFI_cdesc_t *)&source_storage)->dim[{axis}].lower_bound = source_lower_bound_{axis}" - ) - ), - CExpressionStatement( - CodeExpression(f"((CFI_cdesc_t *)&source_storage)->dim[{axis}].extent = source_extent_{axis}") - ), - CExpressionStatement( - CodeExpression( - f"((CFI_cdesc_t *)&source_storage)->dim[{axis}].sm = source_stride_multiplier_{axis}" - ) - ), - ) - ) - nodes.append(CExpressionStatement(CodeExpression("source_descriptor = (CFI_cdesc_t *)&source_storage"))) - return tuple(nodes) - - @staticmethod - def _pointer_association_fact_nodes( - target: str, - index: int, - *, - pointer: bool = False, - unsigned: bool = False, - ) -> tuple[CExpressionStatement, ...]: - """Decode one pointer-association descriptor fact.""" - if pointer: - conversion = "(void *)PyLong_AsVoidPtr(source_item)" - error = f"{target} == NULL && PyErr_Occurred()" - elif unsigned: - conversion = "(size_t)PyLong_AsUnsignedLongLong(source_item)" - error = "PyErr_Occurred()" - else: - conversion = "PyLong_AsLongLong(source_item)" - error = "PyErr_Occurred()" - return ( - CExpressionStatement(CodeExpression(f"source_item = PyTuple_GET_ITEM(source_packed, {index})")), - CExpressionStatement(CodeExpression(f"{target} = {conversion}")), - CExpressionStatement(CodeExpression(f"if ({error}) return NULL")), - ) - - @staticmethod - def _pointer_association_cfi_type( - plan: ArgumentTransferPlan | ResultPlan | ModuleVariablePlan | DerivedFieldPlan, - ) -> str: - """Return the completed standard-descriptor type for pointer assignment.""" - if isinstance(plan, DerivedFieldPlan): - if plan.string_element: - return "CFI_type_char" - elif plan.datatype_family is DatatypeFamily.STRING: - return "CFI_type_char" - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).cfi_type_spelling - - def _native_array_descriptor_record_nodes( - self, - rank: int, - descriptor_name: str, - *, - return_target: str | None = None, - ) -> tuple[CDeclaration | CExpressionStatement | CIf | CReturn, ...]: - """Decode a standard C descriptor into the runtime's mapping protocol.""" - failure_return = CReturn() if return_target is not None else CReturn(CodeExpression("NULL")) - nodes: list[CDeclaration | CExpressionStatement | CIf | CReturn] = [ - CDeclaration("dimensions", "PyObject *", CodeExpression(f"PyList_New({rank})")), - CIf(CodeExpression("dimensions == NULL"), body=(failure_return,)), - ] - for axis in range(rank): - item = f"dimension_{axis}" - nodes.extend( - ( - CDeclaration( - item, - "PyObject *", - CodeExpression( - f'Py_BuildValue("{{sL,sL,sL}}", "lower_bound", ' - f'(long long){descriptor_name}->dim[{axis}].lower_bound, "extent", ' - f'(long long){descriptor_name}->dim[{axis}].extent, "sm", ' - f"(long long){descriptor_name}->dim[{axis}].sm)" - ), - ), - CIf( - CodeExpression(f"{item} == NULL"), - body=( - CExpressionStatement(CodeExpression("Py_DECREF(dimensions)")), - failure_return, - ), - ), - CExpressionStatement(CodeExpression(f"PyList_SET_ITEM(dimensions, {axis}, {item})")), - ) - ) - nodes.extend( - ( - CDeclaration( - "descriptor_record", - "PyObject *", - CodeExpression( - f'Py_BuildValue("{{sK,sK,si,sO}}", "base_addr", ' - f'(unsigned long long)(uintptr_t){descriptor_name}->base_addr, "elem_len", ' - f'(unsigned long long){descriptor_name}->elem_len, "rank", ' - f'(int){descriptor_name}->rank, "dim", dimensions)' - ), - ), - CExpressionStatement(CodeExpression("Py_DECREF(dimensions)")), - ( - CExpressionStatement(CodeExpression(f"{return_target} = descriptor_record")) - if return_target is not None - else CReturn(CodeExpression("descriptor_record")) - ), - *((CReturn(),) if return_target is not None else ()), - ) - ) - return tuple(nodes) - def _owned_native_array_deallocate_nodes( self, result: ResultPlan, @@ -5212,7 +5230,7 @@ def _owned_native_array_shape_mutation_body( nodes: list[CDeclaration | CExpressionStatement | CIf | CReturn] = [ CDeclaration("owner_obj", "PyObject *"), *(CDeclaration(name, "PyObject *") for name in extent_objects), - CDeclaration("owner_handle", "prik_native_array_handle *", CodeExpression("NULL")), + CDeclaration("owner_backend", "prik_native_array_backend *", CodeExpression("NULL")), CDeclaration("owner_descriptor", "CFI_cdesc_t *", CodeExpression("NULL")), CDeclaration(f"lower_bounds[{rank}]", "CFI_index_t"), CDeclaration(f"upper_bounds[{rank}]", "CFI_index_t"), @@ -5222,13 +5240,19 @@ def _owned_native_array_shape_mutation_body( ), CExpressionStatement( CodeExpression( - "owner_handle = prik_native_array_handle_from_capsule(owner_obj, " - f"{self._native_array_handle_kind_constant(handle)}, {rank}, {cfi_type}, " - f"{self._native_array_expected_element_size(result)}, sizeof(CFI_CDESC_T({rank})))" + "owner_backend = prik_native_array_backend_for_descriptor(owner_obj, " + f"{self._native_array_handle_kind_constant(handle)}, {rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({rank})), {cfi_type}, " + f"{self._native_array_expected_element_size(result)})" ) ), - CExpressionStatement(CodeExpression("if (owner_handle == NULL) return NULL")), - CExpressionStatement(CodeExpression("owner_descriptor = (CFI_cdesc_t *)owner_handle->descriptor")), + CExpressionStatement(CodeExpression("if (owner_backend == NULL) return NULL")), + CExpressionStatement( + CodeExpression( + "owner_descriptor = (CFI_cdesc_t *)prik_native_array_backend_owned_descriptor(owner_backend)" + ) + ), + CExpressionStatement(CodeExpression("if (owner_descriptor == NULL) return NULL")), ] for axis, item in enumerate(extent_objects): nodes.extend( @@ -5291,15 +5315,14 @@ def _owned_native_array_resize_release_nodes( ), ) - def _owned_native_array_operation_name( + def _owned_native_array_dispatch_name( self, _function: FunctionPlan | None, result: ArgumentTransferPlan | ResultPlan, - operation: NativeArrayOperation, ) -> str: - """Return one stable private operation symbol.""" + """Return one stable private dispatcher symbol.""" owner = re.sub(r"\W", "_", result.owner_path).casefold() - return f"prik_owned_{owner}_{operation.value}" + return f"prik_owned_{owner}_dispatch" def _owned_native_array_bridge_operation_name( self, @@ -5311,15 +5334,6 @@ def _owned_native_array_bridge_operation_name( result.owner_path, f"native_array:owned:{operation.value}" ).symbol_name - def _owned_native_array_operation_def_name( - self, - function: FunctionPlan | None, - result: ArgumentTransferPlan | ResultPlan, - operation: NativeArrayOperation, - ) -> str: - """Return the private PyMethodDef symbol for one operation.""" - return f"{self._owned_native_array_operation_name(function, result, operation)}_def" - def _default_native_array_binder_name(self, argument: ArgumentTransferPlan) -> str: """Return one private lazy descriptor-attachment callable name.""" owner = re.sub(r"\W", "_", argument.owner_path).casefold() @@ -5584,8 +5598,8 @@ def _lower_module_getter_borrowed_array_view(self, plan: ModuleVariablePlan) -> numpy_itemsize = "(int)itemsize" else: scalar = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) - element_size = f"sizeof({scalar.c_spelling})" - numpy_type = str(scalar.numpy_type_macro) + element_size = f"sizeof({scalar.array_c_spelling})" + numpy_type = str(scalar.array_numpy_type) numpy_itemsize = "0" owner = self._module_native_array_owner_name(plan) width = ("itemsize",) if character else () @@ -5634,13 +5648,14 @@ def _lower_module_getter_borrowed_array_view(self, plan: ModuleVariablePlan) -> ) def _lower_module_getter_native_array_handle(self, plan: ModuleVariablePlan) -> tuple[CFunction, ...]: - """Create one stable borrowed runtime handle from planned module operations.""" + """Create one stable borrowed runtime handle from its planned dispatcher.""" handle = plan.native_array_handle if handle is None or handle.array.rank is None: raise ValueError(f"Module native array handle {plan.owner_path!r} is incomplete") cache = self._module_native_array_cache_name(plan) owner = self._module_native_array_owner_name(plan) prefix = f"{cache}_build" + dispatch = self._module_native_array_dispatch_name(plan) nodes: list[CDeclaration | CExpressionStatement | CIf | CReturn] = [ CIf( CodeExpression(f"{cache} != NULL"), @@ -5649,62 +5664,78 @@ def _lower_module_getter_native_array_handle(self, plan: ModuleVariablePlan) -> CReturn(CodeExpression(cache)), ), ), - CDeclaration(f"{prefix}_ops", "PyObject *", CodeExpression("PyDict_New()")), - CDeclaration(f"{prefix}_operation", "PyObject *", CodeExpression("NULL")), + CDeclaration( + f"{prefix}_capabilities", + "PyObject *", + CodeExpression("NULL"), + ), + CDeclaration( + f"{prefix}_invoke", + "PyObject *", + CodeExpression("NULL"), + ), CDeclaration(f"{prefix}_runtime", "PyObject *", CodeExpression("NULL")), CDeclaration(f"{prefix}_helper", "PyObject *", CodeExpression("NULL")), - CIf(CodeExpression(f"{prefix}_ops == NULL"), body=(CReturn(CodeExpression("NULL")),)), - ] - for operation in handle.operations: - definition = self._module_native_array_operation_def_name(plan, operation) - nodes.extend( + *self._module_native_array_backend_declaration_nodes(plan, prefix), + CExpressionStatement( + CodeExpression( + f"{prefix}_capabilities = {self._native_array_capabilities_expression(handle.operations)}" + ) + ), + CIf(CodeExpression(f"{prefix}_capabilities == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CExpressionStatement(CodeExpression(f"{prefix}_invoke = PyCFunction_NewEx(&{dispatch}_def, NULL, NULL)")), + CIf( + CodeExpression(f"{prefix}_invoke == NULL"), + body=( + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), + CReturn(CodeExpression("NULL")), + ), + ), + *( ( CExpressionStatement( - CodeExpression(f"{prefix}_operation = PyCFunction_NewEx(&{definition}, NULL, NULL)") - ), - CIf( - CodeExpression(f"{prefix}_operation == NULL"), - body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), - CReturn(CodeExpression("NULL")), - ), + CodeExpression(f"{prefix}_native_backend = {self._module_native_array_backend_capsule(plan)}") ), CIf( - CodeExpression( - f'PyDict_SetItemString({prefix}_ops, "{operation.value}", {prefix}_operation) < 0' - ), + CodeExpression(f"{prefix}_native_backend == NULL"), body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_operation)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), CReturn(CodeExpression("NULL")), ), ), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_operation)")), ) - ) - nodes.extend( - ( + if self._uses_module_descriptor_backend(plan) + else () + ), + ] + nodes.extend( + ( CExpressionStatement( CodeExpression(f'{prefix}_runtime = PyImport_ImportModule("prik.runtime.handles")') ), CIf( CodeExpression(f"{prefix}_runtime == NULL"), body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), + *self._module_native_array_backend_release_nodes(plan, prefix), CReturn(CodeExpression("NULL")), ), ), CExpressionStatement( CodeExpression( f"{prefix}_helper = PyObject_GetAttrString({prefix}_runtime, " - '"_native_array_handle_from_generated_ops")' + '"_native_array_handle_from_generated_dispatch")' ) ), CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_runtime)")), CIf( CodeExpression(f"{prefix}_helper == NULL"), body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), + *self._module_native_array_backend_release_nodes(plan, prefix), CReturn(CodeExpression("NULL")), ), ), @@ -5717,15 +5748,19 @@ def _lower_module_getter_native_array_handle(self, plan: ModuleVariablePlan) -> semantic_type_name=plan.semantic_type_name, datatype_family=plan.datatype_family, rank=handle.array.rank, - ops=f"{prefix}_ops", + invoke=f"{prefix}_invoke", + capabilities=f"{prefix}_capabilities", owner=f"{owner} != NULL ? {owner} : Py_None", descriptor_ownership="borrowed", + native_backend=self._module_native_array_backend_capsule_name(plan, prefix), extraction_action=handle.extraction_action.value, ) ) ), CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_helper)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), + *self._module_native_array_backend_release_nodes(plan, prefix), CIf(CodeExpression(f"{cache} == NULL"), body=(CReturn(CodeExpression("NULL")),)), CExpressionStatement(CodeExpression(f"Py_INCREF({cache})")), CReturn(CodeExpression(cache)), @@ -6727,9 +6762,176 @@ def _lower_argument_required_array_storage( self._array_validation_statement(plan, names), *self._array_shape_checks(plan, context, array), ] + if self._array_crosses_as_descriptor(plan): + if plan.transformations: + return tuple(nodes) + nodes.extend(self._numpy_descriptor_nodes(plan, names, numpy_validated=True)) + return tuple(nodes) nodes.extend(self._array_extraction_nodes(plan, names, array)) return tuple(nodes) + def _numpy_descriptor_nodes( + self, + plan: ArgumentTransferPlan, + names: _CArgumentNames, + *, + numpy_validated: bool = False, + ) -> tuple[CComment | CExpressionStatement | CIf, ...]: + """Reach one descriptor dummy from whichever source the caller supplied. + + A handle already has a descriptor, and it is valid only inside its own + entry point, so what is recorded here is the backend and the chain + enters it. A NumPy array has no descriptor, so one is made over its + storage as it stands, in this frame, which outlives the call. Both + arrive at the same slot, and the callee cannot tell them apart. + """ + array = plan.array + if array is None: + raise ValueError(f"Array argument {plan.owner_path!r} is missing its handoff") + prefix = names.value_name + declared = self._declared_character_width(plan) + width_guard: tuple = () + if declared: + width_guard = ( + CComment("A character dummy is matched on its declared width."), + CIf( + CodeExpression(f"PyArray_ITEMSIZE((PyArrayObject *){names.object_name}) != {declared}"), + body=( + CExpressionStatement( + CodeExpression( + "PyErr_Format(PyExc_TypeError, " + f"\"{plan.binding.python_name} does not match expected dtype dtype('S%d')\", " + f"{declared})" + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + ) + describe: tuple = ( + CComment("No descriptor of its own, so one is made over the array as it is."), + *(() if numpy_validated else (self._array_validation_statement(plan, names),)), + *width_guard, + CIf( + CodeExpression( + f"{self.NUMPY_DESCRIPTOR_BUILDER}((CFI_cdesc_t *)&{prefix}_parent, " + f"(CFI_cdesc_t *)&{prefix}_section, (PyArrayObject *){names.object_name}, " + f'{self._native_array_cfi_type(plan)}, "{plan.binding.python_name}") < 0' + ), + body=(CReturn(CodeExpression("NULL")),), + ), + CExpressionStatement(CodeExpression(f"{prefix} = (CFI_cdesc_t *)&{prefix}_section")), + *( + ( + CExpressionStatement( + CodeExpression( + f"{names.runtime_rank_name} = (int64_t)PyArray_NDIM((PyArrayObject *){names.object_name})" + ) + ), + ) + if plan.array.runtime_rank_role is not None + else () + ), + *( + CIf( + CodeExpression(f"PyArray_NDIM((PyArrayObject *){names.object_name}) > {axis}"), + body=( + CExpressionStatement( + CodeExpression( + f"{name} = (int64_t)PyArray_DIM((PyArrayObject *){names.object_name}, {axis})" + ) + ), + ), + ) + for axis, name in enumerate(names.extent_names) + ), + ) + if not self._takes_array_handle(plan): + return describe + capsule = f"{prefix}_capsule" + backend = self._descriptor_backend_local(names) + return ( + CExpressionStatement( + CodeExpression(f'{capsule} = PyObject_GetAttrString({names.object_name}, "_native_backend")') + ), + CExpressionStatement(CodeExpression(f"if ({capsule} == NULL) {{ PyErr_Clear(); }}")), + CIf( + CodeExpression(f"{capsule} != NULL && {capsule} != Py_None"), + body=( + CComment("A handle's descriptor is the runtime's; the chain enters it."), + CExpressionStatement( + CodeExpression( + f"{backend} = prik_native_array_backend_for_actual({capsule}, " + f"{plan.array.minimum_rank}, {plan.array.maximum_rank}, " + f"{self._native_array_cfi_type(plan)}, " + f"{self._native_array_expected_element_size(plan)}, " + f'"{plan.native_array_actual.dtype}", "{plan.binding.python_name}")' + ) + ), + CExpressionStatement(CodeExpression(f"Py_DECREF({capsule})")), + CIf(CodeExpression(f"{backend} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CComment("Its state and extents are read while its own descriptor is live."), + CExpressionStatement(CodeExpression(f"{prefix}_extents_out.rank = (int){backend}->rank")), + CExpressionStatement(CodeExpression(f"{prefix}_extents_out.present = 0")), + CExpressionStatement(CodeExpression(f"{prefix}_extents_out.contiguous = 0")), + CExpressionStatement(CodeExpression(f"{prefix}_extents_out.elem_len = 0")), + CExpressionStatement(CodeExpression(f"{prefix}_extents_out.extents = {prefix}_extents")), + CExpressionStatement( + CodeExpression( + f"{backend}->with_descriptor({backend}->context, " + f"{self.ARRAY_EXTENTS_READER}, &{prefix}_extents_out)" + ) + ), + *self._descriptor_character_width_guard(plan, prefix), + CIf( + CodeExpression(f"!{prefix}_extents_out.present"), + body=( + CExpressionStatement( + CodeExpression( + "PyErr_SetString(PyExc_ValueError, " + f"{backend}->descriptor_kind == PRIK_NATIVE_ARRAY_KIND_POINTER " + '? "pointer handle is unassociated and cannot be passed as an array actual" ' + ': "allocatable handle is unallocated and cannot be passed as an array actual")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + *( + ( + CIf( + CodeExpression(f"!{prefix}_extents_out.contiguous"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_ValueError, "pointer handle target is ' + f'noncontiguous and cannot be passed to argument {plan.binding.python_name}")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + ) + if array.contiguous is True + else () + ), + *( + (CExpressionStatement(CodeExpression(f"{names.runtime_rank_name} = (int64_t){backend}->rank")),) + if plan.array.runtime_rank_role is not None + else () + ), + *( + CExpressionStatement(CodeExpression(f"{name} = {prefix}_extents[{axis}]")) + for axis, name in enumerate(names.extent_names) + ), + ), + else_body=( + CExpressionStatement(CodeExpression(f"Py_XDECREF({capsule})")), + *describe, + ), + ), + ) + def _ordinary_array_argument_declarations( self, plan: ArgumentTransferPlan, @@ -6739,6 +6941,17 @@ def _ordinary_array_argument_declarations( array = plan.array if array is None: raise ValueError(f"Array argument {plan.owner_path!r} is missing its handoff") + if self._array_crosses_as_descriptor(plan): + return self._descriptor_array_argument_declarations(plan, names, array) + return self._raw_array_argument_declarations(plan, names, array) + + def _raw_array_argument_declarations( + self, + plan: ArgumentTransferPlan, + names: _CArgumentNames, + array: ArrayHandoffPlan, + ) -> tuple[CDeclaration, ...]: + """Declare the address and metadata selected by a raw-array plan.""" declarations = [ CDeclaration(names.object_name, "PyObject *"), CDeclaration(names.value_name, "void *", CodeExpression("NULL")), @@ -6763,6 +6976,41 @@ def _ordinary_array_argument_declarations( declarations.append(CDeclaration(names.itemsize_name, "int64_t", CodeExpression("0"))) return tuple(declarations) + def _descriptor_array_argument_declarations( + self, + plan: ArgumentTransferPlan, + names: _CArgumentNames, + array: ArrayHandoffPlan, + ) -> tuple[CDeclaration, ...]: + """Declare call-local storage for a NumPy or native descriptor source.""" + descriptor_rank = "PRIK_MAX_ARRAY_RANK" if array.rank is None else str(array.rank) + declarations = [ + CDeclaration(names.object_name, "PyObject *"), + CDeclaration(names.value_name, "CFI_cdesc_t *", CodeExpression("NULL")), + CDeclaration( + self._descriptor_backend_local(names), + "prik_native_array_backend *", + CodeExpression("NULL"), + ), + CDeclaration(f"{names.value_name}_capsule", "PyObject *", CodeExpression("NULL")), + CDeclaration(f"{names.value_name}_parent", f"CFI_CDESC_T({descriptor_rank})"), + CDeclaration(f"{names.value_name}_section", f"CFI_CDESC_T({descriptor_rank})"), + CDeclaration(f"{names.value_name}_extents[{descriptor_rank}]", "int64_t", CodeExpression("{0}")), + CDeclaration(f"{names.value_name}_extents_out", self.ARRAY_EXTENTS_RECORD), + *(CDeclaration(name, "int64_t", CodeExpression("0")) for name in names.extent_names), + ] + if plan.transformations: + declarations.append( + CDeclaration( + self._array_transformation_temp_name(names), + "PyObject *", + CodeExpression("NULL"), + ) + ) + if array.runtime_rank_role is not None: + declarations.append(CDeclaration(names.runtime_rank_name, "int64_t", CodeExpression("0"))) + return tuple(declarations) + @staticmethod def _array_dense_actual_declarations( array: ArrayHandoffPlan, @@ -6787,6 +7035,24 @@ def _lower_argument_required_array_actual( names = context.arguments[plan.owner_path] prefix = names.value_name array_object = f"(PyArrayObject *){names.object_name}" + if self._array_crosses_as_descriptor(plan): + # One dummy, one descriptor; the source only decides where it comes + # from, and the shape checks apply to whichever supplied it. + return ( + *self._ordinary_array_argument_declarations(plan, names), + *self._numpy_descriptor_nodes(plan, names), + *self._descriptor_array_shape_checks(plan, context, names), + ) + if not self._takes_array_handle(plan): + outlined = self._outlined_array_bind_nodes(plan, context, names) + if outlined is not None: + return (*self._ordinary_array_argument_declarations(plan, names), *outlined) + return ( + *self._ordinary_array_argument_declarations(plan, names), + self._array_validation_statement(plan, names), + *self._array_shape_checks(plan, context, array_object), + *self._array_extraction_nodes(plan, names, array_object), + ) direct_nodes = ( self._array_validation_statement(plan, names, object_kind_checked=True), *self._array_shape_checks(plan, context, array_object), @@ -6797,7 +7063,6 @@ def _lower_argument_required_array_actual( if outlined is not None: return (*self._ordinary_array_argument_declarations(plan, names), *outlined) handle_nodes = ( - CDeclaration(f"{prefix}_shape", "PyObject *", CodeExpression("NULL")), CDeclaration(f"{prefix}_actual", "prik_array_actual"), *self._native_array_actual_call_nodes(plan, context, names), *self._native_array_actual_unpack_nodes(plan, names), @@ -6827,7 +7092,6 @@ def _outlined_array_bind_nodes( if fixed is None: return None array = plan.array - actual = plan.native_array_actual rank = array.rank prefix = names.value_name numpy_type, python_type = self._array_dtype_selectors(plan, array) @@ -6843,19 +7107,8 @@ def _outlined_array_bind_nodes( int(array.contiguous is True), int(plan.binding.writable), f'"{python_type}"', - f'"{actual.dtype}"', f'"{plan.binding.python_name}"', - "NULL" if actual.order is None else f'"{actual.order}"', array.flat_axis if array.flatten_python_storage else rank - 1, - int(actual.writable), - int(actual.require_native_byte_order), - int(actual.require_aligned), - 0, - 0, - 0, - int(actual.require_contiguous), - int(actual.flatten_storage), - self._native_array_actual_flat_axis(actual), ) ) # Declarations hoist above argument parsing, so a required extent that @@ -6867,11 +7120,16 @@ def _outlined_array_bind_nodes( CExpressionStatement(CodeExpression(f"{prefix}_bind_fixed[{axis}] = {value}")) for axis, value in enumerate(fixed) ), - CExpressionStatement( - CodeExpression( - f"if (prik_bind_array({names.object_name}, {selectors}, " - f"{prefix}_bind_fixed, &{names.value_name}, {prefix}_bind_extents) < 0) return NULL" - ) + *self._array_actual_backend_nodes( + plan, + context, + names, + CExpressionStatement( + CodeExpression( + f"if (prik_bind_array({names.object_name}, {selectors}, " + f"{prefix}_bind_fixed, &{names.value_name}, {prefix}_bind_extents) < 0) return NULL" + ) + ), ), *( CExpressionStatement(CodeExpression(f"{names.extent_names[axis]} = {prefix}_bind_extents[{axis}]")) @@ -6879,6 +7137,224 @@ def _outlined_array_bind_nodes( ), ) + def _array_actual_backend_nodes( + self, + plan: ArgumentTransferPlan, + context: _CFunctionContext, + names: _CArgumentNames, + fallback: CExpressionStatement, + ) -> tuple: + """Take an array handle's storage from its backend when it publishes one. + + A handle standing for native storage names it through its backend, so the + address and extents an ordinary dummy needs are read here rather than + asked for through the runtime one operation at a time. Anything else -- + an ndarray, or a handle with no backend -- takes the shared binder. + """ + function = context.function + if ( + not self._reads_native_descriptors + or function is None + or not any( + argument.owner_path == plan.owner_path + for _function, argument in self._array_actual_handle_arguments_for(function, context) + ) + ): + return (fallback,) + prefix = names.value_name + rank = plan.array.rank + record = self._array_actual_reader_record_name(function, plan) + reader = self._array_actual_reader_name(function, plan) + capsule = f"{prefix}_actual_capsule" + backend = f"{prefix}_actual_backend" + found = f"{prefix}_actual_found" + actual = plan.native_array_actual + contiguous_check: tuple = () + if actual is not None and (actual.require_contiguous or plan.array.contiguous is True): + contiguous_check = ( + CIf( + CodeExpression(f"!{found}.contiguous"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_ValueError, "pointer handle target is noncontiguous ' + 'and cannot use the pointer/shape array-actual handoff")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + ) + checks: list = [ + CComment("Each condition is reported the way the runtime reports it,"), + CComment("so a handle reads alike whether or not it publishes a backend."), + CIf( + CodeExpression(f"{found}.refused == 2"), + body=( + CExpressionStatement( + CodeExpression( + f"PyErr_Format(PyExc_TypeError, \"%s handle dtype dtype('S%zu') does not " + f"match expected dtype dtype('S%d')\", " + f"{backend}->descriptor_kind == PRIK_NATIVE_ARRAY_KIND_POINTER " + f'? "pointer" : "allocatable", {found}.width, ' + f"{self._declared_character_width(plan)})" + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + CIf( + CodeExpression(f"!{found}.present"), + body=( + CExpressionStatement( + CodeExpression( + f"PyErr_SetString(PyExc_ValueError, " + f"{backend}->descriptor_kind == PRIK_NATIVE_ARRAY_KIND_POINTER " + f'? "pointer handle is unassociated and cannot be passed as an array actual" ' + f': "allocatable handle is unallocated and cannot be passed as an array actual")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + *contiguous_check, + ] + for axis in range(rank): + checks.append( + CIf( + CodeExpression( + f"{prefix}_bind_fixed[{axis}] >= 0 && {found}.extents[{axis}] " + f"!= (int64_t){prefix}_bind_fixed[{axis}]" + ), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_TypeError, "Argument {plan.binding.python_name} ' + f'has incompatible shape at axis %d", {axis})' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ) + ) + checks.extend( + ( + CExpressionStatement(CodeExpression(f"{names.value_name} = {found}.data")), + *( + CExpressionStatement(CodeExpression(f"{prefix}_bind_extents[{axis}] = {found}.extents[{axis}]")) + for axis in range(rank) + ), + ) + ) + return ( + CDeclaration(capsule, "PyObject *", CodeExpression("NULL")), + CDeclaration(backend, "prik_native_array_backend *", CodeExpression("NULL")), + CDeclaration(found, record), + CExpressionStatement(CodeExpression(f"{found}.present = 0")), + CExpressionStatement(CodeExpression(f"{found}.contiguous = 0")), + CExpressionStatement(CodeExpression(f"{found}.refused = 1")), + CExpressionStatement(CodeExpression(f"{found}.width = 0")), + CExpressionStatement( + CodeExpression(f'{capsule} = PyObject_GetAttrString({names.object_name}, "_native_backend")') + ), + CExpressionStatement(CodeExpression(f"if ({capsule} == NULL) {{ PyErr_Clear(); }}")), + CIf( + CodeExpression(f"{capsule} != NULL && {capsule} != Py_None"), + body=( + CExpressionStatement( + CodeExpression( + f"{backend} = prik_native_array_backend_for_actual({capsule}, " + f"{plan.array.minimum_rank}, {plan.array.maximum_rank}, " + f"{self._native_array_cfi_type(plan)}, " + f"{self._native_array_expected_element_size(plan)}, " + f'"{plan.native_array_actual.dtype}", "{plan.binding.python_name}")' + ) + ), + CExpressionStatement(CodeExpression(f"Py_DECREF({capsule})")), + CIf(CodeExpression(f"{backend} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CExpressionStatement( + CodeExpression(f"{backend}->with_descriptor({backend}->context, {reader}, &{found})") + ), + *checks, + ), + else_body=( + CExpressionStatement(CodeExpression(f"Py_XDECREF({capsule})")), + CIf( + CodeExpression(f"!PyArray_Check({names.object_name})"), + body=(self._native_array_actual_type_refusal(plan, names),), + ), + fallback, + ), + ), + ) + + @staticmethod + def _native_array_actual_type_refusal( + plan: ArgumentTransferPlan, + names: _CArgumentNames, + ) -> CExpressionStatement: + """Report a value that is neither an ndarray nor an accepted handle.""" + return CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_TypeError, "Expected a compatible {plan.native_array_actual.dtype} array or ' + f'native array handle for argument {plan.binding.python_name}. Received %s", ' + f"Py_TYPE({names.object_name})->tp_name); return NULL" + ) + ) + + def _inline_array_actual_handle_arguments(self, plan: ModulePlan): + """Return array arguments that take a handle through the shared record. + + These are the ones the outlined binder cannot serve -- an assumed-shape + dummy carries strides and bounds beyond a pointer and extents -- so + they fill the record instead. + """ + for function in self._functions(plan): + context = self._function_context(function) + for argument in function.arguments: + if self._inline_array_actual_fast_path(argument) and ( + self._outlined_array_bind_fixed_extents(argument, context) is None + ): + yield argument + + @staticmethod + def _takes_array_handle(argument: ArgumentTransferPlan) -> bool: + """Return whether an ordinary array argument accepts an array handle.""" + actual = argument.native_array_actual + if actual is None or argument.array is None: + return False + return bool( + { + NativeArraySourceKind.ALLOCATABLE_HANDLE, + NativeArraySourceKind.POINTER_HANDLE, + }.intersection(actual.accepted_sources) + ) + + def _inline_array_actual_fast_path(self, plan: ArgumentTransferPlan) -> bool: + """Report whether this argument can fill its record from a descriptor.""" + if not self._reads_native_descriptors or not self._takes_array_handle(plan): + return False + actual = plan.native_array_actual + return not (actual.flatten_storage or plan.array.flatten_python_storage) + + def _array_actual_handle_arguments_for(self, function: FunctionPlan, context: _CFunctionContext): + """Return this function's array arguments an array handle may be passed to.""" + for argument in function.arguments: + actual = argument.native_array_actual + if actual is None or argument.array is None or argument.array.rank is None: + continue + if not { + NativeArraySourceKind.ALLOCATABLE_HANDLE, + NativeArraySourceKind.POINTER_HANDLE, + }.intersection(actual.accepted_sources): + continue + if self._outlined_array_bind_fixed_extents(argument, context) is None: + continue + flattens = actual.flatten_storage or argument.array.flatten_python_storage + if flattens and self._flattened_reader_axis(argument) is None: + continue + yield function, argument + def _outlined_array_bind_fixed_extents( self, plan: ArgumentTransferPlan, @@ -6975,68 +7451,154 @@ def _native_array_actual_call_nodes( context: _CFunctionContext, names: _CArgumentNames, ) -> tuple[CExpressionStatement, ...]: - """Call the shared normal-array native-handle slow path.""" + """Read a normal-array native handle through its descriptor backend.""" actual = plan.native_array_actual if actual is None: return () prefix = names.value_name - layout = "NULL" if actual.order is None else f'"{actual.order}"' - nodes = [ - *self._native_array_actual_shape_object_nodes(plan, names), - *self._native_array_actual_shape_nodes(plan, context, names), - CExpressionStatement( - CodeExpression( - f'if (prik_array_actual_unpack({names.object_name}, "{actual.dtype}", ' - f"{self._native_array_actual_expected_rank(actual)}, {prefix}_shape, {layout}, " - f"{int(actual.writable)}, {int(actual.require_native_byte_order)}, {int(actual.require_aligned)}, " - f"{int(plan.array.runtime_rank_role is not None)}, " - f"{int(plan.array.itemsize_role is not None)}, {int(bool(plan.array.stride_roles))}, " - f"{int(actual.require_contiguous)}, {int(actual.flatten_storage)}, " - f"{self._native_array_actual_flat_axis(actual)}, &{prefix}_actual) < 0) {{ " - f"Py_DECREF({prefix}_shape); return NULL; }}" - ) - ), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_shape)")), - ] - return tuple(nodes) + backend_nodes = self._native_array_actual_backend_nodes(plan, context, names) + refuse = self._native_array_actual_type_refusal(plan, names) + if not backend_nodes: + return (refuse,) + return ( + *backend_nodes, + CIf(CodeExpression(f"{prefix}_actual.data == NULL"), body=(refuse,)), + ) - def _native_array_actual_shape_object_nodes( + def _native_array_actual_backend_nodes( self, plan: ArgumentTransferPlan, + context: _CFunctionContext, names: _CArgumentNames, - ) -> tuple[CExpressionStatement, ...]: - """Create the expected-shape object consumed by the runtime helper.""" - actual = plan.native_array_actual - if actual is None: + ) -> tuple: + """Fill the array-actual record from a handle's backend when it has one. + + A handle standing for native storage names it through its backend, so the + record is filled here rather than assembled by asking the runtime one + operation at a time. Anything without a compatible backend is refused. + """ + if not self._inline_array_actual_fast_path(plan): return () prefix = names.value_name + capsule = f"{prefix}_backend_capsule" + backend = f"{prefix}_native_backend" + record = self._array_actual_struct_reader_record_name(plan) + found = f"{prefix}_backend_result" + refusals = ( + CIf( + CodeExpression(f"{found}.refused == 1"), + body=( + CExpressionStatement( + CodeExpression( + f"PyErr_SetString(PyExc_ValueError, " + f"{backend}->descriptor_kind == PRIK_NATIVE_ARRAY_KIND_POINTER " + f'? "pointer handle is unassociated and cannot be passed as an array actual" ' + f': "allocatable handle is unallocated and cannot be passed as an array actual")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + CIf( + CodeExpression(f"{found}.refused == 2"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_TypeError, "handle with %zu-byte elements does not match ' + f"expected dtype {plan.native_array_actual.dtype} for argument " + f'{plan.binding.python_name}", {found}.width)' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + CIf( + CodeExpression(f"{found}.refused == 3"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_ValueError, "pointer handle target is noncontiguous ' + 'and cannot use the pointer/shape array-actual handoff")' + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + CIf( + CodeExpression(f"{found}.refused == 4"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_TypeError, "native array handle descriptor rank %d is outside ' + f'the supported range {plan.array.minimum_rank}..{plan.array.maximum_rank}", ' + f"{found}.rank)" + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + ) return ( - CExpressionStatement(CodeExpression(f"{prefix}_shape = PyTuple_New({actual.rank})")), - CExpressionStatement(CodeExpression(f"if ({prefix}_shape == NULL) return NULL")), + CDeclaration(capsule, "PyObject *", CodeExpression("NULL")), + CDeclaration(backend, "prik_native_array_backend *", CodeExpression("NULL")), + CDeclaration(found, record), + CExpressionStatement(CodeExpression(f"{found}.actual = &{prefix}_actual")), + CExpressionStatement(CodeExpression(f"{found}.refused = 1")), + CExpressionStatement(CodeExpression(f"{found}.width = 0")), + CExpressionStatement(CodeExpression(f"{prefix}_actual.data = NULL")), + CExpressionStatement( + CodeExpression(f'{capsule} = PyObject_GetAttrString({names.object_name}, "_native_backend")') + ), + CExpressionStatement(CodeExpression(f"if ({capsule} == NULL) {{ PyErr_Clear(); }}")), + CIf( + CodeExpression(f"{capsule} != NULL && {capsule} != Py_None"), + body=( + CExpressionStatement( + CodeExpression( + f"{backend} = prik_native_array_backend_for_actual({capsule}, " + f"{plan.array.minimum_rank}, {plan.array.maximum_rank}, " + f"{self._native_array_cfi_type(plan)}, " + f"{self._native_array_expected_element_size(plan)}, " + f'"{plan.native_array_actual.dtype}", "{plan.binding.python_name}")' + ) + ), + CExpressionStatement(CodeExpression(f"Py_DECREF({capsule})")), + CIf(CodeExpression(f"{backend} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + CExpressionStatement( + CodeExpression( + f"{backend}->with_descriptor({backend}->context, " + f"{self._array_actual_struct_reader_name(plan)}, &{found})" + ) + ), + *refusals, + *self._native_array_actual_shape_checks(plan, context, names, found), + ), + else_body=( + CExpressionStatement(CodeExpression(f"Py_XDECREF({capsule})")), + CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_TypeError, "Expected a compatible numpy.ndarray or native ' + f"array handle for argument {plan.binding.python_name}. Received \", " + f"Py_TYPE({names.object_name})->tp_name)" + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), ) - @staticmethod - def _native_array_actual_expected_rank(actual: NativeArrayActualPlan) -> int: - """Return the runtime-helper rank selector selected by completed policy.""" - return actual.rank - - @staticmethod - def _native_array_actual_flat_axis(actual: NativeArrayActualPlan) -> int: - """Return the flattened contract axis marker consumed by the runtime helper.""" - return -1 if actual.flat_axis is None else actual.flat_axis - - def _native_array_actual_shape_nodes( + def _native_array_actual_shape_checks( self, plan: ArgumentTransferPlan, context: _CFunctionContext, names: _CArgumentNames, - ) -> tuple[CExpressionStatement, ...]: - """Build the planned expected-shape tuple for runtime validation.""" + record: str, + ) -> tuple[CIf, ...]: + """Validate a handle descriptor against every binding-known extent.""" actual = plan.native_array_actual array = plan.array - if actual is None or array is None: + if actual is None or array is None or array.rank is None: return () - prefix = names.value_name nodes = [] for axis, expression in enumerate(actual.shape): if ( @@ -7044,18 +7606,20 @@ def _native_array_actual_shape_nodes( or (actual.flatten_storage and axis == actual.flat_axis) or array.extent_evaluation[axis] == "bridge" ): - nodes.append(CExpressionStatement(CodeExpression("Py_INCREF(Py_None)"))) - item = "Py_None" - else: - expected = self._array_extent_expression(array, axis, expression, context) - item = f"PyLong_FromLongLong((long long)({expected}))" - nodes.append(CExpressionStatement(CodeExpression(f"PyTuple_SET_ITEM({prefix}_shape, {axis}, {item})"))) + continue + expected = self._array_extent_expression(array, axis, expression, context) nodes.append( - CExpressionStatement( - CodeExpression( - f"if (PyTuple_GET_ITEM({prefix}_shape, {axis}) == NULL) {{ " - f"Py_DECREF({prefix}_shape); return NULL; }}" - ) + CIf( + CodeExpression(f"{record}.logical_extents[{axis}] != (int64_t)({expected})"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_TypeError, "Argument {plan.binding.python_name} ' + f'has incompatible shape at axis %d", {axis})' + ) + ), + CReturn(CodeExpression("NULL")), + ), ) ) return tuple(nodes) @@ -7136,9 +7700,9 @@ def _numeric_array_dtype_selectors(plan: ArgumentTransferPlan) -> tuple[str, str ) return native.numpy_type_macro, native.python_type_name scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) - if scalar_type.numpy_type_macro is None or scalar_type.python_type_name is None: + if scalar_type.array_numpy_type is None or scalar_type.array_dtype_name is None: raise ValueError(f"Unsupported array element type {plan.semantic_type_name!r}") - return scalar_type.numpy_type_macro, scalar_type.python_type_name + return scalar_type.array_numpy_type, scalar_type.array_dtype_name @staticmethod def _array_rank_bounds(handoff: ArrayHandoffPlan) -> tuple[int, int]: @@ -7150,6 +7714,7 @@ def _array_rank_bounds(handoff: ArrayHandoffPlan) -> tuple[int, int]: ArrayPythonLayout.C_CONTIGUOUS: "PRIK_ARRAY_LAYOUT_C_CONTIGUOUS", ArrayPythonLayout.F_CONTIGUOUS: "PRIK_ARRAY_LAYOUT_F_CONTIGUOUS", ArrayPythonLayout.POSITIVE_STRIDED_F: "PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F", + ArrayPythonLayout.SIGNED_STRIDED_F: "PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F", ArrayPythonLayout.ANY_STRIDED: "PRIK_ARRAY_LAYOUT_ANY_STRIDED", } @@ -7188,6 +7753,32 @@ def _array_shape_checks( ) return tuple(checks) + def _descriptor_array_shape_checks( + self, + plan: ArgumentTransferPlan, + context: _CFunctionContext, + names: _CArgumentNames, + ) -> tuple[CExpressionStatement, ...]: + """Validate declared extents after either descriptor source has been read.""" + handoff = plan.array + if handoff is None or handoff.rank is None: + return () + checks = [] + for axis, expression in enumerate(handoff.shape): + if expression in {":", "::Strided", "Flat"} or handoff.extent_evaluation[axis] == "bridge": + continue + expected = self._array_extent_expression(handoff, axis, expression, context) + checks.append( + CExpressionStatement( + CodeExpression( + f"if ({names.extent_names[axis]} != (int64_t)({expected})) {{ " + f'PyErr_SetString(PyExc_TypeError, "Argument {plan.binding.python_name} has incompatible ' + f'shape at axis {axis}"); return NULL; }}' + ) + ) + ) + return tuple(checks) + @staticmethod def _array_actual_axis_expression(handoff, array: str, axis: int) -> str: """Map one contract axis to the runtime ndarray axis selected by the plan.""" @@ -7563,53 +8154,92 @@ def _lower_argument_native_array_handle( handle = plan.native_array_handle if handle is None: return () - if handle.handoff.abi is NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL: - return self._lower_argument_native_array_facts(plan, context) if handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR: return self._lower_argument_native_array_direct(plan, context) raise ValueError(f"Unsupported C native descriptor ABI for {plan.owner_path!r}: {handle.handoff.abi!r}") - def _lower_argument_native_array_facts( + def _inverted_descriptor_backend_nodes( self, plan: ArgumentTransferPlan, - context: _CFunctionContext, - ) -> tuple[CDeclaration | CExpressionStatement | CIf, ...]: - """Establish call-local CFI storage from validated descriptor facts.""" - handle = plan.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Native descriptor {plan.owner_path!r} has no concrete rank") - names = context.arguments[plan.owner_path] - rank = handle.array.rank + names: _CArgumentNames, + handle: NativeArrayHandlePlan, + fallback: tuple, + ) -> tuple: + """Reach this argument's descriptor by whichever route its handle offers. + + A handle standing for a module array or a field publishes a backend, and + the descriptor it names is built inside the consumer that makes the + call -- the only place a callee can change what the dummy stands for + (an allocatable's allocation, a pointer's association) and have that + reach the caller's entity. An unattached contract handle instead takes + the fallback that first gives it persistent descriptor storage. + """ prefix = names.value_name - nodes: list[CDeclaration | CExpressionStatement | CIf] = [ - self._native_descriptor_object_declaration(plan, names), - CDeclaration(f"{prefix}_storage", f"CFI_CDESC_T({rank})"), - CDeclaration(names.value_name, "CFI_cdesc_t *", CodeExpression("NULL")), - CDeclaration(f"{prefix}_base_addr", "void *", CodeExpression("NULL")), - CDeclaration(f"{prefix}_elem_len", "size_t", CodeExpression("0")), - CDeclaration(f"{prefix}_descriptor_rank", "CFI_rank_t", CodeExpression("0")), - CDeclaration(f"{prefix}_cfi_extents[{rank}]", "CFI_index_t"), - *( - CDeclaration(f"{prefix}_{label}_{axis}", "CFI_index_t", CodeExpression("0")) - for axis in range(rank) - for label in ("lower_bound", "descriptor_extent", "stride_multiplier") + capsule = f"{prefix}_backend_capsule" + backend = f"{prefix}_native_backend" + # Presence is otherwise decided by the packing helper, which only the + # other branch calls. A handle that published a backend was supplied, so + # an optional argument reaching this branch is present. + present = ( + ( + CComment("This handle was supplied, so an optional argument is present."), + CExpressionStatement(CodeExpression(f"{names.present_name} = {backend}")), + ) + if plan.entrypoint.pass_descriptor_presence + else () + ) + resolve = ( + CExpressionStatement( + CodeExpression(f'{capsule} = PyObject_GetAttrString({names.object_name}, "_native_backend")') ), - CDeclaration(f"{prefix}_establish_status", "int", CodeExpression("CFI_SUCCESS")), - *self._native_descriptor_helper_declarations(prefix), - *(self._native_descriptor_presence_declarations(plan, names)), - ] - nodes.extend( - self._native_descriptor_helper_call_nodes( - plan, - context, - names, - "_native_array_descriptor_argument_for_binding_positional", + CExpressionStatement(CodeExpression(f"if ({capsule} == NULL) {{ PyErr_Clear(); }}")), + CIf( + CodeExpression(f"{capsule} != NULL && {capsule} != Py_None"), + body=( + CExpressionStatement( + CodeExpression( + f"{backend} = prik_native_array_backend_for_descriptor({capsule}, " + f"{self._native_array_handle_kind_constant(handle)}, {handle.array.rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({handle.array.rank})), " + f"{self._native_array_cfi_type(plan)}, " + f"{self._native_array_expected_element_size(plan)})" + ) + ), + CExpressionStatement(CodeExpression(f"Py_DECREF({capsule})")), + CIf(CodeExpression(f"{backend} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + *present, + ), + else_body=( + CComment("No backend yet: attach storage to a fresh contract handle."), + CExpressionStatement(CodeExpression(f"Py_XDECREF({capsule})")), + *fallback, + ), + ), + ) + if plan.binding.optional_mode is OptionalMode.DESCRIPTOR: + absent = ( + self._absent_descriptor_placeholder_nodes(plan, names, handle, packed_owner=None) + if plan.entrypoint.pass_descriptor_presence + else () + ) + resolve = ( + CIf( + CodeExpression(f"{names.object_name} == Py_None"), + body=( + CComment("An absent optional has no handle or backend to enter."), + *absent, + ), + else_body=resolve, + ), ) + return ( + CDeclaration(capsule, "PyObject *", CodeExpression("NULL")), + CComment( + f"'{plan.binding.python_name}' may have its {_descriptor_binding_noun(handle)} changed by the callee." + ), + CComment("The backend supplies a live descriptor inside the consumer."), + *resolve, ) - nodes.extend(self._native_descriptor_presence_unpack_nodes(plan, names, 3 + 3 * rank)) - nodes.extend(self._native_descriptor_fact_unpack_nodes(plan, names)) - nodes.append(CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_packed)"))) - return tuple(nodes) def _lower_argument_native_array_direct( self, @@ -7631,8 +8261,8 @@ def _lower_argument_native_array_direct( self._native_descriptor_object_declaration(plan, names), CDeclaration(names.value_name, "CFI_cdesc_t *", CodeExpression("NULL")), CDeclaration( - f"{names.value_name}_native_handle", - "prik_native_array_handle *", + f"{names.value_name}_native_backend", + "prik_native_array_backend *", CodeExpression("NULL"), ), *self._native_descriptor_helper_declarations( @@ -7640,19 +8270,31 @@ def _lower_argument_native_array_direct( include_default_binder=binder_definition is not None, ), *(self._native_descriptor_presence_declarations(plan, names)), + *( + ( + CDeclaration(f"{prefix}_storage", f"CFI_CDESC_T({handle.array.rank})"), + CDeclaration(f"{prefix}_establish_status", "int", CodeExpression("CFI_SUCCESS")), + ) + if plan.entrypoint.pass_descriptor_presence + else () + ), ] - nodes.extend( + # A handle with no storage yet has no backend to enter, so storage is + # attached first and the backend that gives is what the chain enters. + attach: list[CDeclaration | CExpressionStatement | CIf] = [] + attach.extend( self._native_descriptor_helper_call_nodes( plan, context, names, - "_native_array_descriptor_handoff_for_binding_positional", + "_native_array_backend_for_binding_positional", default_binder_definition=binder_definition, ) ) - nodes.extend(self._native_descriptor_presence_unpack_nodes(plan, names, 1)) - nodes.extend(self._native_descriptor_pointer_unpack_nodes(plan, names)) - nodes.append(CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_packed)"))) + attach.extend(self._native_descriptor_presence_unpack_nodes(plan, names, 1)) + attach.extend(self._native_descriptor_pointer_unpack_nodes(plan, names)) + attach.append(CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_packed)"))) + nodes.extend(self._inverted_descriptor_backend_nodes(plan, names, handle, tuple(attach))) return tuple(nodes) def _native_descriptor_object_declaration( @@ -7673,7 +8315,7 @@ def _native_descriptor_helper_declarations( """Return binding-local Python objects used by one runtime helper call.""" declarations = tuple( CDeclaration(f"{prefix}_{suffix}", "PyObject *", CodeExpression("NULL")) - for suffix in ("runtime", "helper", "shape", "packed", "item") + for suffix in ("runtime", "helper", "packed", "item") ) if include_default_binder: return (*declarations, CDeclaration(f"{prefix}_default_binder", "PyObject *", CodeExpression("NULL"))) @@ -7713,11 +8355,6 @@ def _native_descriptor_helper_call_nodes( ), CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_runtime)")), CExpressionStatement(CodeExpression(f"if ({prefix}_helper == NULL) return NULL")), - CExpressionStatement(CodeExpression(f"{prefix}_shape = PyTuple_New({handle.array.rank})")), - CExpressionStatement( - CodeExpression(f"if ({prefix}_shape == NULL) {{ Py_DECREF({prefix}_helper); return NULL; }}") - ), - *self._native_descriptor_expected_shape_nodes(plan, context, names), ] binder_argument = "" binder_format = "" @@ -7729,10 +8366,7 @@ def _native_descriptor_helper_call_nodes( CodeExpression(f"{binder} = PyCFunction_NewEx(&{default_binder_definition}, NULL, NULL)") ), CExpressionStatement( - CodeExpression( - f"if ({binder} == NULL) {{ Py_DECREF({prefix}_helper); " - f"Py_DECREF({prefix}_shape); return NULL; }}" - ) + CodeExpression(f"if ({binder} == NULL) {{ Py_DECREF({prefix}_helper); return NULL; }}") ), ) ) @@ -7741,9 +8375,9 @@ def _native_descriptor_helper_call_nodes( nodes.append( CExpressionStatement( CodeExpression( - f'{prefix}_packed = PyObject_CallFunction({prefix}_helper, "Os{dtype_format}iOi{binder_format}", ' + f'{prefix}_packed = PyObject_CallFunction({prefix}_helper, "Os{dtype_format}ii{binder_format}", ' f'{names.object_name}, "{handle.descriptor_kind.value}", {dtype_argument}, ' - f"{handle.array.rank}, {prefix}_shape, {int(handle.optional_absent)}{binder_argument})" + f"{handle.array.rank}, {int(handle.optional_absent)}{binder_argument})" ) ) ) @@ -7752,44 +8386,11 @@ def _native_descriptor_helper_call_nodes( nodes.extend( ( CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_helper)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_shape)")), CExpressionStatement(CodeExpression(f"if ({prefix}_packed == NULL) return NULL")), ) ) return tuple(nodes) - def _native_descriptor_expected_shape_nodes( - self, - plan: ArgumentTransferPlan, - context: _CFunctionContext, - names: _CArgumentNames, - ) -> tuple[CExpressionStatement, ...]: - """Materialize the descriptor's declared shape for runtime validation.""" - handle = plan.native_array_handle - if handle is None: - return () - prefix = names.value_name - nodes = [] - for axis, expression in enumerate(handle.array.shape): - if expression in {":", "::Strided", "Flat"}: - nodes.append(CExpressionStatement(CodeExpression("Py_INCREF(Py_None)"))) - item = "Py_None" - else: - expected = self._array_extent_expression(handle.array, axis, expression, context) - item = f"PyLong_FromLongLong((long long)({expected}))" - nodes.extend( - ( - CExpressionStatement(CodeExpression(f"PyTuple_SET_ITEM({prefix}_shape, {axis}, {item})")), - CExpressionStatement( - CodeExpression( - f"if (PyTuple_GET_ITEM({prefix}_shape, {axis}) == NULL) {{ " - f"Py_DECREF({prefix}_helper); Py_DECREF({prefix}_shape); return NULL; }}" - ) - ), - ) - ) - return tuple(nodes) - def _native_descriptor_presence_unpack_nodes( self, plan: ArgumentTransferPlan, @@ -7826,6 +8427,11 @@ def _native_descriptor_pointer_unpack_nodes( cfi_type = self._native_array_cfi_type(plan) prefix = names.value_name condition = "1" if plan.binding.optional_mode is OptionalMode.REQUIRED else f"{names.present_name} != NULL" + absent = ( + self._absent_descriptor_placeholder_nodes(plan, names, handle) + if plan.entrypoint.pass_descriptor_presence + else () + ) return ( CExpressionStatement(CodeExpression(f"{prefix}_item = PyTuple_GetItem({prefix}_packed, 0)")), CExpressionStatement( @@ -7836,210 +8442,86 @@ def _native_descriptor_pointer_unpack_nodes( body=( CExpressionStatement( CodeExpression( - f"{prefix}_native_handle = prik_native_array_handle_from_capsule({prefix}_item, " - f"{self._native_array_handle_kind_constant(handle)}, {handle.array.rank}, {cfi_type}, " - f"{self._native_array_expected_element_size(plan)}, " - f"sizeof(CFI_CDESC_T({handle.array.rank})))" + f"{prefix}_native_backend = prik_native_array_backend_for_descriptor({prefix}_item, " + f"{self._native_array_handle_kind_constant(handle)}, {handle.array.rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({handle.array.rank})), {cfi_type}, " + f"{self._native_array_expected_element_size(plan)})" ) ), CIf( - CodeExpression(f"{prefix}_native_handle == NULL"), + CodeExpression(f"{prefix}_native_backend == NULL"), body=( CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_packed)")), CReturn(CodeExpression("NULL")), ), ), - CExpressionStatement( - CodeExpression(f"{names.value_name} = (CFI_cdesc_t *){prefix}_native_handle->descriptor") - ), + CComment("Attaching storage published the backend the chain enters."), ), + else_body=absent, ), ) - def _native_descriptor_fact_unpack_nodes( - self, - plan: ArgumentTransferPlan, - names: _CArgumentNames, - ) -> tuple[CExpressionStatement | CIf, ...]: - """Decode facts and establish a call-local standard descriptor.""" - handle = plan.native_array_handle - if handle is None or handle.array.rank is None: - return () - if plan.binding.optional_mode is OptionalMode.REQUIRED: - return self._native_descriptor_fact_present_nodes(plan, names) - return ( - CIf( - CodeExpression(f"{names.present_name} != NULL"), - body=self._native_descriptor_fact_present_nodes(plan, names), - else_body=self._native_descriptor_fact_absent_nodes(plan, names), - ), - ) - - def _native_descriptor_fact_present_nodes( + def _absent_descriptor_placeholder_nodes( self, plan: ArgumentTransferPlan, names: _CArgumentNames, - ) -> tuple[CExpressionStatement, ...]: - """Unpack one present fact tuple and initialize its CFI dimensions.""" - handle = plan.native_array_handle - if handle is None or handle.array.rank is None: - return () + handle: NativeArrayHandlePlan, + *, + packed_owner: str | None = "packed", + ) -> tuple[CComment | CExpressionStatement | CIf, ...]: + """Establish the unallocated descriptor an absent optional hands over. + + A generated bridge takes its descriptor dummy unconditionally and reads + the separate present flag, so something valid has to cross even when the + argument is absent. A null base address is the one form the standard + allows C to establish for this attribute, and absence is exactly when + there is nothing to point at. + + A direct ``bind(c)`` entrypoint has no such flag -- PRIK cannot add a + parameter to a signature the user wrote -- so there absence is a null + descriptor pointer and no placeholder is built. That is also what keeps + an absent argument distinct from a present but unallocated one. + """ prefix = names.value_name rank = handle.array.rank - nodes = [ - *self._native_descriptor_integer_field_nodes(prefix, f"{prefix}_base_addr", 0, pointer=True), - *self._native_descriptor_integer_field_nodes(prefix, f"{prefix}_elem_len", 1), - *self._native_descriptor_integer_field_nodes(prefix, f"{prefix}_descriptor_rank", 2), - ] - for axis in range(rank): - offset = 3 + 3 * axis - nodes.extend(self._native_descriptor_integer_field_nodes(prefix, f"{prefix}_lower_bound_{axis}", offset)) - nodes.extend( - self._native_descriptor_integer_field_nodes(prefix, f"{prefix}_descriptor_extent_{axis}", offset + 1) - ) - nodes.extend( - self._native_descriptor_integer_field_nodes(prefix, f"{prefix}_stride_multiplier_{axis}", offset + 2) - ) - nodes.append( + cfi_type = self._native_array_cfi_type(plan) + elem_len = self._native_array_expected_element_size(plan) + status = f"{prefix}_establish_status" + cleanup = ( + (CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_{packed_owner})")),) + if packed_owner is not None + else () + ) + return ( + CComment("Absent: hand over an unallocated placeholder, not a descriptor of"), + CComment("someone else's storage. The present flag tells the bridge to ignore it."), CExpressionStatement( CodeExpression( - f"if ({prefix}_descriptor_rank != {rank}) {{ PyErr_Format(PyExc_ValueError, " - f'"native descriptor rank %lld does not match planned rank {rank} for argument ' - f'{plan.binding.python_name}", (long long){prefix}_descriptor_rank); ' - f"Py_DECREF({prefix}_packed); return NULL; }}" + f"{status} = CFI_establish((CFI_cdesc_t *)&{prefix}_storage, NULL, " + f"{self._owned_native_array_cfi_attribute(handle)}, {cfi_type}, {elem_len}, {rank}, NULL)" ) - ) + ), + CIf( + CodeExpression(f"{status} != CFI_SUCCESS"), + body=( + CExpressionStatement( + CodeExpression( + f'PyErr_Format(PyExc_RuntimeError, "Unable to establish absent native descriptor ' + f'for argument {plan.binding.python_name}: %d", {status})' + ) + ), + *cleanup, + CReturn(CodeExpression("NULL")), + ), + ), + CExpressionStatement(CodeExpression(f"{names.value_name} = (CFI_cdesc_t *)&{prefix}_storage")), ) - nodes.extend(self._native_descriptor_establish_nodes(plan, names)) - return tuple(nodes) - def _native_descriptor_fact_absent_nodes( - self, - plan: ArgumentTransferPlan, - names: _CArgumentNames, - ) -> tuple[CExpressionStatement, ...]: - """Establish one valid placeholder descriptor for an omitted argument.""" - handle = plan.native_array_handle - if handle is None or handle.array.rank is None: - return () - prefix = names.value_name - nodes = [ - CExpressionStatement( - CodeExpression(f"{prefix}_elem_len = {self._native_descriptor_placeholder_elem_len(plan)}") - ), - CExpressionStatement(CodeExpression(f"{prefix}_descriptor_rank = {handle.array.rank}")), - ] - for axis in range(handle.array.rank): - nodes.append( - CExpressionStatement( - CodeExpression(f"{prefix}_stride_multiplier_{axis} = (CFI_index_t){prefix}_elem_len") - ) - ) - nodes.extend(self._native_descriptor_establish_nodes(plan, names)) - return tuple(nodes) - - def _native_descriptor_establish_nodes( - self, - plan: ArgumentTransferPlan, - names: _CArgumentNames, - ) -> tuple[CExpressionStatement, ...]: - """Establish call-local descriptor storage from already completed facts.""" - handle = plan.native_array_handle - if handle is None or handle.array.rank is None: - return () - prefix = names.value_name - rank = handle.array.rank - cfi_type = self._native_array_cfi_type(plan) - if cfi_type is None: - raise ValueError(f"Missing CFI type for {plan.owner_path!r}") - attribute = ( - "CFI_attribute_allocatable" - if handle.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE - else "CFI_attribute_pointer" - ) - nodes = [ - *( - CExpressionStatement( - CodeExpression(f"{prefix}_cfi_extents[{axis}] = {prefix}_descriptor_extent_{axis}") - ) - for axis in range(rank) - ), - CExpressionStatement( - CodeExpression( - f"{prefix}_establish_status = CFI_establish((CFI_cdesc_t *)&{prefix}_storage, " - f"{prefix}_base_addr, {attribute}, {cfi_type}, " - f"{prefix}_elem_len, {rank}, {prefix}_cfi_extents)" - ) - ), - CExpressionStatement( - CodeExpression( - f"if ({prefix}_establish_status != CFI_SUCCESS) {{ PyErr_Format(PyExc_RuntimeError, " - f'"Unable to establish native descriptor for argument {plan.binding.python_name}: %d", ' - f"{prefix}_establish_status); Py_DECREF({prefix}_packed); return NULL; }}" - ) - ), - ] - for axis in range(rank): - nodes.extend( - ( - CExpressionStatement( - CodeExpression( - f"((CFI_cdesc_t *)&{prefix}_storage)->dim[{axis}].lower_bound = {prefix}_lower_bound_{axis}" - ) - ), - CExpressionStatement( - CodeExpression( - f"((CFI_cdesc_t *)&{prefix}_storage)->dim[{axis}].extent = " - f"{prefix}_descriptor_extent_{axis}" - ) - ), - CExpressionStatement( - CodeExpression( - f"((CFI_cdesc_t *)&{prefix}_storage)->dim[{axis}].sm = {prefix}_stride_multiplier_{axis}" - ) - ), - ) - ) - nodes.append(CExpressionStatement(CodeExpression(f"{names.value_name} = (CFI_cdesc_t *)&{prefix}_storage"))) - return tuple(nodes) - - @staticmethod - def _native_descriptor_placeholder_elem_len(plan: ArgumentTransferPlan) -> str: - """Return a valid element length for one absent call-local descriptor.""" - if plan.datatype_family is DatatypeFamily.STRING: - return "0" - return f"sizeof({PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).c_spelling})" - - def _native_descriptor_integer_field_nodes( - self, - prefix: str, - target: str, - index: int, - *, - pointer: bool = False, - ) -> tuple[CExpressionStatement, ...]: - """Decode one validated tuple integer with local failure cleanup.""" - converter = "PyLong_AsVoidPtr" if pointer else "PyLong_AsLongLong" - cast = "(void *)" if pointer else "" - error = f"{target} == NULL && PyErr_Occurred()" if pointer else "PyErr_Occurred()" - return ( - CExpressionStatement(CodeExpression(f"{prefix}_item = PyTuple_GetItem({prefix}_packed, {index})")), - CExpressionStatement( - CodeExpression(f"if ({prefix}_item == NULL) {{ Py_DECREF({prefix}_packed); return NULL; }}") - ), - CExpressionStatement(CodeExpression(f"{target} = {cast}{converter}({prefix}_item)")), - CExpressionStatement(CodeExpression(f"if ({error}) {{ Py_DECREF({prefix}_packed); return NULL; }}")), - ) - - def _native_array_dtype(self, plan: ArgumentTransferPlan) -> str | None: - """Return the NumPy dtype spelling already selected by primitive type.""" - return self._native_array_dtype_for_semantic_type(plan.semantic_type_name, plan.datatype_family) - - def _native_array_dtype_for_result(self, plan: ResultPlan) -> str | None: - """Return the NumPy dtype spelling selected for one handle result.""" - return self._native_array_dtype_for_semantic_type(plan.semantic_type_name, plan.datatype_family) - - def _native_array_dtype_for_semantic_type( + def _native_array_dtype(self, plan: ArgumentTransferPlan) -> str | None: + """Return the NumPy dtype spelling already selected by primitive type.""" + return self._native_array_dtype_for_semantic_type(plan.semantic_type_name, plan.datatype_family) + + def _native_array_dtype_for_semantic_type( self, semantic_type_name: str, datatype_family: DatatypeFamily, @@ -8050,6 +8532,7 @@ def _native_array_dtype_for_semantic_type( scalar_type = PrimitiveScalarTypeRegistry.type_for(semantic_type_name) return { "NPY_BOOL": "bool", + "NPY_UINT8": "uint8", "NPY_INT8": "int8", "NPY_INT16": "int16", "NPY_INT32": "int32", @@ -8058,25 +8541,31 @@ def _native_array_dtype_for_semantic_type( "NPY_FLOAT64": "float64", "NPY_COMPLEX64": "complex64", "NPY_COMPLEX128": "complex128", - }[scalar_type.numpy_type_macro] + }[scalar_type.array_numpy_type] def _native_array_cfi_type(self, plan: ArgumentTransferPlan | ResultPlan) -> str | None: """Return the standard-descriptor element type after array-family dispatch.""" if plan.datatype_family is DatatypeFamily.STRING: return "CFI_type_char" - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).cfi_type_spelling + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_cfi_type + + def _field_native_array_cfi_type(self, field: DerivedFieldPlan) -> str | None: + """Return one field handle's standard-descriptor element type.""" + if field.string_element: + return "CFI_type_char" + return PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).array_cfi_type + + def _field_native_array_element_size(self, field: DerivedFieldPlan) -> str: + """Return one field handle's fixed element size, or zero for a runtime width.""" + if field.string_element: + return "0" + return f"sizeof({PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).array_c_spelling})" def _module_native_array_cfi_type(self, plan: ModuleVariablePlan) -> str | None: """Return one module handle's standard-descriptor element type.""" if plan.datatype_family is DatatypeFamily.STRING: return "CFI_type_char" - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).cfi_type_spelling - - def _module_native_array_elem_size(self, plan: ModuleVariablePlan) -> str: - """Return the completed numeric size or runtime character element length.""" - if plan.datatype_family is DatatypeFamily.STRING: - return f"{self._module_native_array_bridge_operation_name(plan, NativeArrayOperation.ELEMENT_LENGTH)}()" - return f"sizeof({PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).c_spelling})" + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_cfi_type def _native_array_handle_factory_call( self, @@ -8087,23 +8576,35 @@ def _native_array_handle_factory_call( semantic_type_name: str, datatype_family: DatatypeFamily, rank: int, - ops: str, + invoke: str, + capabilities: str, owner: str, descriptor_ownership: str, + native_backend: str, extraction_action: str, ) -> str: """Call the runtime factory with a fixed dtype or deferred character dtype.""" dtype = self._native_array_dtype_for_semantic_type(semantic_type_name, datatype_family) if dtype is None: return ( - f'{target} = PyObject_CallFunction({helper}, "sOiOOssO", "{descriptor_kind}", Py_None, ' - f'{rank}, {ops}, {owner}, "{descriptor_ownership}", "{extraction_action}", Py_None)' + f'{target} = PyObject_CallFunction({helper}, "sOiOOOssOO", "{descriptor_kind}", Py_None, ' + f'{rank}, {invoke}, {capabilities}, {owner}, "{descriptor_ownership}", "{extraction_action}", ' + f"{native_backend}, Py_None)" ) return ( - f'{target} = PyObject_CallFunction({helper}, "ssiOOssO", "{descriptor_kind}", "{dtype}", ' - f'{rank}, {ops}, {owner}, "{descriptor_ownership}", "{extraction_action}", Py_None)' + f'{target} = PyObject_CallFunction({helper}, "ssiOOOssOO", "{descriptor_kind}", "{dtype}", ' + f'{rank}, {invoke}, {capabilities}, {owner}, "{descriptor_ownership}", "{extraction_action}", ' + f"{native_backend}, Py_None)" ) + @staticmethod + def _native_array_capabilities_expression(operations: tuple[NativeArrayOperation, ...]) -> str: + """Build one immutable Python tuple from completed handle operations.""" + if not operations: + return "PyTuple_New(0)" + values = ", ".join(f'"{operation.value}"' for operation in operations) + return f'Py_BuildValue("({"s" * len(operations)})", {values})' + def _lower_argument_nullable_value( self, plan: ArgumentTransferPlan, @@ -8392,13 +8893,14 @@ def _lower_result_owned_native_array_handle( if python_name is None or handle is None or handle.array.rank is None: raise ValueError(f"Owned native array result {plan.owner_path!r} has no binding consumer") prefix = f"{descriptor_name}_handle" + dispatch = self._owned_native_array_dispatch_name(None, plan) cleanup = self._owned_descriptor_failure_cleanup(plan, descriptor_name) nodes: list[CDeclaration | CExpressionStatement | CIf] = [ CDeclaration(f"{prefix}_runtime", "PyObject *", CodeExpression("NULL")), CDeclaration(f"{prefix}_helper", "PyObject *", CodeExpression("NULL")), - CDeclaration(f"{prefix}_ops", "PyObject *", CodeExpression("NULL")), + CDeclaration(f"{prefix}_capabilities", "PyObject *", CodeExpression("NULL")), + CDeclaration(f"{prefix}_invoke", "PyObject *", CodeExpression("NULL")), CDeclaration(f"{prefix}_owner", "PyObject *", CodeExpression("NULL")), - CDeclaration(f"{prefix}_operation", "PyObject *", CodeExpression("NULL")), CDeclaration(python_name, "PyObject *", CodeExpression("NULL")), *self._owned_pointer_result_normalization_nodes( plan, @@ -8407,10 +8909,25 @@ def _lower_result_owned_native_array_handle( failure_cleanup, pending_native_cleanup, ), - CExpressionStatement(CodeExpression(f"{prefix}_ops = PyDict_New()")), + CExpressionStatement( + CodeExpression( + f"{prefix}_capabilities = {self._native_array_capabilities_expression(handle.operations)}" + ) + ), + CIf( + CodeExpression(f"{prefix}_capabilities == NULL"), + body=( + *cleanup, + *pending_native_cleanup, + *self._decref_names(failure_cleanup), + CReturn(CodeExpression("NULL")), + ), + ), + CExpressionStatement(CodeExpression(f"{prefix}_invoke = PyCFunction_NewEx(&{dispatch}_def, NULL, NULL)")), CIf( - CodeExpression(f"{prefix}_ops == NULL"), + CodeExpression(f"{prefix}_invoke == NULL"), body=( + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), *cleanup, *pending_native_cleanup, *self._decref_names(failure_cleanup), @@ -8418,15 +8935,6 @@ def _lower_result_owned_native_array_handle( ), ), ] - nodes.extend( - self._owned_native_array_ops_dictionary_nodes( - plan, - prefix, - cleanup, - failure_cleanup, - pending_native_cleanup, - ) - ) nodes.extend( ( CExpressionStatement( @@ -8437,7 +8945,8 @@ def _lower_result_owned_native_array_handle( CIf( CodeExpression(f"{prefix}_owner == NULL"), body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), *cleanup, *pending_native_cleanup, *self._decref_names(failure_cleanup), @@ -8452,7 +8961,8 @@ def _lower_result_owned_native_array_handle( CodeExpression(f"{prefix}_runtime == NULL"), body=( CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_owner)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), *cleanup, *pending_native_cleanup, *self._decref_names(failure_cleanup), @@ -8462,7 +8972,7 @@ def _lower_result_owned_native_array_handle( CExpressionStatement( CodeExpression( f"{prefix}_helper = PyObject_GetAttrString({prefix}_runtime, " - '"_native_array_handle_from_generated_ops")' + '"_native_array_handle_from_generated_dispatch")' ) ), CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_runtime)")), @@ -8470,7 +8980,8 @@ def _lower_result_owned_native_array_handle( CodeExpression(f"{prefix}_helper == NULL"), body=( CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_owner)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), *cleanup, *pending_native_cleanup, *self._decref_names(failure_cleanup), @@ -8486,16 +8997,19 @@ def _lower_result_owned_native_array_handle( semantic_type_name=plan.semantic_type_name, datatype_family=plan.datatype_family, rank=handle.array.rank, - ops=f"{prefix}_ops", + invoke=f"{prefix}_invoke", + capabilities=f"{prefix}_capabilities", owner=f"{prefix}_owner", descriptor_ownership="owned", + native_backend=f"{prefix}_owner", extraction_action=handle.extraction_action.value, ) ) ), CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_helper)")), CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_owner)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_invoke)")), + CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_capabilities)")), CIf( CodeExpression(f"{python_name} == NULL"), body=( @@ -8562,54 +9076,6 @@ def _owned_pointer_result_normalization_nodes( ), ) - def _owned_native_array_ops_dictionary_nodes( - self, - result: ResultPlan, - prefix: str, - cleanup: tuple[CExpressionStatement | CIf, ...], - failure_cleanup: tuple[str, ...], - pending_native_cleanup: tuple[CExpressionStatement, ...], - ) -> tuple[CExpressionStatement | CIf, ...]: - """Populate a result handle's operation dictionary from planned roles.""" - handle = result.native_array_handle - if handle is None: - return () - nodes = [] - for operation in handle.operations: - definition = self._owned_native_array_operation_def_name(None, result, operation) - nodes.extend( - ( - CExpressionStatement( - CodeExpression(f"{prefix}_operation = PyCFunction_NewEx(&{definition}, NULL, NULL)") - ), - CIf( - CodeExpression(f"{prefix}_operation == NULL"), - body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), - *cleanup, - *pending_native_cleanup, - *self._decref_names(failure_cleanup), - CReturn(CodeExpression("NULL")), - ), - ), - CIf( - CodeExpression( - f'PyDict_SetItemString({prefix}_ops, "{operation.value}", {prefix}_operation) < 0' - ), - body=( - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_operation)")), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_ops)")), - *cleanup, - *pending_native_cleanup, - *self._decref_names(failure_cleanup), - CReturn(CodeExpression("NULL")), - ), - ), - CExpressionStatement(CodeExpression(f"Py_DECREF({prefix}_operation)")), - ) - ) - return tuple(nodes) - # Ordinary-array result lowering. def _lower_result_array_copy( self, @@ -9000,124 +9466,1733 @@ def _holder_wrapper_nodes( *failure_cleanup, CReturn(CodeExpression("NULL")), ), - ), - CExpressionStatement( - CodeExpression( - f'{target} = PyObject_CallFunction({helper}, "OOOs", {capsule}, {owner}, {ops}, "{origin}")' + ), + CExpressionStatement( + CodeExpression( + f'{target} = PyObject_CallFunction({helper}, "OOOs", {capsule}, {owner}, {ops}, "{origin}")' + ) + ), + CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({helper})")), + CExpressionStatement(CodeExpression(f"Py_DECREF({capsule})")), + CIf( + CodeExpression(f"{target} == NULL"), + body=(*failure_cleanup, CReturn(CodeExpression("NULL"))), + ), + ) + + @staticmethod + def _derived_target_owner(handoff: DerivedHandoffPlan) -> str: + """Select the retained pointer-target owner from completed policy.""" + if handoff.target_owner_retention is DerivedOwnerRetention.NATIVE_MODULE: + return "self" + if handoff.target_owner_retention is DerivedOwnerRetention.NONE: + return "Py_None" + raise ValueError(f"Unsupported derived target owner retention: {handoff.target_owner_retention.value}") + + def _holder_wrapper_symbols( + self, + type_symbol: str, + storage: DerivedObjectStorage, + ) -> tuple[str, str, str, str, str]: + """Return mechanical symbols for one completed holder storage choice.""" + if storage is DerivedObjectStorage.ALLOCATABLE_HOLDER: + return ( + self._allocatable_holder_capsule_name(type_symbol), + self._allocatable_holder_capsule_destructor_name(type_symbol), + self._allocatable_holder_destroy_bridge_name(type_symbol), + self._allocatable_holder_ops_name(type_symbol), + storage.value, + ) + if storage is DerivedObjectStorage.POINTER_HOLDER: + return ( + self._pointer_holder_capsule_name(type_symbol), + self._pointer_holder_capsule_destructor_name(type_symbol), + self._pointer_holder_destroy_bridge_name(type_symbol), + self._pointer_holder_ops_name(type_symbol), + storage.value, + ) + raise ValueError(f"Unsupported derived holder storage: {storage.value}") + + # Scalar result lowering. + def _lower_result_direct_value( + self, + plan: ResultPlan, + context: _CFunctionContext, + failure_cleanup: tuple[str, ...], + failure_label: str | None = None, + pending_native_cleanup: tuple[CExpressionStatement, ...] = (), + ) -> tuple[CExpressionStatement | CDeclaration | CIf, ...]: + """Lower result direct value from the supplied completed binding records without inferring semantic policy.""" + return self._lower_result_value( + plan, + context, + failure_cleanup, + failure_label, + pending_native_cleanup, + ) + + def _lower_result_value( + self, + plan: ResultPlan, + context: _CFunctionContext, + failure_cleanup: tuple[str, ...], + failure_label: str | None = None, + pending_native_cleanup: tuple[CExpressionStatement, ...] = (), + ) -> tuple[CExpressionStatement | CDeclaration | CIf, ...]: + """Convert one native result into its binding-owned Python consumer.""" + scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) + native_name = self._result_native_name(plan, context) + python_name = context.python_results.get(plan.owner_path) + if scalar_type.python_result_kind is None or python_name is None: + raise ValueError(f"Unsupported scalar result type {plan.semantic_type_name!r}") + converted_name = native_name + conversion = () + if plan.entrypoint.native_scalar_c_type is not None: + converted_name = f"{native_name}_contract" + conversion = ( + CDeclaration( + converted_name, + scalar_type.c_spelling, + CodeExpression(f"({scalar_type.c_spelling}){native_name}"), + ), + ) + return ( + *conversion, + CDeclaration( + python_name, + "PyObject *", + CodeExpression(self._scalar_result_expression(scalar_type, f"&{converted_name}")), + ), + CIf( + CodeExpression(f"{python_name} == NULL"), + body=( + *pending_native_cleanup, + *self._output_failure_nodes(failure_cleanup, failure_label), + ), + ), + ) + + def _result_native_name(self, plan: ResultPlan, context: _CFunctionContext) -> str: + """Return the validated C storage consumed by one result conversion.""" + if plan.source_kind == "direct_return": + if context.result_name is None: + raise ValueError(f"Direct result {plan.owner_path!r} has no C storage") + return context.result_name + try: + return context.native_outputs[plan.entrypoint.native_result_role] + except KeyError: + raise ValueError(f"Hidden result {plan.owner_path!r} has no C output storage") from None + + def _inverted_descriptor_arguments(self, plan: FunctionPlan) -> tuple[ArgumentTransferPlan, ...]: + """Return every argument whose descriptor must be live across the call. + + The descriptor the runtime builds for a module array or a field exists + only while the consumer it was handed to is running. Making the call + inside that consumer is what lets a callee change the allocation of a + writable dummy and have it reach the caller's entity, and it means no + descriptor is ever copied: a read-only argument is placed the same way, + so C only ever passes on a descriptor Fortran made. A handle that owns + its descriptor enters the same way, because its entry point hands the + consumer the storage directly. + + Each one is entered in turn, so several descriptors are live together by + the time the innermost consumer makes the call. An optional argument + that is absent has no entity to enter and contributes its unallocated + placeholder to the same chain instead. + """ + return tuple( + argument + for argument in plan.arguments + if self._array_crosses_as_descriptor(argument) + or ( + argument.native_array_handle is not None + and argument.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + and argument.native_array_handle.array.rank is not None + ) + ) + + def _lower_entrypoint_call(self, plan: FunctionPlan, context: _CFunctionContext) -> tuple: + """Emit the native call, inside the consumers holding its descriptors.""" + if not context.inverted_descriptors: + return self._lower_native_call(plan, self._entrypoint_call_statement(plan, context)) + names = context.arguments[context.inverted_descriptors[0]] + record = self._inverted_context_name(plan) + chain = self._inverted_chain_fields(plan, context) + fields = self._inverted_context_fields(plan, context) + values = [value for _declaration, value in chain + fields] + if self._inverted_carries_result(plan): + values.append("0") + return ( + CComment("Everything the call needs apart from the descriptors themselves is"), + CComment("gathered here, because the consumers run outside this frame."), + CDeclaration("call_context", record, CodeExpression(f"{{{', '.join(values)}}}")), + CComment("Each descriptor is entered in turn and the call is made inside the last"), + CComment("consumer, where they are all live, so what the callee writes into any of"), + CComment("them is what Fortran copies back to that caller's entity."), + *self._inverted_enter_nodes( + plan, + 0, + backend=self._descriptor_backend_local(names), + call_context="&call_context", + placeholder=f"call_context.{self._inverted_descriptor_field(0)}", + ), + *( + (CExpressionStatement(CodeExpression(f"{context.result_name} = call_context.result")),) + if self._inverted_carries_result(plan) and context.result_name is not None + else () + ), + ) + + @staticmethod + def _array_crosses_as_descriptor(argument: ArgumentTransferPlan) -> bool: + """Report whether completed policy hands this array over as a descriptor.""" + array = argument.array + return ( + argument.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER + and array is not None + and array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + ) + + def _array_actual_reader_name(self, function: FunctionPlan, argument: ArgumentTransferPlan) -> str: + """Return the reader that copies one handle's storage out of its descriptor.""" + owner = re.sub(r"\W", "_", argument.owner_path).casefold() + return f"prik_read_array_actual_{owner}" + + def _array_actual_reader_record_name(self, function: FunctionPlan, argument: ArgumentTransferPlan) -> str: + """Return the record one array-actual reader fills.""" + return f"{self._array_actual_reader_name(function, argument)}_result" + + def _array_actual_handle_arguments(self, plan: ModulePlan): + """Return ordinary array arguments an array handle may be passed to.""" + for function in self._functions(plan): + yield from self._array_actual_handle_arguments_for(function, self._function_context(function)) + + @staticmethod + def _declared_character_width(argument: ArgumentTransferPlan) -> int: + """Return the element width a character dummy declares, or zero.""" + actual = argument.native_array_actual + declared = re.search(r"S(\d+)$", "" if actual is None else str(actual.dtype)) + return int(declared.group(1)) if declared else 0 + + @staticmethod + def _declared_character_width_guard(argument: ArgumentTransferPlan) -> tuple: + """Decline storage whose element width is not the one the dummy declares. + + A character dummy is matched on its width as well as its kind, and the + descriptor states the width the storage actually has. Declining leaves + the record empty, so the runtime reports the dtype it expected. + """ + if argument.datatype_family is not DatatypeFamily.STRING: + return () + actual = argument.native_array_actual + declared = re.search(r"S(\d+)$", "" if actual is None else str(actual.dtype)) + if declared is None: + return () + return ( + CComment("A character dummy is matched on its declared width."), + CIf( + CodeExpression(f"source->elem_len != (size_t){declared.group(1)}"), + body=(CReturn(),), + ), + ) + + @staticmethod + def _flattened_reader_axis(argument: ArgumentTransferPlan) -> int | None: + """Return the contract axis a flattened dummy collapses into, if any.""" + actual = argument.native_array_actual + if actual is None or not actual.flatten_storage: + return None + rank = argument.array.rank + axis = 0 if actual.flat_axis is None or int(actual.flat_axis) < 0 else int(actual.flat_axis) + return axis if axis in {0, rank - 1} else None + + def _flattened_array_actual_reader( + self, + function: FunctionPlan, + argument: ArgumentTransferPlan, + record: str, + rank: int, + flat_axis: int, + ) -> CFunction: + """Read a handle's storage for a dummy that flattens it. + + Such a dummy takes an actual of any rank and collapses it into one + contract axis, so this walks the rank the descriptor reports rather + than the dummy's own, and folds every collapsed axis into a product. + Kept extents come from the axes the contract still names. + """ + body: list = [ + CDeclaration("source", "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)descriptor")), + CDeclaration("out", f"{record} *", CodeExpression(f"({record} *)context")), + CDeclaration("expected", "CFI_index_t", CodeExpression("0")), + CDeclaration("collapsed", "int64_t", CodeExpression("1")), + CDeclaration("extent", "int64_t", CodeExpression("0")), + CDeclaration("axis", "int", CodeExpression("0")), + CDeclaration("kept", "int", CodeExpression(str(rank - 1))), + CExpressionStatement(CodeExpression("out->present = 0")), + CExpressionStatement(CodeExpression("out->contiguous = 1")), + CExpressionStatement(CodeExpression("out->refused = 1")), + CComment("Unallocated or disassociated storage has no address to pass."), + CIf(CodeExpression("source->base_addr == NULL"), body=(CReturn(),)), + CExpressionStatement(CodeExpression("out->refused = 2")), + CExpressionStatement(CodeExpression("out->width = source->elem_len")), + CExpressionStatement(CodeExpression("expected = (CFI_index_t)source->elem_len")), + *self._declared_character_width_guard(argument), + CExpressionStatement(CodeExpression("out->refused = 0")), + CComment("Every axis is walked once: contiguity is a property of them all,"), + CComment("and the collapsed extent is the product of the ones not kept."), + CFor( + "axis = 0", + CodeExpression("axis < (int)source->rank"), + CodeExpression("axis += 1"), + body=( + CExpressionStatement( + CodeExpression( + "extent = (int64_t)(source->dim[axis].extent == -1 ? 0 : source->dim[axis].extent)" + ) + ), + CIf( + CodeExpression("source->dim[axis].sm != expected"), + body=(CExpressionStatement(CodeExpression("out->contiguous = 0")),), + ), + CExpressionStatement(CodeExpression("expected *= (CFI_index_t)extent")), + CIf( + CodeExpression("axis < kept" if flat_axis == rank - 1 else "axis >= (int)source->rank - kept"), + body=( + CExpressionStatement( + CodeExpression( + f"out->extents[{'axis' if flat_axis == rank - 1 else 'axis - ((int)source->rank - kept) + 1'}] = extent" + ) + ), + ), + else_body=(CExpressionStatement(CodeExpression("collapsed *= extent")),), + ), + ), + ), + CExpressionStatement( + CodeExpression(f"out->extents[{rank - 1 if flat_axis == rank - 1 else 0}] = collapsed") + ), + CExpressionStatement(CodeExpression("out->data = source->base_addr")), + CExpressionStatement(CodeExpression("out->present = 1")), + ] + return CFunction( + self._array_actual_reader_name(function, argument), + "void", + parameters=(CParameter("descriptor", "void *"), CParameter("context", "void *")), + storage="static", + body=tuple(body), + doc=("Read one handle's storage for a dummy that flattens it into one axis.",), + ) + + def _array_actual_struct_reader_name(self, argument: ArgumentTransferPlan) -> str: + """Return the reader that fills one array actual from its descriptor.""" + owner = re.sub(r"\W", "_", argument.owner_path).casefold() + return f"prik_fill_array_actual_{owner}" + + def _array_actual_struct_reader_record_name(self, argument: ArgumentTransferPlan) -> str: + """Return the record one array-actual reader fills alongside its reason.""" + return f"{self._array_actual_struct_reader_name(argument)}_result" + + def _array_actual_struct_reader_record(self, argument: ArgumentTransferPlan) -> CStructDefinition: + """Define the record carrying one filled array actual and why it was refused.""" + rank = argument.array.rank + extent_count = "PRIK_MAX_ARRAY_RANK" if rank is None else str(rank) + return CStructDefinition( + self._array_actual_struct_reader_record_name(argument), + ( + CParameter("actual", "prik_array_actual *"), + CParameter(f"logical_extents[{extent_count}]", "int64_t"), + # 0 accepted, 1 no storage, 2 element width, 3 layout, 4 rank + CParameter("refused", "int"), + CParameter("width", "size_t"), + CParameter("rank", "int"), + ), + ) + + def _array_actual_struct_reader_function(self, argument: ArgumentTransferPlan) -> CFunction: + """Fill one ordinary-array ABI record from a live handle descriptor.""" + rank = argument.array.rank + active_rank = "source->rank" if rank is None else str(rank) + record = self._array_actual_struct_reader_record_name(argument) + body: list = [ + CDeclaration("source", "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)descriptor")), + CDeclaration("wrap", f"{record} *", CodeExpression(f"({record} *)context")), + CDeclaration("out", "prik_array_actual *", CodeExpression("wrap->actual")), + CDeclaration("extent", "int64_t", CodeExpression("0")), + CDeclaration("empty", "int", CodeExpression("0")), + CExpressionStatement(CodeExpression("out->data = NULL")), + CExpressionStatement(CodeExpression(f"out->rank = {active_rank}")), + CExpressionStatement(CodeExpression("wrap->refused = 1")), + CComment("Storage that is not there leaves the record empty."), + CIf(CodeExpression("source->base_addr == NULL"), body=(CReturn(),)), + CExpressionStatement(CodeExpression("wrap->rank = (int)source->rank")), + CExpressionStatement(CodeExpression("wrap->refused = 4")), + CIf( + CodeExpression( + f"source->rank < {argument.array.minimum_rank} || source->rank > {argument.array.maximum_rank}" + ), + body=(CReturn(),), + ), + CExpressionStatement(CodeExpression("wrap->refused = 2")), + CExpressionStatement(CodeExpression("wrap->width = source->elem_len")), + CExpressionStatement(CodeExpression("out->itemsize = (int64_t)source->elem_len")), + *self._declared_character_width_guard(argument), + ] + if rank is None: + body.extend( + ( + CFor( + "int axis = 0", + CodeExpression("axis < PRIK_MAX_ARRAY_RANK"), + CodeExpression("++axis"), + body=( + CExpressionStatement(CodeExpression("wrap->logical_extents[axis] = 0")), + CExpressionStatement(CodeExpression("out->extents[axis] = 0")), + CExpressionStatement(CodeExpression("out->upper_bounds[axis] = -1")), + CExpressionStatement(CodeExpression("out->strides[axis] = 1")), + ), + ), + CFor( + "int axis = 0", + CodeExpression("axis < (int)source->rank"), + CodeExpression("++axis"), + body=( + CExpressionStatement( + CodeExpression( + "extent = (int64_t)(source->dim[axis].extent == -1 ? 0 : source->dim[axis].extent)" + ) + ), + CExpressionStatement(CodeExpression("wrap->logical_extents[axis] = extent")), + CExpressionStatement(CodeExpression("if (extent == 0) empty = 1")), + ), + ), + ) + ) + else: + for axis in range(rank): + body.extend( + ( + CExpressionStatement( + CodeExpression( + f"extent = (int64_t)(source->dim[{axis}].extent == -1 ? 0 : source->dim[{axis}].extent)" + ) + ), + CExpressionStatement(CodeExpression(f"wrap->logical_extents[{axis}] = extent")), + CExpressionStatement(CodeExpression("if (extent == 0) empty = 1")), + ) + ) + empty_body: list = [] + if rank is None: + empty_body.append( + CFor( + "int axis = 0", + CodeExpression("axis < (int)source->rank"), + CodeExpression("++axis"), + body=( + CExpressionStatement(CodeExpression("out->extents[axis] = wrap->logical_extents[axis]")), + CExpressionStatement( + CodeExpression( + "out->upper_bounds[axis] = wrap->logical_extents[axis] == 0 " + "? -1 : wrap->logical_extents[axis] - 1" + ) + ), + CExpressionStatement(CodeExpression("out->strides[axis] = 1")), + ), + ) + ) + else: + for axis in range(rank): + empty_body.extend( + ( + CExpressionStatement(CodeExpression(f"out->extents[{axis}] = wrap->logical_extents[{axis}]")), + CExpressionStatement( + CodeExpression( + f"out->upper_bounds[{axis}] = wrap->logical_extents[{axis}] == 0 " + f"? -1 : wrap->logical_extents[{axis}] - 1" + ) + ), + CExpressionStatement(CodeExpression(f"out->strides[{axis}] = 1")), + ) + ) + empty_body.extend( + ( + CExpressionStatement(CodeExpression("wrap->refused = 0")), + CExpressionStatement(CodeExpression("out->data = source->base_addr")), + CReturn(), + ) + ) + body.append(CIf(CodeExpression("empty"), body=tuple(empty_body))) + body.append(CExpressionStatement(CodeExpression("wrap->refused = 3"))) + if argument.array.stride_roles: + body.extend(self._strided_native_array_actual_reader_nodes(argument)) + elif rank is None: + body.extend( + ( + CDeclaration("packed", "CFI_index_t", CodeExpression("0")), + CExpressionStatement(CodeExpression("packed = (CFI_index_t)source->elem_len")), + CFor( + "int axis = 0", + CodeExpression("axis < (int)source->rank"), + CodeExpression("++axis"), + body=( + CIf(CodeExpression("source->dim[axis].sm != packed"), body=(CReturn(),)), + CExpressionStatement(CodeExpression("out->extents[axis] = wrap->logical_extents[axis]")), + CExpressionStatement( + CodeExpression("out->upper_bounds[axis] = wrap->logical_extents[axis] - 1") + ), + CExpressionStatement(CodeExpression("out->strides[axis] = 1")), + CExpressionStatement(CodeExpression("packed *= (CFI_index_t)wrap->logical_extents[axis]")), + ), + ), + ) + ) + else: + body.append(CDeclaration("packed", "CFI_index_t", CodeExpression("source->elem_len"))) + for axis in range(rank): + body.extend( + ( + CIf(CodeExpression(f"source->dim[{axis}].sm != packed"), body=(CReturn(),)), + CExpressionStatement(CodeExpression(f"out->extents[{axis}] = wrap->logical_extents[{axis}]")), + CExpressionStatement( + CodeExpression(f"out->upper_bounds[{axis}] = wrap->logical_extents[{axis}] - 1") + ), + CExpressionStatement(CodeExpression(f"out->strides[{axis}] = 1")), + CExpressionStatement(CodeExpression(f"packed *= (CFI_index_t)wrap->logical_extents[{axis}]")), + ) + ) + body.extend( + ( + CExpressionStatement(CodeExpression("wrap->refused = 0")), + CExpressionStatement(CodeExpression("out->data = source->base_addr")), + ) + ) + return CFunction( + self._array_actual_struct_reader_name(argument), + "void", + parameters=(CParameter("descriptor", "void *"), CParameter("context", "void *")), + storage="static", + body=tuple(body), + doc=("Fill one array actual record from the descriptor the runtime opened.",), + ) + + @staticmethod + def _strided_native_array_actual_reader_nodes(argument: ArgumentTransferPlan) -> tuple: + """Preserve one positive-strided Fortran descriptor in bridge slice roles.""" + rank = argument.array.rank + if rank is None: + raise ValueError(f"Strided native array actual {argument.owner_path!r} requires a fixed rank") + nodes: list = [ + CDeclaration("base_bytes", "CFI_index_t", CodeExpression("(CFI_index_t)source->elem_len")), + CDeclaration("relative_stride", "int64_t", CodeExpression("0")), + CDeclaration("upper_bound", "int64_t", CodeExpression("0")), + ] + for axis in range(rank): + nodes.extend( + ( + CIf( + CodeExpression(f"source->dim[{axis}].sm <= 0 || source->dim[{axis}].sm % base_bytes != 0"), + body=(CReturn(),), + ), + CExpressionStatement( + CodeExpression(f"relative_stride = (int64_t)(source->dim[{axis}].sm / base_bytes)") + ), + CExpressionStatement( + CodeExpression(f"upper_bound = (wrap->logical_extents[{axis}] - 1) * relative_stride") + ), + CExpressionStatement(CodeExpression(f"out->strides[{axis}] = relative_stride")), + CExpressionStatement(CodeExpression(f"out->upper_bounds[{axis}] = upper_bound")), + ) + ) + if axis + 1 < rank: + nodes.extend( + ( + CIf( + CodeExpression( + f"source->dim[{axis + 1}].sm <= 0 || source->dim[{axis + 1}].sm % base_bytes != 0" + ), + body=(CReturn(),), + ), + CExpressionStatement( + CodeExpression(f"out->extents[{axis}] = (int64_t)(source->dim[{axis + 1}].sm / base_bytes)") + ), + CIf(CodeExpression(f"out->extents[{axis}] <= upper_bound"), body=(CReturn(),)), + ) + ) + else: + nodes.append(CExpressionStatement(CodeExpression(f"out->extents[{axis}] = upper_bound + 1"))) + nodes.append(CExpressionStatement(CodeExpression(f"base_bytes *= (CFI_index_t)out->extents[{axis}]"))) + return tuple(nodes) + + # One live descriptor answers every handle inquiry. + # + # `with_descriptor` is the only way to reach a handle's descriptor, and it + # works the same whether the descriptor is one Fortran built for the call + # or persistent storage this wrapper owns. Every inquiry a handle answers + # is therefore one of these consumers run through that entry point: the + # descriptor is read where it is valid and only the finished Python object + # leaves. Nothing copies a descriptor out, and no operation needs a Fortran + # procedure of its own to report what the descriptor already says. + NUMPY_DESCRIPTOR_BUILDER = "prik_describe_numpy_array" + ARRAY_EXTENTS_READER = "prik_read_array_extents" + ARRAY_EXTENTS_RECORD = "prik_array_extents_out" + + def _array_extents_reader_function(self, plan: ModulePlan) -> tuple: + """Emit the consumer that reads a descriptor's extents into the frame. + + A declaration elsewhere may be written in terms of this array's shape -- + a result declared ``dimension(size(values))`` is -- and a borrowed + descriptor is gone once its consumer returns. So the extents are taken + while it is live and kept in the caller's own storage, which is what + every later use reads. + """ + if not self._module_describes_numpy_arrays(plan): + return () + return ( + CStructDefinition( + self.ARRAY_EXTENTS_RECORD, + ( + CParameter("rank", "int"), + CParameter("present", "int"), + CParameter("contiguous", "int"), + CParameter("elem_len", "int64_t"), + CParameter("extents", "int64_t *"), + ), + ), + CFunction( + self.ARRAY_EXTENTS_READER, + "void", + parameters=(CParameter("descriptor", "void *"), CParameter("context", "void *")), + storage="static", + doc=( + "Copy one descriptor's extents out while it is live, and say" + " whether there is any storage behind it.", + "A descriptor with no address describes an allocatable that was never" + " allocated, or a pointer that points at nothing. Neither may be handed" + " to a dummy that expects data.", + ), + body=( + CDeclaration("source", "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)descriptor")), + CDeclaration( + "out", + f"{self.ARRAY_EXTENTS_RECORD} *", + CodeExpression(f"({self.ARRAY_EXTENTS_RECORD} *)context"), + ), + CDeclaration("axis", "int", CodeExpression("0")), + CExpressionStatement(CodeExpression("out->present = source->base_addr != NULL")), + CExpressionStatement( + CodeExpression("out->contiguous = out->present && CFI_is_contiguous(source) != 0") + ), + CExpressionStatement(CodeExpression("out->elem_len = (int64_t)source->elem_len")), + CFor( + "axis = 0", + CodeExpression("axis < out->rank"), + CodeExpression("++axis"), + body=( + CComment("A compiler may report an empty dimension as extent -1."), + CExpressionStatement( + CodeExpression( + "out->extents[axis] = (int64_t)(source->dim[axis].extent == -1 " + "? 0 : source->dim[axis].extent)" + ) + ), + ), + ), + CReturn(), + ), + ), + ) + + def _numpy_descriptor_builder_function(self, plan: ModulePlan) -> tuple: + """Emit the one constructor every NumPy array reaching a descriptor uses.""" + if not self._module_describes_numpy_arrays(plan): + return () + return ( + CFunction( + self.NUMPY_DESCRIPTOR_BUILDER, + "int", + parameters=( + CParameter("parent", "CFI_cdesc_t *"), + CParameter("section", "CFI_cdesc_t *"), + CParameter("array", "PyArrayObject *"), + CParameter("cfi_type", "CFI_type_t"), + CParameter("argument_name", "const char *"), + ), + storage="static", + doc=( + "Describe borrowed NumPy storage to Fortran, without copying it.", + "CFI_establish only makes contiguous descriptors, so a strided view has" + " to be a section of one: the parent established here is the smallest" + " contiguous array the view is a section of, and CFI_section cuts the" + " view back out of it. That is the only construction the standard" + " offers, and it is what carries a signed stride.", + "The parent's own steps are chosen to divide the view's, so each axis" + " needs a whole-number step; an axis that runs backwards starts at its" + " far end and walks down. Validation has already refused anything these" + " rules cannot describe, so a failure here is the Fortran runtime's.", + "Both descriptors belong to the caller's frame and last exactly as long" + " as the call, which is as long as the array is borrowed.", + ), + body=( + CDeclaration("rank", "int", CodeExpression("PyArray_NDIM(array)")), + CDeclaration("elem_len", "CFI_index_t", CodeExpression("(CFI_index_t)PyArray_ITEMSIZE(array)")), + CDeclaration("extents[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("parent_extents[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("lower[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("upper[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("step[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("unit[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("element_stride[PRIK_MAX_ARRAY_RANK]", "CFI_index_t"), + CDeclaration("step_magnitude", "CFI_index_t", CodeExpression("0")), + CDeclaration("section_span", "CFI_index_t", CodeExpression("0")), + CDeclaration("contribution", "CFI_index_t", CodeExpression("0")), + CDeclaration("previous_unit", "CFI_index_t", CodeExpression("0")), + CDeclaration("offset", "CFI_index_t", CodeExpression("0")), + CDeclaration("empty", "int", CodeExpression("0")), + CDeclaration("axis", "int", CodeExpression("0")), + CDeclaration("next_axis", "int", CodeExpression("0")), + CDeclaration("status", "int", CodeExpression("CFI_SUCCESS")), + CIf( + CodeExpression("elem_len <= 0"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_TypeError, "Argument %s has no element width", argument_name)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CFor( + "axis = 0", + CodeExpression("axis < rank"), + CodeExpression("++axis"), + body=( + CExpressionStatement( + CodeExpression("extents[axis] = (CFI_index_t)PyArray_DIM(array, axis)") + ), + CExpressionStatement( + CodeExpression( + "element_stride[axis] = (CFI_index_t)PyArray_STRIDE(array, axis) / elem_len" + ) + ), + CIf( + CodeExpression("extents[axis] == 0"), + body=(CExpressionStatement(CodeExpression("empty = 1")),), + ), + ), + ), + CComment("Nothing steps anywhere in an empty array, so it needs no section."), + CIf( + CodeExpression("empty"), + body=( + CExpressionStatement( + CodeExpression( + "status = CFI_establish(section, PyArray_DATA(array), CFI_attribute_other, " + "cfi_type, (size_t)elem_len, rank, extents)" + ) + ), + CIf( + CodeExpression("status != CFI_SUCCESS"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_TypeError, "Argument %s could not be described ' + 'to Fortran as an empty array: %d", argument_name, status)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CReturn(CodeExpression("0")), + ), + ), + CComment("A singleton axis has no observable stride; choose one that keeps a contiguous parent."), + CFor( + "axis = 0", + CodeExpression("axis < rank"), + CodeExpression("++axis"), + body=( + CIf( + CodeExpression("extents[axis] <= 1"), + body=( + CIf( + CodeExpression("axis == 0"), + body=(CExpressionStatement(CodeExpression("element_stride[axis] = 1")),), + else_body=( + CFor( + "next_axis = axis + 1", + CodeExpression("next_axis < rank && extents[next_axis] <= 1"), + CodeExpression("++next_axis"), + body=(), + ), + CIf( + CodeExpression("next_axis < rank"), + body=( + CExpressionStatement( + CodeExpression( + "element_stride[axis] = element_stride[next_axis] < 0 " + "? -element_stride[next_axis] : element_stride[next_axis]" + ) + ), + ), + else_body=( + CExpressionStatement( + CodeExpression( + "previous_unit = axis == 1 ? 1 : " + "(element_stride[axis - 1] < 0 " + "? -element_stride[axis - 1] : element_stride[axis - 1])" + ) + ), + CExpressionStatement( + CodeExpression( + "step_magnitude = element_stride[axis - 1] < 0 " + "? -element_stride[axis - 1] : element_stride[axis - 1]" + ) + ), + CExpressionStatement( + CodeExpression("step_magnitude /= previous_unit") + ), + CIf( + CodeExpression( + "extents[axis - 1] > 1 && step_magnitude > " + "((CFI_index_t)PTRDIFF_MAX - 1) / " + "(extents[axis - 1] - 1)" + ), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_OverflowError, "Argument %s has ' + 'strides too large for a Fortran descriptor", ' + "argument_name)" + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CExpressionStatement( + CodeExpression( + "section_span = (extents[axis - 1] - 1) * " + "step_magnitude + 1" + ) + ), + CIf( + CodeExpression( + "previous_unit > (CFI_index_t)PTRDIFF_MAX / section_span" + ), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_OverflowError, "Argument %s has ' + 'strides too large for a Fortran descriptor", ' + "argument_name)" + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CExpressionStatement( + CodeExpression( + "element_stride[axis] = previous_unit * section_span" + ) + ), + ), + ), + ), + ), + ), + ), + ), + ), + CComment("The parent's step along each axis, chosen to divide the view's."), + CExpressionStatement(CodeExpression("unit[0] = 1")), + CFor( + "axis = 1", + CodeExpression("axis < rank"), + CodeExpression("++axis"), + body=( + CExpressionStatement( + CodeExpression( + "unit[axis] = element_stride[axis] < 0 ? -element_stride[axis] " + ": element_stride[axis]" + ) + ), + ), + ), + CFor( + "axis = 0", + CodeExpression("axis < rank"), + CodeExpression("++axis"), + body=( + CExpressionStatement(CodeExpression("step[axis] = element_stride[axis] / unit[axis]")), + CExpressionStatement( + CodeExpression("step_magnitude = step[axis] < 0 ? -step[axis] : step[axis]") + ), + CIf( + CodeExpression( + "extents[axis] > 1 && step_magnitude > " + "((CFI_index_t)PTRDIFF_MAX - 1) / (extents[axis] - 1)" + ), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_OverflowError, "Argument %s has strides too large ' + 'for a Fortran descriptor", argument_name)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CExpressionStatement( + CodeExpression("section_span = (extents[axis] - 1) * step_magnitude + 1") + ), + CExpressionStatement( + CodeExpression( + "parent_extents[axis] = axis + 1 < rank ? (element_stride[axis + 1] < 0 " + "? -element_stride[axis + 1] : element_stride[axis + 1]) / unit[axis] " + ": section_span" + ) + ), + CComment("A backward axis starts at its far end and walks down."), + CExpressionStatement(CodeExpression("lower[axis] = step[axis] > 0 ? 0 : section_span - 1")), + CExpressionStatement(CodeExpression("upper[axis] = step[axis] > 0 ? section_span - 1 : 0")), + CIf( + CodeExpression( + "lower[axis] != 0 && unit[axis] > (CFI_index_t)PTRDIFF_MAX / lower[axis]" + ), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_OverflowError, "Argument %s has strides too large ' + 'for a Fortran descriptor", argument_name)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CExpressionStatement(CodeExpression("contribution = lower[axis] * unit[axis]")), + CIf( + CodeExpression("contribution > (CFI_index_t)PTRDIFF_MAX - offset"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_OverflowError, "Argument %s has strides too large ' + 'for a Fortran descriptor", argument_name)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CExpressionStatement(CodeExpression("offset += contribution")), + ), + ), + CIf( + CodeExpression("offset > (CFI_index_t)PTRDIFF_MAX / elem_len"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_OverflowError, "Argument %s has strides too large for a ' + 'Fortran descriptor", argument_name)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CExpressionStatement( + CodeExpression( + "status = CFI_establish(parent, (char *)PyArray_DATA(array) - offset * elem_len, " + "CFI_attribute_other, cfi_type, (size_t)elem_len, rank, parent_extents)" + ) + ), + CIf( + CodeExpression("status == CFI_SUCCESS"), + body=( + CExpressionStatement( + CodeExpression( + "status = CFI_establish(section, NULL, CFI_attribute_other, cfi_type, " + "(size_t)elem_len, rank, NULL)" + ) + ), + ), + ), + CIf( + CodeExpression("status == CFI_SUCCESS"), + body=( + CExpressionStatement( + CodeExpression("status = CFI_section(section, parent, lower, upper, step)") + ), + ), + ), + CIf( + CodeExpression("status != CFI_SUCCESS"), + body=( + CExpressionStatement( + CodeExpression( + 'PyErr_Format(PyExc_TypeError, "Argument %s could not be described to Fortran ' + 'as an array section: %d", argument_name, status)' + ) + ), + CReturn(CodeExpression("-1")), + ), + ), + CReturn(CodeExpression("0")), + ), + ), + ) + + def _descriptor_character_width_guard(self, plan: ArgumentTransferPlan, prefix: str) -> tuple: + """Decline storage whose element width is not the one the dummy declares. + + A character dummy is matched on its width as well as its kind, and the + descriptor states the width the storage actually has. + """ + declared = self._declared_character_width(plan) + if not declared: + return () + return ( + CComment("A character dummy is matched on its declared width."), + CIf( + CodeExpression(f"{prefix}_extents_out.elem_len != {declared}"), + body=( + CExpressionStatement( + CodeExpression( + "PyErr_Format(PyExc_TypeError, " + f"\"{plan.binding.python_name} does not match expected dtype dtype('S%d')\", " + f"{declared})" + ) + ), + CReturn(CodeExpression("NULL")), + ), + ), + ) + + def _module_describes_numpy_arrays(self, plan: ModulePlan) -> bool: + """Report whether any argument in this module takes the descriptor route.""" + return any( + self._array_crosses_as_descriptor(argument) + for function in self._functions(plan) + for argument in function.arguments + ) + + NATIVE_ARRAY_PROJECTION_RECORD = "prik_native_array_projection" + + def _native_array_projection_record(self) -> CStructDefinition: + """Define what one descriptor inquiry is asked for and what it reports.""" + return CStructDefinition( + self.NATIVE_ARRAY_PROJECTION_RECORD, + ( + CParameter("rank", "int"), + CParameter("numpy_type", "int"), + # The object a view must keep alive, or NULL when the storage + # outlives every view of it, as a module variable's does. + CParameter("base", "PyObject *"), + CParameter("result", "PyObject *"), + ), + ) + + def _native_array_projection_consumer( + self, + name: str, + body: tuple, + *doc: str, + ) -> CFunction: + """Wrap one descriptor reader in the shared consumer signature.""" + record = self.NATIVE_ARRAY_PROJECTION_RECORD + return CFunction( + name, + "void", + parameters=(CParameter("descriptor", "void *"), CParameter("context", "void *")), + storage="static", + body=( + CDeclaration("source", "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)descriptor")), + CDeclaration("out", f"{record} *", CodeExpression(f"({record} *)context")), + *body, + ), + doc=doc, + ) + + def _native_array_projection_functions(self, plan: ModulePlan) -> tuple: + """Emit the record and the consumers every handle inquiry runs.""" + if not self._reads_native_descriptors: + return () + return ( + self._native_array_projection_record(), + self._native_array_projection_consumer( + self._native_array_projection_name(NativeArrayOperation.ALLOCATED), + ( + CComment("A descriptor with no address describes storage that is not there."), + CExpressionStatement(CodeExpression("out->result = PyBool_FromLong(source->base_addr != NULL)")), + CReturn(), + ), + "Report whether this handle's storage is present.", + ), + self._native_array_projection_consumer( + self._native_array_projection_name(NativeArrayOperation.CONTIGUOUS), + ( + CComment("CFI_is_contiguous needs a descriptor that describes something,"), + CComment("and absent storage is not contiguous storage."), + CIf( + CodeExpression("source->base_addr == NULL"), + body=( + CExpressionStatement(CodeExpression("out->result = PyBool_FromLong(0)")), + CReturn(), + ), + ), + CExpressionStatement( + CodeExpression("out->result = PyBool_FromLong(CFI_is_contiguous(source) != 0)") + ), + CReturn(), + ), + "Report whether this handle's storage is contiguous.", + ), + self._native_array_projection_consumer( + self._native_array_projection_name(NativeArrayOperation.ELEMENT_LENGTH), + ( + CExpressionStatement( + CodeExpression("out->result = PyLong_FromLongLong((long long)source->elem_len)") + ), + CReturn(), + ), + "Report this handle's element width, which a deferred-length", + "character array only knows at run time.", + ), + self._native_array_projection_consumer( + self._native_array_projection_name(NativeArrayOperation.SHAPE), + ( + CDeclaration("shape", "PyObject *", CodeExpression("NULL")), + CDeclaration("extent", "PyObject *", CodeExpression("NULL")), + CDeclaration("axis", "int", CodeExpression("0")), + CComment("Storage that is not there has no shape to report."), + CIf( + CodeExpression("source->base_addr == NULL"), + body=( + CExpressionStatement(CodeExpression("out->result = Py_NewRef(Py_None)")), + CReturn(), + ), + ), + CExpressionStatement(CodeExpression("shape = PyTuple_New(out->rank)")), + CIf(CodeExpression("shape == NULL"), body=(CReturn(),)), + CFor( + "axis = 0", + CodeExpression("axis < out->rank"), + CodeExpression("++axis"), + body=( + CComment("A compiler may report an empty dimension as extent -1."), + CExpressionStatement( + CodeExpression( + "extent = PyLong_FromLongLong((long long)(source->dim[axis].extent == -1 " + "? 0 : source->dim[axis].extent))" + ) + ), + CIf( + CodeExpression("extent == NULL"), + body=( + CExpressionStatement(CodeExpression("Py_DECREF(shape)")), + CReturn(), + ), + ), + CExpressionStatement(CodeExpression("PyTuple_SET_ITEM(shape, axis, extent)")), + ), + ), + CExpressionStatement(CodeExpression("out->result = shape")), + CReturn(), + ), + "Report this handle's current extents as one Python tuple.", + ), + self._native_array_projection_consumer( + self._native_array_projection_name(NativeArrayOperation.DESCRIPTOR), + ( + CDeclaration("facts", "PyObject *", CodeExpression("NULL")), + CDeclaration("value", "PyObject *", CodeExpression("NULL")), + CDeclaration("axis", "int", CodeExpression("0")), + CDeclaration("slot", "int", CodeExpression("0")), + CExpressionStatement(CodeExpression("facts = PyTuple_New(3 + 3 * out->rank)")), + CIf(CodeExpression("facts == NULL"), body=(CReturn(),)), + CExpressionStatement( + CodeExpression("PyTuple_SET_ITEM(facts, 0, PyLong_FromVoidPtr(source->base_addr))") + ), + CExpressionStatement( + CodeExpression("PyTuple_SET_ITEM(facts, 1, PyLong_FromLongLong((long long)source->elem_len))") + ), + CExpressionStatement( + CodeExpression("PyTuple_SET_ITEM(facts, 2, PyLong_FromLong((long)out->rank))") + ), + CFor( + "axis = 0", + CodeExpression("axis < out->rank"), + CodeExpression("++axis"), + body=( + CExpressionStatement(CodeExpression("slot = 3 + 3 * axis")), + CExpressionStatement( + CodeExpression( + "PyTuple_SET_ITEM(facts, slot, " + "PyLong_FromLongLong((long long)source->dim[axis].lower_bound))" + ) + ), + CComment("A compiler may report an empty dimension as extent -1."), + CExpressionStatement( + CodeExpression( + "PyTuple_SET_ITEM(facts, slot + 1, PyLong_FromLongLong((long long)(" + "source->dim[axis].extent == -1 ? 0 : source->dim[axis].extent)))" + ) + ), + CExpressionStatement( + CodeExpression( + "PyTuple_SET_ITEM(facts, slot + 2, " + "PyLong_FromLongLong((long long)source->dim[axis].sm))" + ) + ), + ), + ), + CFor( + "slot = 0", + CodeExpression("slot < 3 + 3 * out->rank"), + CodeExpression("++slot"), + body=( + CExpressionStatement(CodeExpression("value = PyTuple_GET_ITEM(facts, slot)")), + CIf( + CodeExpression("value == NULL"), + body=( + CExpressionStatement(CodeExpression("Py_DECREF(facts)")), + CReturn(), + ), + ), + ), + ), + CExpressionStatement(CodeExpression("out->result = facts")), + CReturn(), + ), + "Report what this pointer is associated with, as the flat facts a", + "pointer with no storage of its own records and later replays.", + ), + self._native_array_projection_consumer( + self._native_array_projection_name(NativeArrayOperation.TO_NUMPY), + ( + CDeclaration("dimensions[PRIK_MAX_ARRAY_RANK]", "npy_intp"), + CDeclaration("strides[PRIK_MAX_ARRAY_RANK]", "npy_intp"), + CDeclaration("view", "PyObject *", CodeExpression("NULL")), + CDeclaration("axis", "int", CodeExpression("0")), + CComment("Storage that is not there exposes no view."), + CIf( + CodeExpression("source->base_addr == NULL"), + body=( + CExpressionStatement(CodeExpression("out->result = Py_NewRef(Py_None)")), + CReturn(), + ), + ), + CFor( + "axis = 0", + CodeExpression("axis < out->rank"), + CodeExpression("++axis"), + body=( + CComment("A compiler may report an empty dimension as extent -1."), + CExpressionStatement( + CodeExpression( + "dimensions[axis] = (npy_intp)(source->dim[axis].extent == -1 " + "? 0 : source->dim[axis].extent)" + ) + ), + CComment("The descriptor's stride multiplier is already the byte stride"), + CComment("NumPy wants, negative strides for a reversed target included."), + CExpressionStatement(CodeExpression("strides[axis] = (npy_intp)source->dim[axis].sm")), + ), + ), + CComment("The element width is only consulted for a flexible dtype, which is"), + CComment("how a character array takes its width from the descriptor."), + CExpressionStatement( + CodeExpression( + "view = PyArray_New(&PyArray_Type, out->rank, dimensions, out->numpy_type, strides, " + "source->base_addr, (int)source->elem_len, NPY_ARRAY_WRITEABLE, NULL)" + ) + ), + CIf(CodeExpression("view == NULL"), body=(CReturn(),)), + CComment("The view borrows this storage, so it retains whatever keeps the"), + CComment("storage valid: the parent object of a field, the capsule owning an"), + CComment("owned descriptor. A module variable outlives every view of it."), + CIf( + CodeExpression("out->base != NULL"), + body=( + CExpressionStatement(CodeExpression("Py_INCREF(out->base)")), + CIf( + CodeExpression("PyArray_SetBaseObject((PyArrayObject *)view, out->base) < 0"), + body=( + CExpressionStatement(CodeExpression("Py_DECREF(out->base)")), + CExpressionStatement(CodeExpression("Py_DECREF(view)")), + CReturn(), + ), + ), + ), + ), + CExpressionStatement(CodeExpression("out->result = view")), + CReturn(), + ), + "Build the NumPy view over this handle's storage.", + ), + ) + + @staticmethod + def _native_array_projection_name(operation: NativeArrayOperation) -> str: + """Return the shared consumer that answers one descriptor inquiry.""" + return f"prik_native_array_read_{operation.value}" + + def _native_array_projection_call_nodes( + self, + operation: NativeArrayOperation, + *, + rank: int, + numpy_type: str, + element_size: str, + descriptor_attribute: NativeArrayDescriptorAttribute, + base: str, + entry_point: str, + context: str, + ) -> tuple: + """Run one inquiry through a handle's descriptor entry point. + + A true allocatable or pointer descriptor reports absence itself. An + ordinary projection has no descriptor while its native entity is + absent, so its entry point does not call the consumer and the binding + returns the operation's absent value. + """ + record = self.NATIVE_ARRAY_PROJECTION_RECORD + consumer = self._native_array_projection_name( + NativeArrayOperation.ALLOCATED if operation is NativeArrayOperation.ASSOCIATED else operation + ) + if descriptor_attribute is NativeArrayDescriptorAttribute.OTHER: + missing = self._absent_native_array_projection_nodes(operation, rank, element_size) + else: + missing = ( + CExpressionStatement( + CodeExpression( + 'PyErr_SetString(PyExc_RuntimeError, "native array handle did not report a descriptor")' + ) + ), + ) + return ( + CDeclaration( + "projection", + record, + CodeExpression(f"{{{rank}, {numpy_type}, {base}, NULL}}"), + ), + CExpressionStatement(CodeExpression(f"{entry_point}({context}, {consumer}, &projection)")), + CIf( + CodeExpression("projection.result == NULL && !PyErr_Occurred()"), + body=missing, + ), + CReturn(CodeExpression("projection.result")), + ) + + @staticmethod + def _absent_native_array_projection_nodes( + operation: NativeArrayOperation, + rank: int, + element_size: str, + ) -> tuple: + """Return one inquiry result when an ordinary projection has no storage.""" + if operation in { + NativeArrayOperation.ALLOCATED, + NativeArrayOperation.ASSOCIATED, + NativeArrayOperation.CONTIGUOUS, + }: + return (CExpressionStatement(CodeExpression("projection.result = PyBool_FromLong(0)")),) + if operation in {NativeArrayOperation.SHAPE, NativeArrayOperation.TO_NUMPY}: + return (CExpressionStatement(CodeExpression("projection.result = Py_NewRef(Py_None)")),) + if operation is NativeArrayOperation.ELEMENT_LENGTH: + return ( + CExpressionStatement( + CodeExpression(f"projection.result = PyLong_FromSize_t((size_t)({element_size}))") + ), + ) + if operation is NativeArrayOperation.DESCRIPTOR: + facts = "absent_descriptor_facts" + assignments = [ + CExpressionStatement(CodeExpression(f"PyTuple_SET_ITEM({facts}, 0, PyLong_FromLongLong(0))")), + CExpressionStatement( + CodeExpression(f"PyTuple_SET_ITEM({facts}, 1, PyLong_FromSize_t((size_t)({element_size})))") + ), + CExpressionStatement(CodeExpression(f"PyTuple_SET_ITEM({facts}, 2, PyLong_FromLong({rank}))")), + ] + for axis in range(rank): + offset = 3 + 3 * axis + assignments.extend( + ( + CExpressionStatement( + CodeExpression(f"PyTuple_SET_ITEM({facts}, {offset}, PyLong_FromLongLong(0))") + ), + CExpressionStatement( + CodeExpression(f"PyTuple_SET_ITEM({facts}, {offset + 1}, PyLong_FromLongLong(0))") + ), + CExpressionStatement( + CodeExpression( + f"PyTuple_SET_ITEM({facts}, {offset + 2}, PyLong_FromSize_t((size_t)({element_size})))" + ) + ), + ) ) - ), - CExpressionStatement(CodeExpression(f"Py_DECREF({ops})")), - CExpressionStatement(CodeExpression(f"Py_DECREF({helper})")), - CExpressionStatement(CodeExpression(f"Py_DECREF({capsule})")), - CIf( - CodeExpression(f"{target} == NULL"), - body=(*failure_cleanup, CReturn(CodeExpression("NULL"))), - ), + return ( + CDeclaration(facts, "PyObject *", CodeExpression(f"PyTuple_New({3 + 3 * rank})")), + CIf(CodeExpression(f"{facts} == NULL"), body=(CReturn(CodeExpression("NULL")),)), + *assignments, + CIf( + CodeExpression("PyErr_Occurred()"), + body=( + CExpressionStatement(CodeExpression(f"Py_DECREF({facts})")), + CReturn(CodeExpression("NULL")), + ), + ), + CExpressionStatement(CodeExpression(f"projection.result = {facts}")), + ) + raise ValueError(f"Native array operation {operation.value!r} has no absent projection") + + def _array_actual_reader_functions(self, plan: ModulePlan) -> tuple: + """Emit the record and reader for each array dummy a handle may reach. + + An ordinary array dummy takes an address and extents, not a descriptor, + and an ordinary call cannot change the allocation behind them. The + storage is therefore read out of the descriptor once, while the runtime + holds it open, and the call proceeds on the address it names. + """ + nodes: list = [] + seen: set[str] = set() + if not self._reads_native_descriptors: + return () + for argument in self._inline_array_actual_handle_arguments(plan): + name = self._array_actual_struct_reader_name(argument) + if name in seen: + continue + seen.add(name) + nodes.append(self._array_actual_struct_reader_record(argument)) + nodes.append(self._array_actual_struct_reader_function(argument)) + for function, argument in self._array_actual_handle_arguments(plan): + record = self._array_actual_reader_record_name(function, argument) + if record in seen: + continue + seen.add(record) + rank = argument.array.rank + flat_axis = self._flattened_reader_axis(argument) + nodes.append( + CStructDefinition( + record, + ( + CParameter("data", "void *"), + CParameter(f"extents[{rank}]", "int64_t"), + CParameter("contiguous", "int"), + CParameter("present", "int"), + # 0 accepted, 1 no storage, 2 element width + CParameter("refused", "int"), + CParameter("width", "size_t"), + ), + ) + ) + if flat_axis is not None: + nodes.append(self._flattened_array_actual_reader(function, argument, record, rank, flat_axis)) + continue + body: list = [ + CDeclaration("source", "CFI_cdesc_t *", CodeExpression("(CFI_cdesc_t *)descriptor")), + CDeclaration("out", f"{record} *", CodeExpression(f"({record} *)context")), + CDeclaration("expected", "CFI_index_t", CodeExpression("0")), + CExpressionStatement(CodeExpression("out->present = 0")), + CExpressionStatement(CodeExpression("out->contiguous = 1")), + CExpressionStatement(CodeExpression("out->refused = 1")), + CComment("Unallocated or disassociated storage has no address to pass."), + CIf(CodeExpression("source->base_addr == NULL"), body=(CReturn(),)), + CExpressionStatement(CodeExpression("out->refused = 2")), + CExpressionStatement(CodeExpression("out->width = source->elem_len")), + CExpressionStatement(CodeExpression("expected = (CFI_index_t)source->elem_len")), + *self._declared_character_width_guard(argument), + CExpressionStatement(CodeExpression("out->refused = 0")), + ] + for axis in range(rank): + body.extend( + ( + # A compiler may report an empty dimension as extent -1. + CExpressionStatement( + CodeExpression( + f"out->extents[{axis}] = (int64_t)(source->dim[{axis}].extent == -1 " + f"? 0 : source->dim[{axis}].extent)" + ) + ), + CIf( + CodeExpression(f"source->dim[{axis}].sm != expected"), + body=(CExpressionStatement(CodeExpression("out->contiguous = 0")),), + ), + CExpressionStatement(CodeExpression(f"expected *= (CFI_index_t)out->extents[{axis}]")), + ) + ) + body.extend( + ( + CExpressionStatement(CodeExpression("out->data = source->base_addr")), + CExpressionStatement(CodeExpression("out->present = 1")), + ) + ) + nodes.append( + CFunction( + self._array_actual_reader_name(function, argument), + "void", + parameters=(CParameter("descriptor", "void *"), CParameter("context", "void *")), + storage="static", + body=tuple(body), + doc=( + "Copy one handle's storage out of the descriptor the runtime opened.", + "An ordinary array dummy receives that address and its extents.", + ), + ) + ) + return tuple(nodes) + + def _inverted_descriptor_consumer_functions(self, plan: ModulePlan) -> tuple: + """Emit the record and consumer chain for each entrypoint called inside one.""" + nodes: list = [] + for function in self._functions(plan): + context = self._function_context(function) + if not context.inverted_descriptors: + continue + nodes.append(self._inverted_context_record(function, context)) + nodes.extend(self._inverted_consumer_chain(function, context)) + return tuple(nodes) + + def _inverted_context_record(self, plan: FunctionPlan, context: _CFunctionContext) -> CStructDefinition: + """Declare everything the consumer chain carries between its links.""" + chain = self._inverted_chain_fields(plan, context) + fields = self._inverted_context_fields(plan, context) + result = self._direct_result(plan) + result_field = ( + (CParameter("result", self._inverted_result_type(plan, result)),) + if self._inverted_carries_result(plan) + else () + ) + return CStructDefinition( + self._inverted_context_name(plan), + tuple(declaration for declaration, _value in chain + fields) + result_field, ) - @staticmethod - def _derived_target_owner(handoff: DerivedHandoffPlan) -> str: - """Select the retained pointer-target owner from completed policy.""" - if handoff.target_owner_retention is DerivedOwnerRetention.NATIVE_MODULE: - return "self" - if handoff.target_owner_retention is DerivedOwnerRetention.NONE: - return "Py_None" - raise ValueError(f"Unsupported derived target owner retention: {handoff.target_owner_retention.value}") + def _inverted_consumer_chain(self, plan: FunctionPlan, context: _CFunctionContext) -> tuple[CFunction, ...]: + """Emit one consumer per descriptor, each entering the next. - def _holder_wrapper_symbols( - self, - type_symbol: str, - storage: DerivedObjectStorage, - ) -> tuple[str, str, str, str, str]: - """Return mechanical symbols for one completed holder storage choice.""" - if storage is DerivedObjectStorage.ALLOCATABLE_HOLDER: - return ( - self._allocatable_holder_capsule_name(type_symbol), - self._allocatable_holder_capsule_destructor_name(type_symbol), - self._allocatable_holder_destroy_bridge_name(type_symbol), - self._allocatable_holder_ops_name(type_symbol), - storage.value, + Every link records the descriptor it was handed and then enters the one + after it, so by the time the last link runs, every descriptor the call + needs is live at once and none of them has been copied or outlived the + entity it describes. The call is made there, where a callee that + reallocates or reassociates any of its arguments writes into the + descriptor the Fortran runtime copies back to that caller's entity. + """ + record = self._inverted_context_name(plan) + slots = len(context.inverted_descriptors) + fields = self._inverted_context_fields(plan, context) + functions: list[CFunction] = [] + for slot in range(slots): + last = slot == slots - 1 + body: tuple = ( + CExpressionStatement( + CodeExpression(f"call->{self._inverted_descriptor_field(slot)} = (CFI_cdesc_t *)descriptor") + ), ) - if storage is DerivedObjectStorage.POINTER_HOLDER: - return ( - self._pointer_holder_capsule_name(type_symbol), - self._pointer_holder_capsule_destructor_name(type_symbol), - self._pointer_holder_destroy_bridge_name(type_symbol), - self._pointer_holder_ops_name(type_symbol), - storage.value, + if last: + body += self._lower_native_call( + plan, + CExpressionStatement(CodeExpression(self._inverted_consumer_call(plan, context, fields))), + ) + doc = ( + f"Call {self._entrypoint_function_name(plan)} with every descriptor live.", + "Each argument's descriptor was recorded by the consumer that was handed" + " it, and every one of those is still running, so what the callee writes" + " into any of them is what Fortran copies back to that caller's entity.", + "Every other value the call needs arrives through the context record," + " because this runs outside the frame that computed them.", + ) + else: + body += self._inverted_enter_nodes( + plan, + slot + 1, + backend=f"call->{self._inverted_backend_field(slot + 1)}", + call_context="call", + placeholder=f"call->{self._inverted_descriptor_field(slot + 1)}", + ) + doc = ( + f"Record descriptor {slot} of {self._entrypoint_function_name(plan)} and enter the next.", + "This descriptor is valid only while this function runs, so the call is" + " not made until every argument's descriptor has been entered and they" + " are all live together.", + ) + functions.append( + CFunction( + self._inverted_consumer_name(plan, slot), + "void", + parameters=(CParameter("descriptor", "void *"), CParameter("context", "void *")), + storage="static", + doc=doc, + body=( + CDeclaration("call", f"{record} *", CodeExpression(f"({record} *)context")), + *body, + CReturn(), + ), + ) ) - raise ValueError(f"Unsupported derived holder storage: {storage.value}") + # A link may only be named once the one it enters has been defined. + return tuple(reversed(functions)) - # Scalar result lowering. - def _lower_result_direct_value( + def _inverted_carries_result(self, plan: FunctionPlan) -> bool: + """Report whether the entrypoint returns a value the chain must carry out. + + A result the entrypoint writes through a parameter is already reaching + this frame by address; only a returned value has to be brought back. + """ + return self._direct_result(plan) is not None and self._entrypoint_return_type(plan) != "void" + + def _inverted_result_type(self, plan: FunctionPlan, result) -> str: + """Return the C storage a carried direct result is written into. + + It is whatever the entrypoint returns, which is not always the scalar + the semantic type names: a function whose result is an array returns + the address of its storage. + """ + del result + return self._entrypoint_return_type(plan) + + def _inverted_consumer_call( self, - plan: ResultPlan, + plan: FunctionPlan, context: _CFunctionContext, - failure_cleanup: tuple[str, ...], - failure_label: str | None = None, - pending_native_cleanup: tuple[CExpressionStatement, ...] = (), - ) -> tuple[CExpressionStatement | CDeclaration | CIf, ...]: - """Lower result direct value from the supplied completed binding records without inferring semantic policy.""" - return self._lower_result_value( - plan, - context, - failure_cleanup, - failure_label, - pending_native_cleanup, - ) + fields: tuple[tuple[CParameter, str], ...], + ) -> str: + """Assemble the entrypoint call as the consumer makes it.""" + carried = {value: f"call->{declaration.name}" for declaration, value in fields} + recorded = { + context.arguments[owner_path].value_name: f"call->{self._inverted_descriptor_field(slot)}" + for slot, owner_path in enumerate(context.inverted_descriptors) + } + owners = set(context.inverted_descriptors) + arguments = [] + for group in sorted(plan.entrypoint.parameters, key=lambda item: item.position): + values = self._entrypoint_parameter_values(plan, group, context) + if group.owner_path in owners: + # Only the descriptor itself comes from its slot. An optional + # one is planned alongside its present flag, and that flag is an + # ordinary carried value like any other. + arguments.extend(recorded.get(value) or carried[value] for value in values) + continue + arguments.extend(carried[value] for value in values) + call = f"{self._entrypoint_function_name(plan)}({', '.join(arguments)})" + return f"call->result = {call}" if self._inverted_carries_result(plan) else call - def _lower_result_value( + def _inverted_context_name(self, plan: FunctionPlan) -> str: + """Return the record carrying one inverted call's other values.""" + return f"{self._binding_function_name(plan)}_call_context" + + def _inverted_context_fields( self, - plan: ResultPlan, + plan: FunctionPlan, context: _CFunctionContext, - failure_cleanup: tuple[str, ...], - failure_label: str | None = None, - pending_native_cleanup: tuple[CExpressionStatement, ...] = (), - ) -> tuple[CExpressionStatement | CDeclaration | CIf, ...]: - """Convert one native result into its binding-owned Python consumer.""" - scalar_type = PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name) - native_name = self._result_native_name(plan, context) - python_name = context.python_results.get(plan.owner_path) - if scalar_type.python_result_kind is None or python_name is None: - raise ValueError(f"Unsupported scalar result type {plan.semantic_type_name!r}") - converted_name = native_name - conversion = () - if plan.entrypoint.native_scalar_c_type is not None: - converted_name = f"{native_name}_contract" - conversion = ( - CDeclaration( - converted_name, - scalar_type.c_spelling, - CodeExpression(f"({scalar_type.c_spelling}){native_name}"), - ), + ) -> tuple[tuple[CParameter, str], ...]: + """Pair every entrypoint value the consumers need with its declaration. + + The descriptors themselves are excluded: each arrives as its own + consumer's argument and is recorded in the slot named for it. + Everything else the call needs is carried through the context record, + because the consumers run outside the frame that computed it -- + including the present flag planned beside an optional descriptor, which + is a value like any other, and the address of a hidden output, which + stays valid because this frame outlives every consumer it enters. + """ + descriptor_values = { + context.arguments[owner_path].value_name: owner_path for owner_path in context.inverted_descriptors + } + pairs: list[tuple[CParameter, str]] = [] + for group in sorted(plan.entrypoint.parameters, key=lambda item: item.position): + declarations = self._entrypoint_parameter_declarations(plan, group) + values = self._entrypoint_parameter_values(plan, group, context) + if len(declarations) != len(values): + raise ValueError(f"Entrypoint parameter {group.owner_path!r} has mismatched declarations and values") + pairs.extend( + (declaration, value) + for declaration, value in zip(declarations, values, strict=True) + if descriptor_values.get(value) != group.owner_path + ) + return tuple(pairs) + + def _inverted_chain_fields( + self, + plan: FunctionPlan, + context: _CFunctionContext, + ) -> tuple[tuple[CParameter, str], ...]: + """Pair the slot each descriptor is recorded in with what starts it. + + A slot begins as whatever this frame already has for it: nothing for a + handle that will be entered, and the unallocated placeholder for an + absent optional, which is never entered and so is passed on as it + stands. Every backend after the first is carried too, because the + consumer that enters it runs outside this frame. + """ + pairs: list[tuple[CParameter, str]] = [] + for slot, owner_path in enumerate(context.inverted_descriptors): + if slot > 0: + pairs.append( + ( + CParameter(self._inverted_backend_field(slot), "prik_native_array_backend *"), + self._descriptor_backend_local(context.arguments[owner_path]), + ) + ) + for slot, owner_path in enumerate(context.inverted_descriptors): + pairs.append( + ( + CParameter(self._inverted_descriptor_field(slot), "CFI_cdesc_t *"), + context.arguments[owner_path].value_name, + ) ) + return tuple(pairs) + + @staticmethod + def _descriptor_backend_local(names: _CArgumentNames) -> str: + """Return the local holding one descriptor argument's backend.""" + return f"{names.value_name}_native_backend" + + @staticmethod + def _inverted_descriptor_field(slot: int) -> str: + """Return the record slot one entered descriptor is recorded in.""" + return f"descriptor_{slot}" + + @staticmethod + def _inverted_backend_field(slot: int) -> str: + """Return the record slot one not-yet-entered backend is carried in.""" + return f"backend_{slot}" + + def _inverted_consumer_name(self, plan: FunctionPlan, slot: int) -> str: + """Return the consumer that enters one descriptor of an inverted call.""" + return f"{self._binding_function_name(plan)}_call_with_descriptor_{slot}" + + def _inverted_enter_nodes( + self, + plan: FunctionPlan, + slot: int, + *, + backend: str, + call_context: str, + placeholder: str, + ) -> tuple[CIf, ...]: + """Enter one descriptor's backend, or pass on what stands for it. + + A backend is absent only for an optional argument that was not + supplied: everything else publishes one, whether it borrows its + descriptor or owns it. There is no entity to enter for an absent one, + so the placeholder already recorded in its slot goes straight to the + same consumer and the chain continues from there. + """ + consumer = self._inverted_consumer_name(plan, slot) return ( - *conversion, - CDeclaration( - python_name, - "PyObject *", - CodeExpression(self._scalar_result_expression(scalar_type, f"&{converted_name}")), - ), CIf( - CodeExpression(f"{python_name} == NULL"), + CodeExpression(f"{backend} != NULL"), body=( - *pending_native_cleanup, - *self._output_failure_nodes(failure_cleanup, failure_label), + CExpressionStatement( + CodeExpression(f"{backend}->with_descriptor({backend}->context, {consumer}, {call_context})") + ), + ), + else_body=( + CComment("This optional argument is absent, so there is nothing to enter for it."), + CExpressionStatement(CodeExpression(f"{consumer}({placeholder}, {call_context})")), ), ), ) - def _result_native_name(self, plan: ResultPlan, context: _CFunctionContext) -> str: - """Return the validated C storage consumed by one result conversion.""" - if plan.source_kind == "direct_return": - if context.result_name is None: - raise ValueError(f"Direct result {plan.owner_path!r} has no C storage") - return context.result_name - try: - return context.native_outputs[plan.entrypoint.native_result_role] - except KeyError: - raise ValueError(f"Hidden result {plan.owner_path!r} has no C output storage") from None - def _output_nodes( self, plan: FunctionPlan, @@ -9126,7 +11201,7 @@ def _output_nodes( """Return the native envelope, status projection, and Python result.""" nodes = [ *self._callback_context_push_nodes(plan, context), - *self._lower_native_call(plan, self._entrypoint_call_statement(plan, context)), + *self._lower_entrypoint_call(plan, context), *self._callback_context_pop_nodes(plan), *self._derived_call_failure_nodes(plan, context), *self._derived_after_native_failure_nodes(plan, context), @@ -10108,6 +12183,8 @@ def _function_context(self, plan: FunctionPlan) -> _CFunctionContext: python_result, python_results, role_values, + tuple(argument.owner_path for argument in self._inverted_descriptor_arguments(plan)), + plan, ) def _argument_contexts(self, plan: FunctionPlan) -> dict[str, _CArgumentNames]: @@ -10384,7 +12461,7 @@ def _native_call_setup_nodes( cfi_type = self._native_array_cfi_type(result) if cfi_type is None: raise ValueError(f"Owned result {result.owner_path!r} is missing a CFI element type") - elem_len = f"sizeof({PrimitiveScalarTypeRegistry.type_for(result.semantic_type_name).c_spelling})" + elem_len = f"sizeof({PrimitiveScalarTypeRegistry.type_for(result.semantic_type_name).array_c_spelling})" cleanup = tuple( node for previous_result, previous_descriptor in reversed(initialized) @@ -10510,19 +12587,33 @@ def _native_array_handle_kind_constant(handle: NativeArrayHandlePlan) -> str: return "PRIK_NATIVE_ARRAY_KIND_POINTER" return "PRIK_NATIVE_ARRAY_KIND_ALLOCATABLE" + @staticmethod + def _native_array_descriptor_attribute_constant(handle: NativeArrayHandlePlan) -> str: + """Return the descriptor attribute the completed backend supplies.""" + return { + NativeArrayDescriptorAttribute.ALLOCATABLE: "PRIK_NATIVE_ARRAY_ATTRIBUTE_ALLOCATABLE", + NativeArrayDescriptorAttribute.POINTER: "PRIK_NATIVE_ARRAY_ATTRIBUTE_POINTER", + NativeArrayDescriptorAttribute.OTHER: "PRIK_NATIVE_ARRAY_ATTRIBUTE_OTHER", + }[handle.descriptor_attribute] + @staticmethod def _native_array_expected_element_size(plan: ArgumentTransferPlan | ResultPlan) -> str: """Return a fixed element-size check or zero for runtime-width strings.""" if plan.datatype_family is DatatypeFamily.STRING: return "0" - return f"sizeof({PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).c_spelling})" + return f"sizeof({PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_c_spelling})" def _native_array_capsule_new_expression( self, plan: ArgumentTransferPlan | ResultPlan, descriptor: str, ) -> str: - """Create one versioned capsule around established descriptor storage.""" + """Publish one backend over descriptor storage this wrapper owns. + + There is no native entity to enter, so the entry point hands the + persistent storage straight to the consumer, and the release callback + marks that storage as this extension's to free. + """ handle = plan.native_array_handle cfi_type = self._native_array_cfi_type(plan) element_size = ( @@ -10531,10 +12622,11 @@ def _native_array_capsule_new_expression( else self._native_array_expected_element_size(plan) ) return ( - "prik_native_array_handle_capsule_new(" - f"{self._native_array_handle_kind_constant(handle)}, {handle.array.rank}, {cfi_type}, " - f"{element_size}, sizeof(CFI_CDESC_T({handle.array.rank})), {descriptor}, " - f"{self._native_array_capsule_release_name(plan)})" + "prik_native_array_backend_capsule_new(" + f"{self._native_array_handle_kind_constant(handle)}, " + f"{self._native_array_descriptor_attribute_constant(handle)}, {handle.array.rank}, " + f"(uint32_t)sizeof(CFI_CDESC_T({handle.array.rank})), {cfi_type}, {element_size}, {descriptor}, " + f"prik_native_array_owned_with_descriptor, {self._native_array_capsule_release_name(plan)})" ) # Binding-owned representation transformations. @@ -10568,8 +12660,30 @@ def _binding_transformation_setup_nodes( CodeExpression(f"{temporary} == NULL"), body=(*prior_cleanup, CReturn(CodeExpression("NULL"))), ), - CExpressionStatement( - CodeExpression(f"{names.value_name} = PyArray_DATA((PyArrayObject *){temporary})") + *( + ( + CIf( + CodeExpression( + f"{self.NUMPY_DESCRIPTOR_BUILDER}((CFI_cdesc_t *)&{names.value_name}_parent, " + f"(CFI_cdesc_t *)&{names.value_name}_section, (PyArrayObject *){temporary}, " + f'{self._native_array_cfi_type(argument)}, "{argument.binding.python_name}") < 0' + ), + body=( + CExpressionStatement(CodeExpression(f"Py_DECREF({temporary})")), + *prior_cleanup, + CReturn(CodeExpression("NULL")), + ), + ), + CExpressionStatement( + CodeExpression(f"{names.value_name} = (CFI_cdesc_t *)&{names.value_name}_section") + ), + ) + if self._array_crosses_as_descriptor(argument) + else ( + CExpressionStatement( + CodeExpression(f"{names.value_name} = PyArray_DATA((PyArrayObject *){temporary})") + ), + ) ), ) ) @@ -10977,6 +13091,8 @@ def _entrypoint_argument_values( if plan.entrypoint.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER: return self._string_entrypoint_argument_values(plan, names, passing=passing) if plan.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER: + if self._array_crosses_as_descriptor(plan): + return (names.value_name,) return self._array_entrypoint_argument_values(plan, names) if plan.entrypoint.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR: return (names.value_name,) @@ -11273,6 +13389,10 @@ def _ordinary_entrypoint_argument_parameters( if argument.entrypoint.handoff_mode is ArgumentHandoffMode.CHARACTER_BUFFER: return self._string_entrypoint_argument_parameters(argument, name, passing=passing) if argument.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER: + if self._array_crosses_as_descriptor(argument): + # Extents and strides travel inside the descriptor, so the + # address and the fields beside it are not needed. + return (CParameter(name, "CFI_cdesc_t *"),) return self._array_entrypoint_argument_parameters(argument, name) if argument.entrypoint.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR: parameters = [CParameter(name, "CFI_cdesc_t *")] @@ -11612,14 +13732,6 @@ def _pointer_holder_ops_name(type_name: str) -> str: """Return the binding-local pointer holder ops name derived from the supplied local lowering values; this helper preserves completed policy.""" return CBindingNames.pointer_holder_ops(type_name) - def _derived_field_descriptor_callback_name( - self, - derived: DerivedTypePlan, - field: DerivedFieldPlan, - ) -> str: - """Return the binding-local derived field descriptor callback name derived from the supplied completed binding records; this helper preserves completed policy.""" - return f"prik_field_{self._derived_field_symbol(derived, field)}_descriptor" - def _derived_handle_operation_name( self, derived: DerivedTypePlan, @@ -11649,14 +13761,6 @@ def _derived_handle_descriptor_callback_name( """Return the binding-local derived handle descriptor callback name derived from the supplied completed binding records; this helper preserves completed policy.""" return f"prik_field_handle_{self._derived_field_symbol(derived, field)}_descriptor_callback" - def _derived_handle_actual_callback_name( - self, - derived: DerivedTypePlan, - field: DerivedFieldPlan, - ) -> str: - """Return the binding-local derived handle actual callback name derived from the supplied completed binding records; this helper preserves completed policy.""" - return f"prik_field_handle_{self._derived_field_symbol(derived, field)}_actual_callback" - @staticmethod def _module_member_symbol(variable: ModuleVariablePlan, member: DerivedMemberPathPlan) -> str: """Return the binding-local module member symbol derived from the supplied completed binding records; this helper preserves completed policy.""" @@ -11682,14 +13786,6 @@ def _module_member_bridge_name( ".".join((variable.owner_path, *member.path)), f"field:module:{action}" ).symbol_name - def _module_member_descriptor_callback_name( - self, - variable: ModuleVariablePlan, - member: DerivedMemberPathPlan, - ) -> str: - """Return the binding-local module member descriptor callback name derived from the supplied completed binding records; this helper preserves completed policy.""" - return f"prik_module_field_{self._module_member_symbol(variable, member)}_descriptor" - def _module_member_handle_operation_name( self, variable: ModuleVariablePlan, @@ -11719,14 +13815,6 @@ def _module_member_handle_descriptor_callback_name( """Return the binding-local module member handle descriptor callback name derived from the supplied completed binding records; this helper preserves completed policy.""" return f"prik_module_field_handle_{self._module_member_symbol(variable, member)}_descriptor_callback" - def _module_member_handle_actual_callback_name( - self, - variable: ModuleVariablePlan, - member: DerivedMemberPathPlan, - ) -> str: - """Return the binding-local module member handle actual callback name derived from the supplied completed binding records; this helper preserves completed policy.""" - return f"prik_module_field_handle_{self._module_member_symbol(variable, member)}_actual_callback" - @staticmethod def _module_member_ops_name(variable: ModuleVariablePlan, prefix: tuple[str, ...]) -> str: """Return the binding-local module member ops name derived from the supplied completed binding records; this helper preserves completed policy.""" diff --git a/prik/codegen/docstrings.py b/prik/codegen/docstrings.py index 654b71632..db7eaa0c6 100644 --- a/prik/codegen/docstrings.py +++ b/prik/codegen/docstrings.py @@ -53,6 +53,20 @@ "String": "str", } +# An aliased array reports the width its Fortran elements really occupy. NumPy +# has no Boolean wider than one byte, and no Fortran kind promises the byte +# values zero and one that its `bool_` requires, so a logical array is described +# by the integer of matching width and read back with `.astype(bool)`. +_ARRAY_ELEMENT_TYPES = { + "Bool": "uint8", + "Bool8": "uint8", + "Bool16": "int16", + "Bool32": "int32", + "Bool64": "int64", +} + +_LOGICAL_ARRAY_NOTE = "Fortran logical elements; compare with .astype(bool) rather than to 1." + _UNKNOWN_EXTENTS = frozenset({"", ":", "::", "*", ".."}) @@ -457,6 +471,7 @@ def module_variable(self, variable: ModuleVariablePlan) -> str: nullable = variable.binding.getter_action is ModuleGetterAction.NULLABLE_SNAPSHOT lines = [f"{name} : {self._type(variable, nullable=nullable, signature=False)}"] lines.extend(self._array_lines(variable.array)) + lines.extend(self._logical_array_lines(variable)) if variable.binding.getter_action in { ModuleGetterAction.CONSTANT_VALUE, ModuleGetterAction.NATIVE_CONSTANT_VALUE, @@ -473,6 +488,20 @@ def module_variable(self, variable: ModuleVariablePlan) -> str: lines.append(" Replacement assignment is not supported.") return "\n".join(lines) + @staticmethod + def _logical_array_lines(transfer) -> tuple[str, ...]: + """Explain the dtype a Fortran logical array reports, where it has one. + + The integer width is the storage the elements actually occupy, so the + note says how to read it rather than leaving the caller to guess that + the values are Booleans. + """ + if getattr(transfer, "semantic_type_name", None) not in _ARRAY_ELEMENT_TYPES: + return () + if getattr(transfer, "array", None) is None and getattr(transfer, "native_array_handle", None) is None: + return () + return (f" {_LOGICAL_ARRAY_NOTE}",) + def field(self, field: DerivedFieldPlan) -> str: """Render one generated class-property docstring from its field plan. @@ -896,6 +925,7 @@ def _base_type(self, transfer) -> str: if getattr(transfer, "datatype_family", None) is DatatypeFamily.DERIVED: return transfer.semantic_type_name scalar = _SCALAR_TYPES.get(transfer.semantic_type_name, transfer.semantic_type_name) + array_element = _ARRAY_ELEMENT_TYPES.get(transfer.semantic_type_name, scalar) handle = getattr(transfer, "native_array_handle", None) if handle is not None: prefix = ( @@ -903,9 +933,9 @@ def _base_type(self, transfer) -> str: if handle.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE else "PointerArray" ) - return f"{prefix}[{scalar}]" + return f"{prefix}[{array_element}]" if getattr(transfer, "array", None) is not None: - element = "bytes" if transfer.semantic_type_name == "String" else scalar + element = "bytes" if transfer.semantic_type_name == "String" else array_element return f"ndarray[{self._exact_array_element_label(transfer, element)}]" return scalar @@ -974,6 +1004,10 @@ def _array_layout_label(array: ArrayHandoffPlan) -> str | None: """Render the layout every accepted actual must already have, if any.""" if array.python_layout is ArrayPythonLayout.ANY_STRIDED: return "Any strides" + if array.python_layout is ArrayPythonLayout.SIGNED_STRIDED_F: + return "Fortran-ordered strides, including reversed axes" + if array.python_layout is ArrayPythonLayout.POSITIVE_STRIDED_F: + return "Fortran-ordered positive strides" if (array.rank is None or array.rank > 1) and array.order in {"ORDER_C", "ORDER_F"}: return "C-contiguous" if array.order == "ORDER_C" else "F-contiguous" return None diff --git a/prik/codegen/fortran/bridge.py b/prik/codegen/fortran/bridge.py index f1c9130c1..13742326e 100644 --- a/prik/codegen/fortran/bridge.py +++ b/prik/codegen/fortran/bridge.py @@ -13,6 +13,7 @@ from dataclasses import replace import re +from prik.naming.native_symbols import NativeSymbolNames from prik.utilities.declaration_expressions import render_declaration_extent from prik.policy.ownership import ( AssignmentMode, @@ -24,6 +25,7 @@ from prik.semantics.metadata import SCALAR_STORAGE_CATEGORY from prik.policy.models import ( ArgumentHandoffMode, + ArrayEntrypointABI, ArrayLogicalABI, ArrayWritebackABI, BridgeDataAction, @@ -40,10 +42,12 @@ DeclarationCallableAction, DirectResultABI, ExternalDeclarationMode, + ModuleArrayAddressMechanism, ModuleGetterAction, ModuleObjectAccessMechanism, CharacterLocalRelease, NativeArrayDescriptorKind, + NativeArrayDescriptorAttribute, NativeArrayDescriptorInterop, NativeArrayDefaultConstruction, NativeArrayOperation, @@ -109,6 +113,24 @@ from prik.codegen.visitor import ClassVisitor +# The C identity function that reports a non-target module array's base +# address. The binding defines it; the bridge declares and calls it. +_MODULE_ARRAY_CAPTURE_NAME = "prik_capture_address" + +# The binding answers these from the live descriptor the handle's entry point +# supplies, so the bridge emits no procedure of its own for them. +_DESCRIPTOR_ANSWERED_OPERATIONS = frozenset( + { + NativeArrayOperation.ALLOCATED, + NativeArrayOperation.ASSOCIATED, + NativeArrayOperation.CONTIGUOUS, + NativeArrayOperation.DESCRIPTOR, + NativeArrayOperation.ELEMENT_LENGTH, + NativeArrayOperation.SHAPE, + NativeArrayOperation.TO_NUMPY, + } +) + _MODULE_GETTER_SUMMARIES = { ModuleGetterAction.CONSTANT_VALUE: "The value is a compile-time constant materialized by the binding.", ModuleGetterAction.NATIVE_CONSTANT_VALUE: "Returns the compiler-evaluated constant by value.", @@ -324,6 +346,7 @@ def _visit_ModulePlan(self, plan: ModulePlan) -> FortranModule: *self._external_interfaces(plan), *self._module_descriptor_callback_interfaces(plan), *self._derived_array_callback_interfaces(plan), + *self._module_array_capture_interfaces(plan), *self._allocator_interfaces(plan), ), declarations=self._prototype_entity_declarations(plan), @@ -462,7 +485,7 @@ def _support_procedure_fortran_type(value: NativeEntrypointABIValuePlan) -> str # declares is spelled. length = ":" if value.character_length is None else str(value.character_length) return f"character(kind=c_char, len={length})" - return PrimitiveScalarTypeRegistry.type_for(value.semantic_type_name).fortran_spelling + return PrimitiveScalarTypeRegistry.type_for(value.semantic_type_name).array_fortran_type try: return types[value.kind] except KeyError: @@ -1896,92 +1919,17 @@ def _owned_native_array_result_operation( result: ArgumentTransferPlan | ResultPlan, operation: NativeArrayOperation, ) -> FortranFunction | None: - """Dispatch one generated operation selected by completed handle policy.""" - if operation in {NativeArrayOperation.ALLOCATED, NativeArrayOperation.ASSOCIATED}: - return self._owned_native_array_result_state_operation(result, operation) - if operation is NativeArrayOperation.CONTIGUOUS: - return self._owned_native_array_result_contiguous_operation(result) - if operation is NativeArrayOperation.SHAPE: - return self._owned_native_array_result_shape_operation(result) + """Dispatch one generated operation selected by completed handle policy. + + An owned handle already holds its descriptor, so every inquiry is read + from it in the binding and only the mutations reach Fortran. + """ if operation is NativeArrayOperation.ASSOCIATE: return self._owned_native_array_result_associate_operation(result) if operation in {NativeArrayOperation.DEALLOCATE, NativeArrayOperation.NULLIFY, NativeArrayOperation.DESTROY}: return self._owned_native_array_result_release_operation(result, operation) return None - def _owned_native_array_result_state_operation( - self, - result: ArgumentTransferPlan | ResultPlan, - operation: NativeArrayOperation, - ) -> FortranFunction: - """Return descriptor presence using its completed compiler inquiry.""" - inquiry = self._owned_native_array_result_presence_inquiry(result) - name = self._owned_native_array_result_operation_name(result, operation) - return FortranFunction( - name=name, - parameters=(self._owned_native_array_result_parameter(result, intent="in"),), - result_name="state", - result_type="logical(c_bool)", - bind_name=name, - body=(FortranAssignment("state", CodeExpression(f"{inquiry}(result)")),), - ) - - def _owned_native_array_result_contiguous_operation( - self, - result: ArgumentTransferPlan | ResultPlan, - ) -> FortranFunction: - """Return target contiguity without querying an absent pointer target.""" - name = self._owned_native_array_result_operation_name(result, NativeArrayOperation.CONTIGUOUS) - return FortranFunction( - name=name, - parameters=(self._owned_native_array_result_parameter(result, intent="in"),), - result_name="state", - result_type="logical(c_bool)", - bind_name=name, - body=( - FortranAssignment("state", CodeExpression(".false._c_bool")), - FortranIf( - CodeExpression("associated(result)"), - body=(FortranAssignment("state", CodeExpression("is_contiguous(result)")),), - ), - ), - ) - - def _owned_native_array_result_shape_operation( - self, - result: ArgumentTransferPlan | ResultPlan, - ) -> FortranFunction: - """Return shape through Fortran when the owned descriptor is allocated.""" - handle = result.native_array_handle - if handle is None or handle.array.rank is None: - raise ValueError(f"Owned result {result.owner_path!r} has no shape rank") - name = self._owned_native_array_result_operation_name(result, NativeArrayOperation.SHAPE) - extents = tuple(FortranParameter(f"extent_{axis}", "integer(c_int64_t)") for axis in range(handle.array.rank)) - present = tuple( - FortranAssignment( - f"extent_{axis}", - CodeExpression(f"size(result, {axis + 1}, kind=c_int64_t)"), - ) - for axis in range(handle.array.rank) - ) - absent = tuple( - FortranAssignment(f"extent_{axis}", CodeExpression("0_c_int64_t")) for axis in range(handle.array.rank) - ) - inquiry = self._owned_native_array_result_presence_inquiry(result) - return FortranFunction( - name=name, - parameters=(self._owned_native_array_result_parameter(result, intent="in"), *extents), - bind_name=name, - body=( - FortranIf( - CodeExpression(f"{inquiry}(result)"), - body=present, - else_body=absent, - ), - ), - is_subroutine=True, - ) - def _owned_native_array_result_release_operation( self, result: ArgumentTransferPlan | ResultPlan, @@ -2536,8 +2484,6 @@ def _lower_module_native_array_bridge_operation( return self._module_native_array_state_operation(plan, operation) if operation is NativeArrayOperation.ELEMENT_LENGTH: return self._module_native_array_element_length_operation(plan) - if operation is NativeArrayOperation.ARRAY_ACTUAL: - return self._module_native_array_actual_operation(plan) if operation is NativeArrayOperation.SHAPE: return self._module_native_array_shape_operation(plan) if operation is NativeArrayOperation.DESCRIPTOR: @@ -2571,29 +2517,6 @@ def _module_native_array_state_operation(self, plan: ModuleVariablePlan, operati body=(FortranAssignment("result", CodeExpression(expression)),), ) - def _module_native_array_actual_operation(self, plan: ModuleVariablePlan) -> FortranFunction: - """Return current module-array data storage without changing ownership.""" - if self._uses_module_allocatable_descriptor(plan): - return self._module_allocatable_descriptor_callback_operation( - plan, - NativeArrayOperation.ARRAY_ACTUAL, - ) - name = self._module_native_array_operation_name(plan, NativeArrayOperation.ARRAY_ACTUAL) - native = self._native_variable_name(plan) - return FortranFunction( - name=name, - result_name="result", - result_type="type(c_ptr)", - bind_name=name, - body=( - FortranIf( - CodeExpression(self._module_native_array_presence_expression(plan)), - body=(FortranAssignment("result", CodeExpression(f"c_loc({native})")),), - else_body=(FortranAssignment("result", CodeExpression("c_null_ptr")),), - ), - ), - ) - def _module_native_array_element_length_operation(self, plan: ModuleVariablePlan) -> FortranFunction: """Return the runtime character element width or zero when absent.""" name = self._module_native_array_operation_name(plan, NativeArrayOperation.ELEMENT_LENGTH) @@ -2613,7 +2536,7 @@ def _module_native_array_element_length_operation(self, plan: ModuleVariablePlan ) def _module_native_array_shape_operation(self, plan: ModuleVariablePlan) -> FortranFunction: - """Return current extents, preserving absent descriptor state as zeroes.""" + """Return whether storage is present and write its current extents.""" handle = plan.native_array_handle if handle is None or handle.array.rank is None: raise ValueError(f"Module handle {plan.owner_path!r} has no shape rank") @@ -2632,68 +2555,70 @@ def _module_native_array_shape_operation(self, plan: ModuleVariablePlan) -> Fort return FortranFunction( name=name, parameters=parameters, + result_name="result", + result_type="logical(c_bool)", bind_name=name, body=( + FortranAssignment("result", CodeExpression(self._module_native_array_presence_expression(plan))), FortranIf( CodeExpression(self._module_native_array_presence_expression(plan)), body=present, else_body=absent, ), ), - is_subroutine=True, ) def _module_native_array_descriptor_operation(self, plan: ModuleVariablePlan) -> FortranFunction | None: """Expose current module descriptor state through the selected mechanism.""" - handle = plan.native_array_handle - if self._uses_module_allocatable_descriptor(plan): - return self._module_allocatable_descriptor_callback_operation( + if self._uses_module_descriptor_backend(plan): + return self._module_descriptor_callback_operation( plan, NativeArrayOperation.DESCRIPTOR, ) - if handle is None or handle.descriptor_kind is not NativeArrayDescriptorKind.POINTER: - return None - if handle.array.rank is None: - raise ValueError(f"Pointer module handle {plan.owner_path!r} has no descriptor rank") - name = self._module_native_array_operation_name(plan, NativeArrayOperation.DESCRIPTOR) - native = self._native_variable_name(plan) - return FortranFunction( - name=name, - parameters=( - FortranParameter( - "descriptor", - self._module_pointer_dummy_element_type(plan), - ("pointer", self._array_dimension_attribute(handle.array.rank), "intent(out)"), - ), - ), - bind_name=name, - body=( - FortranIf( - CodeExpression(f"associated({native})"), - body=(FortranPointerAssignment("descriptor", CodeExpression(native)),), - else_body=(FortranPointerAssignment("descriptor", CodeExpression("null()")),), - ), - ), - is_subroutine=True, - ) + return None @staticmethod - def _uses_module_allocatable_descriptor(plan: ModuleVariablePlan) -> bool: - """Return whether completed policy selected callback-based descriptor access.""" + def _uses_module_descriptor_backend(plan: ModuleVariablePlan) -> bool: + """Return whether a handle reaches its descriptor through a consumer. + + A module array hands a plan-selected descriptor projection to a + consumer rather than filling a record supplied from C. + """ handle = plan.native_array_handle return bool( handle is not None - and handle.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR + and handle.descriptor_interop + in { + NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR, + NativeArrayDescriptorInterop.POINTER_C_DESCRIPTOR, + } ) - def _module_allocatable_descriptor_callback_operation( + def _module_descriptor_callback_operation( self, plan: ModuleVariablePlan, operation: NativeArrayOperation, ) -> FortranFunction: - """Pass the current allocatable descriptor to a C callback without copying.""" + """Run a C callback on the current module-array descriptor projection.""" name = self._module_native_array_operation_name(plan, operation) interface_name = self._module_descriptor_callback_interface_name(plan) + callback = FortranCall( + "callback", + (CodeExpression(self._native_variable_name(plan)), CodeExpression("context")), + ) + handle = plan.native_array_handle + if handle is None: + raise ValueError(f"Module handle {plan.owner_path!r} has no descriptor policy") + invoke = ( + ( + FortranIf( + CodeExpression(self._module_native_array_presence_expression(plan)), + body=(callback,), + ), + ) + if handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER + else (callback,) + ) return FortranFunction( name=name, parameters=( @@ -2707,10 +2632,7 @@ def _module_allocatable_descriptor_callback_operation( "c_f_procpointer", (CodeExpression("callback_address"), CodeExpression("callback")), ), - FortranCall( - "callback", - (CodeExpression(self._native_variable_name(plan)), CodeExpression("context")), - ), + *invoke, ), is_subroutine=True, ) @@ -2810,7 +2732,7 @@ def _module_native_array_element_type(self, plan: ModuleVariablePlan) -> str: if plan.datatype_family is DatatypeFamily.STRING: length = ":" if plan.character_length is None else str(plan.character_length) return f"character(kind=c_char, len={length})" - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).fortran_spelling + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_fortran_type def _module_pointer_dummy_element_type(self, plan: ModuleVariablePlan) -> str: """Return the element type of one module pointer dummy. @@ -2821,28 +2743,28 @@ def _module_pointer_dummy_element_type(self, plan: ModuleVariablePlan) -> str: """ if plan.datatype_family is DatatypeFamily.STRING: return "character(kind=c_char, len=:)" - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).fortran_spelling + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_fortran_type def _module_descriptor_consumer_value_declaration( self, plan: ModuleVariablePlan, rank: int, ) -> tuple[str, tuple[str, ...]]: - """Return the type and attributes of one descriptor-consumer value dummy. - - A ``bind(C)`` allocatable character dummy has to declare deferred - length, while argument association requires the actual to be deferred - exactly when the dummy is. A module array that declares its own width - satisfies neither together, so it travels as an assumed-length - assumed-shape dummy whose descriptor still carries the element length. - The runtime never reaches this operation while the array is - unallocated: ``AllocatableArray.to_numpy`` and ``shape`` both return - early on ``allocated``. + """Return the plan-selected descriptor callback dummy. + + Allocatable and pointer descriptors preserve their entity semantics. + An ``other`` descriptor is the ordinary assumed-shape projection used + for fixed-width character storage; the callback is entered only while + that storage is present. """ dimension = self._array_dimension_attribute(rank) - if plan.datatype_family is DatatypeFamily.STRING and plan.character_length is not None: - return "character(kind=c_char, len=*)", (dimension, "intent(in)") - return self._module_native_array_element_type(plan), ("allocatable", dimension, "intent(in)") + handle = plan.native_array_handle + if handle is None: + raise ValueError(f"Module handle {plan.owner_path!r} has no descriptor policy") + if handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER: + return "character(kind=c_char, len=*)", (dimension, "intent(inout)") + attribute = handle.descriptor_attribute.value + return self._module_native_array_element_type(plan), (attribute, dimension, "intent(inout)") def _module_native_array_operation_name(self, plan: ModuleVariablePlan, operation) -> str: """Return one planner-owned module native-array operation symbol.""" @@ -2996,7 +2918,7 @@ def _lower_module_getter_borrowed_array_view( self, plan: ModuleVariablePlan, ) -> tuple[FortranFunction, ...]: - """Expose one addressable fixed module array through pointer and extents.""" + """Expose one fixed module array through its base pointer and extents.""" array = plan.array if array is None or array.rank is None: raise ValueError(f"Module array view {plan.owner_path!r} has no fixed rank") @@ -3007,6 +2929,7 @@ def _lower_module_getter_borrowed_array_view( # the Fortran variable, not to anything the binding can restate. width = ("itemsize",) if plan.datatype_family is DatatypeFamily.STRING else () extents = tuple(f"extent_{axis}" for axis in range(array.rank)) + address = self._module_array_address(plan, native) return ( FortranFunction( name=name, @@ -3026,11 +2949,65 @@ def _lower_module_getter_borrowed_array_view( ) for axis, extent in enumerate(extents) ), - FortranAssignment("result", CodeExpression(f"c_loc({native})")), + FortranAssignment("result", CodeExpression(address)), ), ), ) + @staticmethod + def _module_array_address(plan: ModuleVariablePlan, native: str) -> str: + """Return the address expression selected by the completed mechanism.""" + mechanism = plan.array_address + if mechanism is ModuleArrayAddressMechanism.TARGET_ADDRESS: + return f"c_loc({native})" + if mechanism is ModuleArrayAddressMechanism.CAPTURED_ADDRESS: + return f"{_MODULE_ARRAY_CAPTURE_NAME}({native})" + raise ValueError(f"Module array view {plan.owner_path!r} has no completed address mechanism: {mechanism!r}") + + def _requires_address_capture(self, plan: ModulePlan) -> bool: + """Report whether any borrowed view must take its address on the C side. + + Both cases name their storage directly rather than reaching it through a + pointer, so neither has a Fortran route to its own address: a module + array whose declaration withheld ``target``, and an array member of a + plain module object, which is likewise not a target. + """ + return any( + variable.array_address is ModuleArrayAddressMechanism.CAPTURED_ADDRESS for variable in self._variables(plan) + ) or any( + member.field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR + for variable in self._derived_member_proxy_variables(plan) + for member in variable.derived.member_paths + ) + + def _module_array_capture_interfaces(self, plan: ModulePlan) -> tuple[FortranInterface, ...]: + """Declare the C capture helper wherever a borrowed view needs it. + + ``c_loc`` requires the variable it names to be a target, so a module + array whose declaration withheld the attribute has no Fortran route to + its own address. Handing the whole array to a `bind(C)` procedure does + have one: an assumed-type assumed-size dummy is passed as the bare base + address, so C receives where the module variable lives and hands it + straight back. Nothing here claims a target or forms a Fortran pointer. + """ + if not self._requires_address_capture(plan): + return () + return ( + FortranInterface( + ( + FortranInterfaceProcedure( + name=_MODULE_ARRAY_CAPTURE_NAME, + imports=("c_ptr",), + parameters=(FortranParameter("base", "type(*)", ("dimension(*)",)),), + result_name="address", + result_type="type(c_ptr)", + bind_name=_MODULE_ARRAY_CAPTURE_NAME, + bind_c=True, + ), + ) + ), + ) + def _lower_module_getter_nullable_snapshot( self, plan: ModuleVariablePlan, @@ -3375,7 +3352,7 @@ def _native_array_argument_element_type(self, plan: ArgumentTransferPlan) -> str """Return one numeric or deferred-character descriptor dummy type.""" if plan.datatype_family is DatatypeFamily.STRING: return "character(kind=c_char, len=:)" - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).fortran_spelling + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_fortran_type def _lower_argument_required(self, plan: ArgumentTransferPlan) -> tuple[FortranParameter, ...]: """Dispatch one required entrypoint parameter from its completed ABI shape.""" @@ -3434,6 +3411,58 @@ def _lower_argument_string_value( FortranParameter(f"{name}_length", "integer(c_int64_t)", ("value",)), ) + @staticmethod + def _array_crosses_as_descriptor(plan: ArgumentTransferPlan) -> bool: + """Report whether completed policy hands this array over as a descriptor.""" + array = plan.array + return ( + plan.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER + and array is not None + and array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + ) + + def _lower_argument_array_descriptor( + self, + plan: ArgumentTransferPlan, + ) -> tuple[FortranParameter, ...]: + """Receive one ordinary array as the descriptor its direct route uses. + + An assumed-shape dummy is interoperable, so C passes a ``CFI_cdesc_t *`` + and the extents and strides arrive inside it. Nothing is reconstructed + here: the dummy is the array, with the bounds and directions the caller + described, and it is handed to the native procedure as it stands. + """ + return ( + self._array_descriptor_parameter( + plan, + plan.entrypoint.parameter_name, + optional=plan.entrypoint.optional_mode is not OptionalMode.REQUIRED, + ), + ) + + def _array_descriptor_parameter( + self, + plan: ArgumentTransferPlan, + name: str, + *, + optional: bool, + ) -> FortranParameter: + """Declare one interoperable ordinary-array descriptor dummy.""" + array = plan.array + if array is None: + raise ValueError(f"Descriptor array argument {plan.owner_path!r} has no handoff") + element_type = self._array_element_fortran_type(plan) + if plan.datatype_family is DatatypeFamily.STRING and array.itemsize is None: + # The width travels in the descriptor, and a bind(C) character dummy + # may not name a variable for it, so it is assumed here. + element_type = "character(kind=c_char, len=*)" + attributes = ["dimension(..)" if array.rank is None else self._array_dimension_attribute(array.rank)] + if optional: + # C omits it by passing no descriptor, which is what optional means + # for an interoperable dummy. + attributes.append("optional") + return FortranParameter(name, element_type, tuple(attributes)) + # Ordinary-array argument lowering. def _lower_argument_array_buffer( self, @@ -3443,6 +3472,8 @@ def _lower_argument_array_buffer( array = plan.array if array is None: raise ValueError(f"Array argument {plan.owner_path!r} has no handoff spec") + if self._array_crosses_as_descriptor(plan): + return self._lower_argument_array_descriptor(plan) name = plan.entrypoint.parameter_name return ( FortranParameter(f"bound_{name}", "type(c_ptr)", ("value",)), @@ -3510,8 +3541,14 @@ def _function_body( and argument.entrypoint.optional_mode in {OptionalMode.NULLABLE_VALUE, OptionalMode.DESCRIPTOR} ) if derived_optional: - procedures = self._derived_optional_dispatch_procedures(plan, derived_optional, result_name) - return (FortranCall(self._derived_optional_step_name(0), ()),), procedures + forwarded = self._contained_optional_descriptor_arguments(plan) + procedures = self._derived_optional_dispatch_procedures( + plan, + derived_optional, + result_name, + forwarded, + ) + return (self._contained_optional_descriptor_call_tree(forwarded, 0, ()),), procedures return self._ordinary_function_body(plan, result_name), () def _ordinary_function_body( @@ -3564,15 +3601,19 @@ def _polymorphic_arguments(plan: FunctionPlan) -> tuple[ArgumentTransferPlan, .. @staticmethod def _assumed_rank_arguments(plan: FunctionPlan) -> tuple[ArgumentTransferPlan, ...]: - """Return assumed-rank arrays in original-Fortran call order.""" + """Return raw-address runtime-rank arrays requiring bridge dispatch.""" return tuple( argument for argument in sorted(plan.arguments, key=lambda item: item.projected_call_slot.native_position) - if argument.array is not None and argument.array.rank is None + if argument.array is not None + and argument.array.rank is None + and argument.array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS ) @staticmethod - def _non_derived_optional_arguments(plan: FunctionPlan) -> tuple[ArgumentTransferPlan, ...]: + def _non_derived_optional_arguments( + plan: FunctionPlan, + ) -> tuple[ArgumentTransferPlan, ...]: """Return optional arguments handled by the ordinary presence tree.""" return tuple( argument @@ -3581,6 +3622,44 @@ def _non_derived_optional_arguments(plan: FunctionPlan) -> tuple[ArgumentTransfe and argument.derived_call is None ) + def _contained_optional_descriptor_arguments( + self, + plan: FunctionPlan, + ) -> tuple[ArgumentTransferPlan, ...]: + """Return descriptor optionals that must not be host-associated on ifx.""" + return tuple( + argument + for argument in sorted(plan.arguments, key=lambda item: item.projected_call_slot.native_position) + if argument.derived_call is None + and argument.entrypoint.optional_mode in {OptionalMode.NULLABLE_VALUE, OptionalMode.DESCRIPTOR} + and argument.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER + and self._array_crosses_as_descriptor(argument) + ) + + def _contained_optional_descriptor_call_tree( + self, + arguments: tuple[ArgumentTransferPlan, ...], + index: int, + passed: tuple[CodeExpression, ...], + ) -> FortranCall | FortranIf: + """Enter the contained chain without forwarding an absent descriptor.""" + if index == len(arguments): + return FortranCall(self._derived_optional_step_name(0), passed) + argument = arguments[index] + local_name = self._forwarded_optional_descriptor_parameter_name(argument) + actual_name = argument.entrypoint.parameter_name + return FortranIf( + CodeExpression(f"present({actual_name})"), + body=( + self._contained_optional_descriptor_call_tree( + arguments, + index + 1, + (*passed, CodeExpression(f"{local_name}={actual_name}")), + ), + ), + else_body=(self._contained_optional_descriptor_call_tree(arguments, index + 1, passed),), + ) + def _polymorphic_call_tree( self, plan: FunctionPlan, @@ -3627,13 +3706,21 @@ def _derived_optional_dispatch_procedures( plan: FunctionPlan, optional: tuple[ArgumentTransferPlan, ...], result_name: str | None, + forwarded: tuple[ArgumentTransferPlan, ...], ) -> tuple[FortranFunction, ...]: """Propagate N optional derived dummies with O(N) adapter procedures.""" procedures = [] + forwarded_parameters = tuple(self._forwarded_optional_descriptor_parameter(item) for item in forwarded) + forwarded_passed = tuple( + CodeExpression(self._forwarded_optional_descriptor_parameter_name(item)) for item in forwarded + ) for index, argument in enumerate(optional): carried = optional[:index] - parameters = tuple(self._derived_optional_parameter(item) for item in carried) - passed = tuple(CodeExpression(self._derived_optional_parameter_name(item)) for item in carried) + parameters = (*forwarded_parameters, *(self._derived_optional_parameter(item) for item in carried)) + passed = ( + *forwarded_passed, + *(CodeExpression(self._derived_optional_parameter_name(item)) for item in carried), + ) expression = CodeExpression(self._native_argument_expression(argument)) procedures.append( FortranFunction( @@ -3654,12 +3741,18 @@ def _derived_optional_dispatch_procedures( is_subroutine=True, ) ) - replacements = {argument.owner_path: self._derived_optional_parameter_name(argument) for argument in optional} + replacements = { + **{argument.owner_path: self._derived_optional_parameter_name(argument) for argument in optional}, + **{ + argument.owner_path: self._forwarded_optional_descriptor_parameter_name(argument) + for argument in forwarded + }, + } present = frozenset(argument.owner_path for argument in optional) procedures.append( FortranFunction( name=self._derived_optional_step_name(len(optional)), - parameters=tuple(self._derived_optional_parameter(item) for item in optional), + parameters=(*forwarded_parameters, *(self._derived_optional_parameter(item) for item in optional)), body=self._ordinary_function_body( plan, result_name, @@ -3671,6 +3764,22 @@ def _derived_optional_dispatch_procedures( ) return tuple(procedures) + def _forwarded_optional_descriptor_parameter( + self, + argument: ArgumentTransferPlan, + ) -> FortranParameter: + """Declare one optional descriptor passed into every contained step.""" + return self._array_descriptor_parameter( + argument, + self._forwarded_optional_descriptor_parameter_name(argument), + optional=True, + ) + + @staticmethod + def _forwarded_optional_descriptor_parameter_name(argument: ArgumentTransferPlan) -> str: + """Name an optional descriptor local to the contained dispatch chain.""" + return f"prik_optional_{argument.entrypoint.parameter_name}" + def _derived_optional_parameter(self, argument: ArgumentTransferPlan) -> FortranParameter: """Mirror the completed native dummy category and add OPTIONAL.""" return self._derived_native_parameter( @@ -3730,7 +3839,11 @@ def _optional_call_tree( argument = optional[index] present_roles = present | {argument.owner_path} return FortranIf( - condition=CodeExpression(self._presence_condition(argument)), + condition=CodeExpression( + f"present({replacements[argument.owner_path]})" + if self._array_crosses_as_descriptor(argument) and argument.owner_path in replacements + else self._presence_condition(argument) + ), body=( *self._present_preparation(argument), self._optional_call_tree(plan, optional, index + 1, present_roles, result_name, replacements), @@ -4017,6 +4130,10 @@ def _presence_condition(self, plan: ArgumentTransferPlan) -> str: name = plan.entrypoint.parameter_name if plan.derived_call is not None: return f"bound_{name}_access /= 0_c_int" + if self._array_crosses_as_descriptor(plan): + # The dummy is the array itself, and C omits it by passing no + # descriptor at all, so Fortran's own inquiry is the condition. + return f"present({name})" suffix = "_present" if plan.entrypoint.optional_mode is OptionalMode.DESCRIPTOR else "" return f"c_associated(bound_{name}{suffix})" @@ -4073,6 +4190,9 @@ def _prepare_present_associated_view( if plan.entrypoint.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR: return () if plan.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER: + if self._array_crosses_as_descriptor(plan): + # The dummy already is the array the caller described. + return () return self._array_pointer_initializer_nodes(plan) if plan.entrypoint.optional_mode is OptionalMode.NULLABLE_VALUE: return ( @@ -4334,6 +4454,9 @@ def _array_declarations(self, plan: FunctionPlan) -> tuple[FortranDeclaration, . for argument in plan.arguments: if argument.entrypoint.handoff_mode is not ArgumentHandoffMode.ARRAY_BUFFER: continue + if self._array_crosses_as_descriptor(argument): + # The dummy is the view; there is no address to make one from. + continue array = argument.array if array is None: raise ValueError(f"Array argument {argument.owner_path!r} is missing its handoff") @@ -4373,7 +4496,7 @@ def _array_declarations(self, plan: FunctionPlan) -> tuple[FortranDeclaration, . declarations.append( FortranDeclaration( self._logical_array_byte_pointer_name(argument), - "integer(c_int8_t)", + self._logical_array_integer_type(argument.semantic_type_name), ("pointer", "dimension(:)"), ) ) @@ -4499,10 +4622,49 @@ def _logical_array_writeback_for_rank( ), FortranAssignment( byte_pointer, - CodeExpression(f"iand({byte_pointer}, 1_c_int8_t)"), + CodeExpression(self._logical_array_canonical_expression(argument.semantic_type_name, byte_pointer)), ), ) + @staticmethod + def _logical_array_integer_type(semantic_type_name: str) -> str: + """Return the integer type covering one Boolean element's own width. + + The mask reinterprets the caller's buffer, so it has to step by the + element width rather than by bytes: a `logical(4)` array is four-byte + integers, not four times as many one-byte ones. + """ + return { + "Bool": "integer(c_int8_t)", + "Bool8": "integer(c_int8_t)", + "Bool16": "integer(c_int16_t)", + "Bool32": "integer(c_int32_t)", + "Bool64": "integer(c_int64_t)", + }[semantic_type_name] + + @staticmethod + def _logical_array_kind_suffix(semantic_type_name: str) -> str: + """Return the integer kind suffix matching one Boolean element's width.""" + return { + "Bool": "c_int8_t", + "Bool8": "c_int8_t", + "Bool16": "c_int16_t", + "Bool32": "c_int32_t", + "Bool64": "c_int64_t", + }[semantic_type_name] + + def _logical_array_canonical_expression(self, semantic_type_name: str, target: str) -> str: + """Return the expression reducing Boolean storage to zero and one. + + The rule is C's: any non-zero value is true, which is what converting to + ``_Bool`` produces and what NumPy, Python and C all read back. It is not + a low-bit test -- that would call ``2`` false, disagreeing with every one + of them -- and it maps both representations compilers emit, ``1`` and + ``-1``, onto the single value the interoperable type is defined to hold. + """ + kind = self._logical_array_kind_suffix(semantic_type_name) + return f"merge(1_{kind}, 0_{kind}, {target} /= 0_{kind})" + @staticmethod def _logical_array_byte_pointer_name(argument: ArgumentTransferPlan) -> str: """Return the bridge-local byte-pointer name for one logical-array rank conversion.""" @@ -4518,6 +4680,8 @@ def _array_initializers(self, plan: FunctionPlan) -> tuple[FortranCall | Fortran continue if argument.array is not None and argument.array.rank is None: continue + if self._array_crosses_as_descriptor(argument): + continue initializers.extend(self._array_pointer_initializer_nodes(argument)) return tuple(initializers) @@ -4595,24 +4759,7 @@ def _array_pointer_initializer_nodes( argument: ArgumentTransferPlan, ) -> tuple[FortranCall | FortranIf, ...]: """Associate base storage and select the planned dense or strided view.""" - association = self._array_pointer_initializer(argument) - array = argument.array - if array is None or array.dense_actual_role is None: - return (association,) - name = argument.entrypoint.parameter_name - return ( - association, - FortranIf( - CodeExpression(f"{name}_dense_actual /= 0_c_int"), - body=(FortranPointerAssignment(name, CodeExpression(f"{name}_base")),), - else_body=( - FortranPointerAssignment( - name, - CodeExpression(self._strided_array_section_expression(argument)), - ), - ), - ), - ) + return (self._array_pointer_initializer(argument),) def _assumed_rank_array_declarations( self, @@ -4668,22 +4815,12 @@ def _array_boundary_argument_expression(self, argument: ArgumentTransferPlan) -> name = argument.entrypoint.parameter_name if array.rank is None: return name - pointer_name = self._array_pointer_name(argument) - if array.contiguous is not False: - return pointer_name - if array.dense_actual_role is not None: + if self._array_crosses_as_descriptor(argument): + # The dummy carries the caller's own bounds and directions. return name - return self._strided_array_section_expression(argument) - - def _strided_array_section_expression(self, argument: ArgumentTransferPlan) -> str: - """Render one positive-stride section from completed layout roles.""" - array = argument.array - if array is None or array.rank is None: - raise ValueError(f"Strided array argument {argument.owner_path!r} requires a concrete rank") - name = argument.entrypoint.parameter_name - pointer_name = self._array_pointer_name(argument) - slices = (f"1:{name}_upper_bound_{axis} + 1:{name}_stride_{axis}" for axis in range(array.rank)) - return f"{pointer_name}({', '.join(slices)})" + # Every other array reaches its dummy as an address, which the + # declaration already says how to read. + return self._array_pointer_name(argument) def _array_element_fortran_type(self, argument: ArgumentTransferPlan) -> str: """Return the completed primitive or fixed-width character element type.""" @@ -4698,7 +4835,7 @@ def _array_element_fortran_type(self, argument: ArgumentTransferPlan) -> str: if array.itemsize <= 0: raise ValueError(f"Character array {argument.owner_path!r} has a non-positive itemsize") return f"character(kind=c_char, len={array.itemsize})" - return PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).fortran_spelling + return PrimitiveScalarTypeRegistry.type_for(argument.semantic_type_name).array_fortran_type def _array_dimension_attribute(self, rank: int) -> str: """Spell one explicit-rank deferred-shape pointer attribute.""" @@ -5471,7 +5608,7 @@ def _direct_array_result_declarations( FortranDeclaration("result_copy", element_type, ("pointer",)), ) copy_type = "character(kind=c_char)" if result.datatype_family is DatatypeFamily.STRING else element_type - if "bridge" in result.array.extent_evaluation: + if "bridge" in result.array.extent_evaluation or self._array_result_depends_on_descriptor(plan, result): return ( FortranDeclaration( "result_value", @@ -5495,8 +5632,11 @@ def _direct_array_result_initializers( result is None or result.object_kind is not ObjectKind.NUMPY_ARRAY or result.array is None - or "bridge" not in result.array.extent_evaluation or self._is_scalar_storage_array(result.array) + or ( + "bridge" not in result.array.extent_evaluation + and not self._array_result_depends_on_descriptor(plan, result) + ) ): return () shape = list(self._array_result_shape(plan, result)) @@ -5505,6 +5645,28 @@ def _direct_array_result_initializers( shape[axis] = self._declaration_extent_result_name(result, axis) return (FortranAllocate(f"result_value({', '.join(shape)})"),) + def _array_result_depends_on_descriptor( + self, + plan: FunctionPlan, + result: ResultPlan, + ) -> bool: + """Use portable allocatable storage for descriptor-derived result extents. + + ifx can leave an automatic local undefined when it is assigned an + array-valued function result and its bounds refer to an assumed-shape + dummy. Allocatable call-local storage has the same ownership and copy + semantics without relying on that compiler path. + """ + descriptor_extent_roles = { + role + for argument in plan.arguments + if self._array_crosses_as_descriptor(argument) + for role in argument.array.extent_roles + } + return any( + role in descriptor_extent_roles for axis_roles in result.array.extent_reference_roles for role in axis_roles + ) + def _direct_result_finalizers( self, plan: FunctionPlan, @@ -5563,7 +5725,7 @@ def _direct_scalar_result_finalizers( return ( FortranAssignment( "result", - CodeExpression("iand(transfer(c_result, 0_c_int8_t), 1_c_int8_t)"), + CodeExpression("merge(1_c_int8_t, 0_c_int8_t, transfer(c_result, 0_c_int8_t) /= 0_c_int8_t)"), ), ) case DirectResultABI.NATIVE_SCALAR: @@ -5930,7 +6092,9 @@ def _scalar_descriptor_copy_nodes( copy_body = ( FortranAssignment( name, - CodeExpression(f"c_malloc(max(1_c_size_t, c_sizeof({value_name})))"), + CodeExpression( + f"c_malloc(max(1_c_size_t, storage_size({value_name}, kind=c_size_t) / 8_c_size_t))" + ), ), FortranIf( CodeExpression(f"c_associated({name})"), @@ -6179,7 +6343,7 @@ def _array_result_element_type( return f"character(kind=c_char, len={itemsize})" if plan.semantic_type_name is None: raise ValueError(f"Array result {plan.owner_path!r} has no element type") - return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).fortran_spelling + return PrimitiveScalarTypeRegistry.type_for(plan.semantic_type_name).array_fortran_type def _array_result_itemsize( self, @@ -6820,19 +6984,19 @@ def _native_handle_field_procedures(self, owner, field: DerivedFieldPlan) -> tup handle = field.native_array_handle if handle is None: raise ValueError(f"Native handle field {field.owner_path!r} has no operation plan") - procedures = [] - for operation in handle.operations: - if operation in { - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, - NativeArrayOperation.TO_NUMPY, - NativeArrayOperation.ARRAY_ACTUAL, - }: - continue - procedures.append(self._native_handle_field_procedure(owner, field, operation)) - return tuple(procedures) + # The descriptor entry point is what the binding runs every inquiry + # through, so it is emitted for the handle itself; the rest are the + # mutations that must reach the field. + planned = [NativeArrayOperation.DESCRIPTOR] if handle.descriptor_inquiries else [] + planned.extend( + operation + for operation in handle.operations + # The descriptor entry point is already planned, and a NumPy view is + # a Python object, which only the binding can build. + if operation not in {NativeArrayOperation.DESCRIPTOR, NativeArrayOperation.TO_NUMPY} + and (not handle.descriptor_inquiries or operation not in _DESCRIPTOR_ANSWERED_OPERATIONS) + ) + return tuple(self._native_handle_field_procedure(owner, field, operation) for operation in planned) def _native_handle_field_procedure( self, @@ -6910,7 +7074,7 @@ def _native_handle_field_length_procedure(self, owner, field) -> FortranFunction ) def _native_handle_field_shape_procedure(self, owner, field) -> FortranFunction: - """Build the per-axis shape inquiry for one native-array-handle field.""" + """Report field presence and write its current per-axis extents.""" handle = field.native_array_handle if handle is None or handle.array.rank is None: raise ValueError(f"Native handle field {field.owner_path!r} has no shape rank") @@ -6931,19 +7095,46 @@ def _native_handle_field_shape_procedure(self, owner, field) -> FortranFunction: return FortranFunction( name=name, parameters=(*self._native_handle_field_owner_parameters(owner), *extents), + result_name="result", + result_type="logical(c_bool)", bind_name=name, declarations=self._native_handle_field_owner_declarations(owner), body=( *self._native_handle_field_owner_body(owner), + FortranAssignment("result", CodeExpression(presence)), FortranIf(CodeExpression(presence), body=present, else_body=absent), ), - is_subroutine=True, ) def _native_handle_field_descriptor_procedure(self, owner, field) -> FortranFunction: """Build the descriptor-export procedure for one native-array-handle field.""" name = self._native_handle_field_bridge_name(owner, field, NativeArrayOperation.DESCRIPTOR) interface = self._native_handle_field_callback_interface_name(owner, field) + callback = FortranCall( + "callback", + ( + CodeExpression(self._native_handle_field_expression(owner, field)), + CodeExpression("context"), + ), + ) + handle = field.native_array_handle + if handle is None: + raise ValueError(f"Native handle field {field.owner_path!r} has no descriptor policy") + invoke = ( + ( + FortranIf( + CodeExpression( + self._native_handle_field_presence( + field, + self._native_handle_field_expression(owner, field), + ) + ), + body=(callback,), + ), + ) + if handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER + else (callback,) + ) return FortranFunction( name=name, parameters=( @@ -6962,13 +7153,7 @@ def _native_handle_field_descriptor_procedure(self, owner, field) -> FortranFunc "c_f_procpointer", (CodeExpression("callback_address"), CodeExpression("callback")), ), - FortranCall( - "callback", - ( - CodeExpression(self._native_handle_field_expression(owner, field)), - CodeExpression("context"), - ), - ), + *invoke, ), is_subroutine=True, ) @@ -7010,7 +7195,7 @@ def _native_handle_field_associate_procedure(self, owner, field) -> FortranFunct element_type = ( "character(kind=c_char, len=:)" if field.string_element - else PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).fortran_spelling + else PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).array_fortran_type ) expression = self._native_handle_field_expression(owner, field) name = self._native_handle_field_bridge_name(owner, field, NativeArrayOperation.ASSOCIATE) @@ -7122,33 +7307,53 @@ def _direct_ordinary_array_field_getter( derived: DerivedTypePlan, field: DerivedFieldPlan, ) -> FortranFunction: - """Pass one fixed field through a standard descriptor callback.""" + """Expose one fixed field through its base pointer and extents. + + The owner arrives as an address and is reached through a Fortran + pointer, so its components are subobjects of a pointer target and + ``c_loc`` can name them directly however the field itself was declared. + """ name = self._derived_field_bridge_name(derived, field, "get") - interface = self._derived_field_callback_interface_name(derived, field) + member = f"owner%{field.native_name}" return FortranFunction( name=name, parameters=( FortranParameter("owner_address", "type(c_ptr)", ("value",)), - FortranParameter("callback_address", "type(c_funptr)", ("value",)), - FortranParameter("context", "type(c_ptr)", ("value",)), + *self._ordinary_array_field_extent_parameters(field), ), + result_name="result", + result_type="type(c_ptr)", bind_name=name, - declarations=( - self._derived_owner_declaration(derived), - FortranDeclaration("callback", f"procedure({interface})", ("pointer",)), - ), + declarations=(self._derived_owner_declaration(derived),), body=( self._derived_owner_association(), - FortranCall( - "c_f_procpointer", - (CodeExpression("callback_address"), CodeExpression("callback")), - ), - FortranCall( - "callback", - (CodeExpression(f"owner%{field.native_name}"), CodeExpression("context")), - ), + *self._ordinary_array_field_extent_assignments(field, member), + FortranAssignment("result", CodeExpression(f"c_loc({member})")), ), - is_subroutine=True, + ) + + @staticmethod + def _ordinary_array_field_extent_parameters(field: DerivedFieldPlan) -> tuple[FortranParameter, ...]: + """Return the reported extent outputs for one fixed array field.""" + array = field.array + if array is None or array.rank is None: + raise ValueError(f"Ordinary array field {field.owner_path!r} has no fixed rank") + return tuple( + FortranParameter(f"extent_{axis}", "integer(c_int64_t)", ("intent(out)",)) for axis in range(array.rank) + ) + + @staticmethod + def _ordinary_array_field_extent_assignments(field: DerivedFieldPlan, member: str) -> tuple[FortranAssignment, ...]: + """Report each axis from the native field rather than restating its declaration.""" + array = field.array + if array is None or array.rank is None: + raise ValueError(f"Ordinary array field {field.owner_path!r} has no fixed rank") + return tuple( + FortranAssignment( + f"extent_{axis}", + CodeExpression(f"int(size({member}, {axis + 1}), c_int64_t)"), + ) + for axis in range(array.rank) ) def _direct_ordinary_array_field_setter( @@ -7184,31 +7389,25 @@ def _module_ordinary_array_member_getter( variable: ModuleVariablePlan, member: DerivedMemberPathPlan, ) -> FortranFunction: - """Pass a fixed module member through a standard descriptor callback.""" + """Expose one fixed module member through its base pointer and extents. + + A plain module object is named directly rather than reached through a + pointer, so nothing here is a target and ``c_loc`` cannot name the + member. The address is taken on the C side instead, exactly as a + non-addressable module array's is. + """ name = self._module_member_bridge_name(variable, member, "get") - interface = self._module_member_callback_interface_name(variable, member) + expression = self._module_member_expression(variable, member) return FortranFunction( name=name, - parameters=( - FortranParameter("callback_address", "type(c_funptr)", ("value",)), - FortranParameter("context", "type(c_ptr)", ("value",)), - ), + parameters=self._ordinary_array_field_extent_parameters(member.field), + result_name="result", + result_type="type(c_ptr)", bind_name=name, - declarations=(FortranDeclaration("callback", f"procedure({interface})", ("pointer",)),), body=( - FortranCall( - "c_f_procpointer", - (CodeExpression("callback_address"), CodeExpression("callback")), - ), - FortranCall( - "callback", - ( - CodeExpression(self._module_member_expression(variable, member)), - CodeExpression("context"), - ), - ), + *self._ordinary_array_field_extent_assignments(member.field, expression), + FortranAssignment("result", CodeExpression(f"{_MODULE_ARRAY_CAPTURE_NAME}({expression})")), ), - is_subroutine=True, ) def _module_ordinary_array_member_setter( @@ -7465,14 +7664,6 @@ def _pointer_holder_field_bridge_name( f"{derived.owner_path}.{field.name}", f"field:pointer:{action}" ).symbol_name - def _derived_field_callback_interface_name( - self, - derived: DerivedTypePlan, - field: DerivedFieldPlan, - ) -> str: - """Return the consumer-interface name associated with one direct derived field.""" - return f"prik_field_{self._derived_field_symbol(derived, field)}_consumer" - def _derived_handle_bridge_name( self, derived: DerivedTypePlan, @@ -7491,7 +7682,9 @@ def _derived_handle_callback_interface_name( field: DerivedFieldPlan, ) -> str: """Return the consumer-interface name associated with one direct native-array-handle field.""" - return f"prik_field_handle_{self._derived_field_symbol(derived, field)}_consumer" + owner_path = f"{derived.owner_path}.{field.name}" + preferred = f"prik_field_handle_{self._derived_field_symbol(derived, field)}_consumer" + return NativeSymbolNames.bounded(f"{owner_path}::field:direct:consumer", preferred) @staticmethod def _module_member_symbol(variable: ModuleVariablePlan, member: DerivedMemberPathPlan) -> str: @@ -7509,14 +7702,6 @@ def _module_member_bridge_name( ".".join((variable.owner_path, *member.path)), f"field:module:{action}" ).symbol_name - def _module_member_callback_interface_name( - self, - variable: ModuleVariablePlan, - member: DerivedMemberPathPlan, - ) -> str: - """Return the consumer-interface name associated with one module member.""" - return f"prik_module_field_{self._module_member_symbol(variable, member)}_consumer" - def _module_member_handle_bridge_name( self, variable: ModuleVariablePlan, @@ -7535,7 +7720,9 @@ def _module_member_handle_callback_interface_name( member: DerivedMemberPathPlan, ) -> str: """Return the consumer-interface name for one module native-array-handle member.""" - return f"prik_module_field_handle_{self._module_member_symbol(variable, member)}_consumer" + owner_path = ".".join((variable.owner_path, *member.path)) + preferred = f"prik_module_field_handle_{self._module_member_symbol(variable, member)}_consumer" + return NativeSymbolNames.bounded(f"{owner_path}::field:module:consumer", preferred) def _derived_member_proxy_variables(self, plan: ModulePlan) -> tuple[ModuleVariablePlan, ...]: """Return derived module variables whose completed access mechanism is member proxying.""" @@ -8040,44 +8227,18 @@ def _module_descriptor_callback_interfaces(self, plan: ModulePlan) -> tuple[Fort procedures = tuple( self._module_descriptor_callback_interface(variable) for variable in self._variables(plan) - if self._uses_module_allocatable_descriptor(variable) + if self._uses_module_descriptor_backend(variable) ) return (FortranInterface(procedures),) if procedures else () def _derived_array_callback_interfaces(self, plan: ModulePlan) -> tuple[FortranInterface, ...]: """Declare standard-descriptor callbacks for live ordinary array fields.""" procedures = ( - *self._direct_ordinary_array_callback_interfaces(plan), - *self._module_ordinary_array_callback_interfaces(plan), *self._direct_handle_callback_interfaces(plan), *self._module_handle_callback_interfaces(plan), ) return (FortranInterface(procedures),) if procedures else () - def _direct_ordinary_array_callback_interfaces(self, plan: ModulePlan) -> tuple: - """Return callback interfaces required by direct ordinary-array field procedures.""" - return tuple( - self._ordinary_array_callback_interface( - field, - self._derived_field_callback_interface_name(derived, field), - ) - for derived in self._derived_types(plan) - for field in derived.fields - if field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR - ) - - def _module_ordinary_array_callback_interfaces(self, plan: ModulePlan) -> tuple: - """Return callback interfaces required by module ordinary-array member procedures.""" - return tuple( - self._ordinary_array_callback_interface( - member.field, - self._module_member_callback_interface_name(variable, member), - ) - for variable in self._derived_member_proxy_variables(plan) - for member in variable.derived.member_paths - if member.field.access is DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR - ) - def _direct_handle_callback_interfaces(self, plan: ModulePlan) -> tuple: """Return callback interfaces required by direct native-array-handle field procedures.""" return tuple( @@ -8088,6 +8249,8 @@ def _direct_handle_callback_interfaces(self, plan: ModulePlan) -> tuple: for derived in self._derived_types(plan) for field in derived.fields if field.access is DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE + and field.native_array_handle is not None + and field.native_array_handle.descriptor_inquiries ) def _module_handle_callback_interfaces(self, plan: ModulePlan) -> tuple: @@ -8100,31 +8263,8 @@ def _module_handle_callback_interfaces(self, plan: ModulePlan) -> tuple: for variable in self._derived_member_proxy_variables(plan) for member in variable.derived.member_paths if member.field.access is DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE - ) - - def _ordinary_array_callback_interface( - self, - field: DerivedFieldPlan, - name: str, - ) -> FortranInterfaceProcedure: - """Return one element- and rank-typed descriptor consumer interface.""" - array = field.array - if array is None or array.rank is None: - raise ValueError(f"Ordinary array field {field.owner_path!r} has no callback rank") - scalar = PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name) - return FortranInterfaceProcedure( - name=name, - imports=(self._iso_symbol(field.semantic_type_name), "c_ptr"), - parameters=( - FortranParameter( - "value", - scalar.fortran_spelling, - (self._array_dimension_attribute(array.rank), "intent(in)"), - ), - FortranParameter("context", "type(c_ptr)", ("value",)), - ), - is_subroutine=True, - bind_name=name, + and member.field.native_array_handle is not None + and member.field.native_array_handle.descriptor_inquiries ) def _native_handle_callback_interface( @@ -8136,12 +8276,20 @@ def _native_handle_callback_interface( handle = field.native_array_handle if handle is None or handle.array.rank is None: raise ValueError(f"Native handle field {field.owner_path!r} has no callback rank") - attribute = "allocatable" if handle.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE else "pointer" - element_type = ( - "character(kind=c_char, len=:)" - if field.string_element - else PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).fortran_spelling - ) + if handle.descriptor_attribute is NativeArrayDescriptorAttribute.OTHER: + element_type = "character(kind=c_char, len=*)" + attributes = (self._array_dimension_attribute(handle.array.rank), "intent(inout)") + else: + element_type = ( + "character(kind=c_char, len=:)" + if field.string_element + else PrimitiveScalarTypeRegistry.type_for(field.semantic_type_name).array_fortran_type + ) + attributes = ( + handle.descriptor_attribute.value, + self._array_dimension_attribute(handle.array.rank), + "intent(inout)", + ) imports = (self._iso_symbol(field.semantic_type_name), "c_ptr") return FortranInterfaceProcedure( name=name, @@ -8150,7 +8298,10 @@ def _native_handle_callback_interface( FortranParameter( "value", element_type, - (attribute, self._array_dimension_attribute(handle.array.rank), "intent(in)"), + # A true descriptor attribute carries allocation or + # association changes. An ordinary projection writes only + # through the storage already present. + attributes, ), FortranParameter("context", "type(c_ptr)", ("value",)), ), @@ -8179,7 +8330,8 @@ def _module_descriptor_callback_interface( def _module_descriptor_callback_interface_name(self, plan: ModuleVariablePlan) -> str: """Return one unique typed callback interface name.""" - return f"prik_{plan.symbol_name}_descriptor_consumer" + preferred = f"prik_{plan.symbol_name}_descriptor_consumer" + return NativeSymbolNames.bounded(f"{plan.owner_path}::module:descriptor:consumer", preferred) def _allocator_interfaces(self, plan: ModulePlan) -> tuple[FortranInterface, ...]: """Return the allocator interface required by detached bridge copies.""" @@ -8595,15 +8747,20 @@ def _array_shape_from_roles(self, array: ArrayHandoffPlan, plan: FunctionPlan) - for axis, expression in enumerate(array.shape) ) - @staticmethod - def _array_shape_role_names(plan: FunctionPlan) -> dict[str, str]: + def _array_shape_role_names(self, plan: FunctionPlan) -> dict[str, str]: """Map planned scalar, extent, and callable roles to bridge spellings.""" role_names = { argument.entrypoint.handoff_role: argument.entrypoint.parameter_name for argument in plan.arguments } role_names.update( { - role: f"{argument.entrypoint.parameter_name}_extent_{axis}" + role: ( + # A descriptor argument brought its extents with it, so + # Fortran asks the array rather than a parameter beside it. + f"size({argument.entrypoint.parameter_name}, {axis + 1})" + if self._array_crosses_as_descriptor(argument) + else f"{argument.entrypoint.parameter_name}_extent_{axis}" + ) for argument in plan.arguments if argument.array is not None for axis, role in enumerate(argument.array.extent_roles) @@ -8770,9 +8927,7 @@ def _extended_precision_iso_symbols(self, plan: ModulePlan) -> tuple[str, ...]: def _uses_c_function_pointer_symbols(self, plan: ModulePlan) -> bool: """Return whether completed module or field descriptor actions require C procedure-pointer support.""" - module_descriptors = any( - self._uses_module_allocatable_descriptor(variable) for variable in self._variables(plan) - ) + module_descriptors = any(self._uses_module_descriptor_backend(variable) for variable in self._variables(plan)) field_descriptors = any( field.access in { diff --git a/prik/codegen/nodes.py b/prik/codegen/nodes.py index 54fd96dc2..a41548d3d 100644 --- a/prik/codegen/nodes.py +++ b/prik/codegen/nodes.py @@ -31,6 +31,41 @@ class BackendScalarType(StageRecord): python_type_name: str | None = None python_module_result_kind: str | None = None cfi_type_spelling: str | None = None + # Array facts, where an element aliased in a NumPy buffer is described + # differently from the same type crossing as a scalar value. A Fortran + # logical is the only such type: a scalar converts to a Python bool, while + # an array is aliased and must report the width its elements really have. + # Every other type leaves these unset and reuses its scalar spellings. + array_numpy_type_macro: str | None = None + array_element_c_spelling: str | None = None + array_python_type_name: str | None = None + array_fortran_spelling: str | None = None + array_cfi_type_spelling: str | None = None + + @property + def array_numpy_type(self) -> str | None: + """Return the NumPy type macro describing one aliased array element.""" + return self.array_numpy_type_macro or self.numpy_type_macro + + @property + def array_c_spelling(self) -> str: + """Return the C type whose width one aliased array element occupies.""" + return self.array_element_c_spelling or self.c_spelling + + @property + def array_dtype_name(self) -> str | None: + """Return the NumPy dtype expression one aliased array reports.""" + return self.array_python_type_name or self.python_type_name + + @property + def array_fortran_type(self) -> str: + """Return the Fortran type declaring one aliased array's elements.""" + return self.array_fortran_spelling or self.fortran_spelling + + @property + def array_cfi_type(self) -> str | None: + """Return the C descriptor type code for one array element.""" + return self.array_cfi_type_spelling or self.cfi_type_spelling @dataclass diff --git a/prik/codegen/primitive_scalar_types.py b/prik/codegen/primitive_scalar_types.py index fb11ece46..85fafde52 100644 --- a/prik/codegen/primitive_scalar_types.py +++ b/prik/codegen/primitive_scalar_types.py @@ -142,11 +142,72 @@ def expression_for(cls, semantic_dtype: str | None) -> str: ) +# A Fortran logical array is aliased element for element, so its NumPy dtype has +# to occupy the same width. `logical(c_bool)` is one byte holding zero or one -- +# the compiler profiles request the flag that guarantees it -- which is exactly +# what `numpy.bool_` describes, so it is exposed as one. No wider kind can be: +# NumPy has no Boolean larger than a byte, and a one-byte dtype cannot describe +# four-byte elements, so those report the integer of matching width instead and +# are read back with `.astype(bool)`. +# Only an array element takes the declared width. A logical value crosses the C +# boundary as one byte whatever its Fortran kind, and the bridge converts it, +# so the scalar spelling stays `logical(c_bool)` for every name. +_BOOLEAN_FORTRAN_SPELLINGS = { + "Bool": "logical(c_bool)", + "Bool8": "logical(c_bool)", + "Bool16": "logical(2)", + "Bool32": "logical(4)", + "Bool64": "logical(8)", +} +_BOOLEAN_ARRAY_NUMPY_MACROS = { + "Bool": "NPY_BOOL", + "Bool8": "NPY_BOOL", + "Bool16": "NPY_INT16", + "Bool32": "NPY_INT32", + "Bool64": "NPY_INT64", +} +_BOOLEAN_ARRAY_C_SPELLINGS = { + "Bool": "bool", + "Bool8": "bool", + "Bool16": "int16_t", + "Bool32": "int32_t", + "Bool64": "int64_t", +} +_BOOLEAN_ARRAY_DTYPE_NAMES = { + "Bool": "numpy.bool_", + "Bool8": "numpy.bool_", + "Bool16": "numpy.int16", + "Bool32": "numpy.int32", + "Bool64": "numpy.int64", +} +_BOOLEAN_ARRAY_CFI_TYPES = { + "Bool": "CFI_type_Bool", + "Bool8": "CFI_type_Bool", + # No C-interoperable Boolean type is wider than C_BOOL. The portable + # descriptor spelling for the wider Fortran logical kinds is therefore + # CFI_type_other, with elem_len carrying their exact storage width. + "Bool16": "CFI_type_other", + "Bool32": "CFI_type_other", + "Bool64": "CFI_type_other", +} + + class PrimitiveScalarTypeRegistry: """Return first-lane scalar facts without coupling binding and bridge emitters.""" TYPES: ClassVar[dict[str, BackendScalarType]] = { - **{name: replace(_BOOL_BACKEND_TYPE, semantic_name=name) for name in BOOLEAN_SEMANTIC_TYPE_NAMES}, + **{ + name: replace( + _BOOL_BACKEND_TYPE, + semantic_name=name, + array_fortran_spelling=_BOOLEAN_FORTRAN_SPELLINGS[name], + array_numpy_type_macro=_BOOLEAN_ARRAY_NUMPY_MACROS[name], + array_element_c_spelling=_BOOLEAN_ARRAY_C_SPELLINGS[name], + array_python_type_name=_BOOLEAN_ARRAY_DTYPE_NAMES[name], + array_cfi_type_spelling=_BOOLEAN_ARRAY_CFI_TYPES[name], + ) + for name in BOOLEAN_SEMANTIC_TYPE_NAMES + }, "Int8": BackendScalarType( semantic_name="Int8", c_spelling="int8_t", diff --git a/prik/compiler/compiler_profiles.py b/prik/compiler/compiler_profiles.py index 58fdef46f..c38015af6 100644 --- a/prik/compiler/compiler_profiles.py +++ b/prik/compiler/compiler_profiles.py @@ -100,6 +100,7 @@ def _language( release_flags: tuple[str, ...], general_flags: tuple[str, ...], optional_general_flags: tuple[str, ...] = (), + logical_interop_flags: tuple[str, ...] = (), standard_flags: tuple[str, ...], module_output_flag: str | None = None, openmp: dict[str, tuple[str, ...]] | None = None, @@ -113,6 +114,7 @@ def _language( "release_flags": release_flags, "general_flags": general_flags, "optional_general_flags": optional_general_flags, + "logical_interop_flags": logical_interop_flags, "standard_flags": standard_flags, "mpi": {}, "openmp": openmp or {}, @@ -143,6 +145,12 @@ def _language( openmp={"flags": ("-fopenmp",), "libs": ("gomp",)}, openacc={"flags": ("-ta=multicore", "-Minfo=accel")}, ) +# A Fortran `logical` has no fixed representation, and several compilers default +# to one their own C compiler cannot read: `logical(c_bool)` is interoperable +# with `_Bool`, which holds zero or one, and Intel and PGI otherwise store all +# bits set for `.true.`. Each offers a flag selecting the interoperable form, so +# the profiles below request it wherever one is needed; gfortran, Cray and IBM XL +# already use it. See https://www.fortran90.org/src/gotchas.html for the survey. _GNU_FORTRAN = _language( "gfortran", "mpif90", @@ -182,6 +190,7 @@ def _language( debug_flags=("-check", "bounds", "-g", "-O0"), release_flags=("-O3", "-DNDEBUG"), general_flags=("-fPIC", "-fpp"), + logical_interop_flags=("-standard-semantics",), standard_flags=("-std=f2003",), module_output_flag="-module", openmp={"flags": ("-qopenmp", "-nostandard-realloc-lhs"), "libs": ("iomp5",)}, @@ -204,6 +213,7 @@ def _language( debug_flags=("-Mbounds", "-g", "-O0"), release_flags=("-O3", "-DNDEBUG"), general_flags=("-fPIC", "-cpp"), + logical_interop_flags=("-Munixlogical",), standard_flags=("-Mstandard",), module_output_flag="-module", openmp={"flags": ("-mp",)}, @@ -236,6 +246,7 @@ def _language( debug_flags=("-Mbounds", "-g", "-O0"), release_flags=("-O3", "-DNDEBUG"), general_flags=("-fPIC", "-cpp"), + logical_interop_flags=("-Munixlogical",), standard_flags=("-Mstandard",), module_output_flag="-module", openmp={"flags": ("-mp",)}, diff --git a/prik/compiler/compilers.py b/prik/compiler/compilers.py index 1ddf5b549..46c5bc5e4 100644 --- a/prik/compiler/compilers.py +++ b/prik/compiler/compilers.py @@ -60,12 +60,17 @@ def from_fortran_executable( debug: bool = False, execute_commands: bool = True, search_path: str | None = None, + standard_logicals: bool = True, ) -> Compiler: """Create a mixed toolchain whose final link uses one Fortran driver. When ``c_executable`` is omitted, the Fortran driver's matching C compiler is selected. An explicit C executable keeps C probing and C compilation on the same driver while Fortran still owns the link. + + ``standard_logicals`` requests the option that makes a Fortran + ``logical`` interoperable with C. Decline it only to match objects + already built without it, whose stored values C cannot read. """ resolved_fortran = shutil.which(executable, path=search_path) if resolved_fortran is None: @@ -105,6 +110,7 @@ def from_fortran_executable( execute_commands=execute_commands, search_path=search_path, executables={"fortran": resolved_fortran, "c": resolved_c}, + standard_logicals=standard_logicals, ) @classmethod @@ -172,9 +178,11 @@ def __init__( execute_commands: bool = True, search_path: str | None = None, executables: Mapping[str, str] | None = None, + standard_logicals: bool = True, ) -> None: self._toolchain = self._load_toolchain(vendor) self._debug = debug + self._standard_logicals = standard_logicals self._execute_commands = execute_commands self._search_path = search_path self._executables = {str(language): str(command) for language, command in (executables or {}).items()} @@ -352,6 +360,12 @@ def _flags( profile = "debug_flags" if self._debug else "release_flags" values = [*self._strings(language.get(profile, ())), *self._strings(language.get("general_flags", ()))] values.extend(self._supported_optional_flags(executable, language.get("optional_general_flags", ()))) + # A Fortran `logical` has no fixed representation, and some compilers + # default to one their own C compiler cannot read. The profile names the + # option selecting the interoperable form; a caller can decline it when + # linking objects already built the other way. + if self._standard_logicals: + values.extend(self._strings(language.get("logical_interop_flags", ()))) for tool in sorted(set(tools)): if tool != "python": values.extend(self._strings(self._tool_mapping(language, tool).get("flags", ()))) diff --git a/prik/contracts/__init__.py b/prik/contracts/__init__.py index 334b64cc4..7bebd6d16 100644 --- a/prik/contracts/__init__.py +++ b/prik/contracts/__init__.py @@ -96,15 +96,15 @@ def __call__(self, *args: object, **kwargs: object) -> object: shape_items = self.array.shape if isinstance(self.array.shape, tuple) else (self.array.shape,) if self.array.rank <= 0 or Ellipsis in shape_items: raise TypeError(f"{self.descriptor_kind} handle constructor requires one concrete positive array rank") - scalar_dtype = getattr(self.array.element_type, "_scalar_dtype", None) - if scalar_dtype is None: + array_dtype = getattr(self.array.element_type, "_array_dtype", None) + if array_dtype is None: name = getattr(self.array.element_type, "__name__", type(self.array.element_type).__name__) raise TypeError(f"{self.descriptor_kind} handle element contract {name!r} has no concrete NumPy dtype") from prik.runtime.handles import _native_array_handle_from_contract return _native_array_handle_from_contract( self.descriptor_kind, - scalar_dtype, + array_dtype, self.array.rank, ) @@ -113,6 +113,7 @@ def _contract_type( name: str, scalar_factory: object | None = None, *, + array_factory: object | None = None, constructor_error: str | None = None, ) -> type[_ContractType]: namespace = { @@ -121,6 +122,7 @@ def _contract_type( } if scalar_factory is not None: namespace["_scalar_dtype"] = np.dtype(scalar_factory) + namespace["_array_dtype"] = np.dtype(scalar_factory if array_factory is None else array_factory) return _ContractTypeMeta(name, (_ContractType,), namespace) @@ -164,9 +166,9 @@ def apply(target): Bool = _contract_type("Bool", _CONTRACT_NUMPY_FACTORIES["Bool"]) Bool8 = _contract_type("Bool8", _CONTRACT_NUMPY_FACTORIES["Bool8"]) -Bool16 = _contract_type("Bool16", _CONTRACT_NUMPY_FACTORIES["Bool16"]) -Bool32 = _contract_type("Bool32", _CONTRACT_NUMPY_FACTORIES["Bool32"]) -Bool64 = _contract_type("Bool64", _CONTRACT_NUMPY_FACTORIES["Bool64"]) +Bool16 = _contract_type("Bool16", _CONTRACT_NUMPY_FACTORIES["Bool16"], array_factory=np.int16) +Bool32 = _contract_type("Bool32", _CONTRACT_NUMPY_FACTORIES["Bool32"], array_factory=np.int32) +Bool64 = _contract_type("Bool64", _CONTRACT_NUMPY_FACTORIES["Bool64"], array_factory=np.int64) Byte = _contract_type("Byte", constructor_error="Byte has no portable NumPy scalar default") CEnum = _contract_type("CEnum", constructor_error="CEnum requires a resolved native underlying type") Char = _contract_type("Char", constructor_error="Char has no portable NumPy scalar default") diff --git a/prik/naming/native_symbols.py b/prik/naming/native_symbols.py index 1be3bffd5..253ce6f48 100644 --- a/prik/naming/native_symbols.py +++ b/prik/naming/native_symbols.py @@ -14,6 +14,13 @@ class NativeSymbolNames: """Create stable backend symbols within native compiler limits.""" + @classmethod + def bounded(cls, owner_path: str, preferred: str, *, limit: int = 63) -> str: + """Keep a valid readable symbol, compacting it only when it is too long.""" + if len(preferred) <= limit: + return preferred + return cls.compact(owner_path, preferred, limit=limit) + @staticmethod def compact(owner_path: str, preferred: str, *, limit: int = 27) -> str: """Return a readable, collision-resistant symbol fragment.""" diff --git a/prik/parsers/fortran/parser.py b/prik/parsers/fortran/parser.py index cf9024d22..9362a07be 100644 --- a/prik/parsers/fortran/parser.py +++ b/prik/parsers/fortran/parser.py @@ -292,6 +292,7 @@ class EnumUnit(SourceUnit): "contiguous": "contiguous", "external": "external", "parameter": "parameter", + "protected": "protected", } ) _EMPTY_COMPILE_TIME_SYMBOLS: Mapping[str, str] = MappingProxyType({}) @@ -356,6 +357,7 @@ class _Declaration: allocatable: bool = False pointer: bool = False target: bool = False + protected: bool = False contiguous: bool = False external: bool = False parameter: bool = False @@ -4324,6 +4326,8 @@ def _apply_internal_type_metadata(arg: FortranVariable, declaration: _Declaratio arg._declared_storage_bits = declaration.declared_storage_bits if declaration.polymorphic: arg._fortran_polymorphic = True + if declaration.protected: + arg._fortran_protected = True @staticmethod def _split_dim_bounds(dim: str) -> tuple[str | None, str | None]: diff --git a/prik/pipeline/build.py b/prik/pipeline/build.py index abb5f43b1..b8becc7a3 100644 --- a/prik/pipeline/build.py +++ b/prik/pipeline/build.py @@ -619,6 +619,7 @@ def _new_compiler( input_compiler: str | None = None, input_c_compiler: str | None = None, requires_fortran: bool = True, + standard_logicals: bool = True, ) -> Compiler: """Create the compiler configured for generated wrapper code. @@ -626,6 +627,10 @@ def _new_compiler( paired C compiler. A C-only plan uses ``input_c_compiler`` directly, so it neither discovers nor requires a Fortran compiler. The choice comes from explicit build-language records, never source suffixes. + + ``standard_logicals`` requests the option that makes a Fortran ``logical`` + interoperable with C, which some compilers do not select by default. Decline + it only to match objects already built the other way. """ search_path = get_condaless_search_path("verbose") if requires_fortran: @@ -635,6 +640,7 @@ def _new_compiler( debug=debug, execute_commands=execute_commands, search_path=search_path, + standard_logicals=standard_logicals, ) return Compiler.from_c_executable( input_c_compiler or "cc", @@ -652,6 +658,7 @@ def _c_build_compiler_and_preprocessing( requires_fortran: bool, execute_commands: bool, debug: bool, + standard_logicals: bool, ) -> tuple[Compiler | None, PreprocessingConfig]: """Select coherent C preprocessing and an optional early mixed compiler. @@ -666,6 +673,7 @@ def _c_build_compiler_and_preprocessing( compiler = _new_compiler( execute_commands=execute_commands, debug=debug, + standard_logicals=standard_logicals, input_compiler=input_compiler, requires_fortran=True, ) @@ -3284,6 +3292,7 @@ def build_fortran_extension( wrapper_compiler_debug: bool = False, wrapper_fortran_flags: Iterable[str] | None = None, wrapper_c_flags: Iterable[str] | None = None, + standard_logicals: bool = True, _on_total_build_time: Callable[[float], None] | None = None, ) -> WrapperBuildResult: """Build a Python extension from one or more Fortran source files. @@ -3351,6 +3360,15 @@ def build_fortran_extension( verbose, wrapper_compiler_debug, wrapper_fortran_flags, wrapper_c_flags Build progress output, generated-wrapper debug mode, and additional flags for generated bridge and binding compilation. + standard_logicals + Pass the compiler option that gives a Fortran ``logical`` the + representation C expects: ``-standard-semantics`` on Intel and + ``-Munixlogical`` on PGI/NVIDIA. On by default, because without it + those compilers store all bits set for ``.true.`` and a + ``logical(c_bool)`` reaching C holds ``255`` where ``_Bool`` is defined + to hold ``1``. Set false only to link prebuilt Intel objects compiled + without the option, which is also required for link compatibility + because it changes Intel module symbol mangling. Returns ------- @@ -3427,6 +3445,7 @@ def build_fortran_extension( compiler = _new_compiler( execute_commands=not generation_only, debug=wrapper_compiler_debug, + standard_logicals=standard_logicals, input_compiler=preprocessing.compiler if preprocessing.uses_compiler else None, ) native_source_objects, native_build_plan = _prepare_native_build_plan(native_inputs, output_path=output_path) @@ -3496,6 +3515,7 @@ def build_c_extension( wrapper_compiler_debug: bool = False, wrapper_fortran_flags: Iterable[str] | None = None, wrapper_c_flags: Iterable[str] | None = None, + standard_logicals: bool = True, _on_total_build_time: Callable[[float], None] | None = None, ) -> WrapperBuildResult: """Build a direct-only C extension from explicit C implementation sources. @@ -3512,6 +3532,11 @@ def build_c_extension( C functions and can explicitly select declarations from included headers. ``native_c_sources`` adds separately compiled C inputs, while explicit Fortran inputs are supported only as ordinary link dependencies. + ``standard_logicals`` controls whether those Fortran inputs are compiled + with the option that gives a ``logical`` the representation C expects + (``-standard-semantics`` on Intel, ``-Munixlogical`` on PGI/NVIDIA); it is + on by default and should be turned off only to link prebuilt Intel objects + compiled without it. ``preprocessing`` supplies the C preprocessing configuration used to expand ``sources`` before parsing; the default runs the selected C compiler. @@ -3549,6 +3574,7 @@ def build_c_extension( requires_fortran=requires_fortran, execute_commands=not generation_only, debug=wrapper_compiler_debug, + standard_logicals=standard_logicals, ) parsed_sources = tuple(_parse_c_wrapper_source(path, preprocessing) for path in source_paths) # Fail forms that are intrinsically outside the primitive lane before the @@ -3564,6 +3590,7 @@ def build_c_extension( compiler = compiler or _new_compiler( execute_commands=not generation_only, debug=wrapper_compiler_debug, + standard_logicals=standard_logicals, input_compiler=input_compiler, input_c_compiler=input_c_compiler, requires_fortran=requires_fortran, @@ -3662,6 +3689,7 @@ def build_pyi_extension( wrapper_compiler_debug: bool = False, wrapper_fortran_flags: Iterable[str] | None = None, wrapper_c_flags: Iterable[str] | None = None, + standard_logicals: bool = True, _on_total_build_time: Callable[[float], None] | None = None, ) -> WrapperBuildResult: """Build a Python extension from an editable semantic ``.pyi`` contract. @@ -3713,6 +3741,11 @@ def build_pyi_extension( verbose, wrapper_compiler_debug, wrapper_fortran_flags, wrapper_c_flags Progress, generated-wrapper debug mode, and generated bridge/binding compiler flags. + standard_logicals + Pass the compiler option that gives a Fortran ``logical`` the + representation C expects (``-standard-semantics`` on Intel, + ``-Munixlogical`` on PGI/NVIDIA). On by default; set false only to + link prebuilt Intel objects compiled without it. Returns ------- @@ -3797,6 +3830,7 @@ def build_pyi_extension( compiler = _new_compiler( execute_commands=not generation_only, debug=wrapper_compiler_debug, + standard_logicals=standard_logicals, input_compiler=input_compiler, input_c_compiler=selected_input_c_compiler, requires_fortran=_native_inputs_require_fortran(native_inputs) or native_language == "fortran", diff --git a/prik/pipeline/wrapper.py b/prik/pipeline/wrapper.py index 6a5f29611..ec9606f0c 100644 --- a/prik/pipeline/wrapper.py +++ b/prik/pipeline/wrapper.py @@ -36,6 +36,7 @@ from prik.semantics.metadata import SCALAR_STORAGE_CATEGORY from prik.policy.models import ( ArgumentHandoffMode, + ArrayEntrypointABI, ArrayLogicalABI, ArrayPythonLayout, ArrayWritebackABI, @@ -62,6 +63,7 @@ DerivedWriteback, DeclarationCallableAction, DirectResultABI, + EntrypointPassingConvention, LifecycleOperation, FIXED_STRING_RESULT_COPY_REASON, OWNED_NATIVE_ARRAY_HANDLE_COPY_REASON, @@ -521,15 +523,19 @@ def _required_header_diagnostics(self, plan: ModulePlan) -> tuple[WrapperPlanDia if handle is not None ) expected_headers = list(self._native_array_required_headers(handles)) - if any( - field.access - in { - DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR, - DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE, - } - for namespace in plan.namespaces - for derived in namespace.derived_types - for field in derived.fields + if ( + any( + field.access + in { + DerivedFieldAccessMechanism.ORDINARY_ARRAY_DESCRIPTOR, + DerivedFieldAccessMechanism.NATIVE_ARRAY_HANDLE, + } + for namespace in plan.namespaces + for derived in namespace.derived_types + for field in derived.fields + ) + or self._accepts_array_handle_actual(plan) + or self._uses_array_descriptor_abi(plan) ): expected_headers.append(NATIVE_ARRAY_POINTER_C_DESCRIPTOR_HEADER) expected = tuple(dict.fromkeys(expected_headers)) @@ -537,6 +543,34 @@ def _required_header_diagnostics(self, plan: ModulePlan) -> tuple[WrapperPlanDia return () return (self._diagnostic(plan.owner_path, "inconsistent-required-headers", plan.required_headers),) + @staticmethod + def _accepts_array_handle_actual(plan: ModulePlan) -> bool: + """Return whether an ordinary array argument accepts an array handle. + + The storage such a handle names is reached through its descriptor, so + the module needs the interop header even when nothing else in it does. + Only a Fortran argument accepts a handle, so the accepted sources are + the whole test. + """ + accepts = {NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE} + return any( + argument.native_array_actual is not None + and accepts.intersection(argument.native_array_actual.accepted_sources) + for namespace in plan.namespaces + for function in namespace.functions + for argument in function.arguments + ) + + @staticmethod + def _uses_array_descriptor_abi(plan: ModulePlan) -> bool: + """Return whether an ordinary argument uses the standard descriptor ABI.""" + return any( + argument.array is not None and argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + for namespace in plan.namespaces + for function in namespace.functions + for argument in function.arguments + ) + def _namespace_native_array_handles( self, namespace: NamespacePlan, @@ -1257,6 +1291,10 @@ def _module_borrowed_array_view_diagnostics( diagnostics.append(self._diagnostic(plan.owner_path, "module-array-view-has-unrelated-facet", None)) if plan.entrypoint.getter_role is None: diagnostics.append(self._diagnostic(plan.owner_path, "missing-module-array-getter-role", None)) + # The route to the array's base address is a policy decision. Bridge + # lowering reads it; it must never fall back to one when it is absent. + if plan.array_address is None: + diagnostics.append(self._diagnostic(plan.owner_path, "missing-module-array-address-mechanism", None)) if plan.bridge.native_assignment is not AssignmentMode.NONE: diagnostics.append( self._diagnostic( @@ -1948,12 +1986,9 @@ def _array_writeback_abi_diagnostics( if plan.entrypoint.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER and ( plan.mutates_native or self._publishes_array_replacement(plan) ): - if plan.array_logical_abi is ArrayLogicalABI.NATIVE_KIND_COPY: - expected = ArrayWritebackABI.NOT_APPLICABLE - elif plan.datatype_family is DatatypeFamily.BOOL: - expected = ArrayWritebackABI.LOGICAL_LOW_BIT_INT8 - else: - expected = ArrayWritebackABI.NATIVE_ARRAY + # Every element type is written back the same way: a Boolean one + # already holds the zero or one its interoperable form requires. + expected = ArrayWritebackABI.NATIVE_ARRAY if plan.array_writeback_abi is expected: return () return ( @@ -2821,7 +2856,11 @@ def _uses_typed_derived_value(plan: ArgumentTransferPlan) -> bool: def _expected_native_descriptor_data_action(self, plan: ArgumentTransferPlan) -> BridgeDataAction: """Distinguish call-local facts from persistent projected descriptors.""" handle = plan.native_array_handle - if handle is not None and handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR: + if ( + handle is not None + and handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + and handle.output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE + ): return BridgeDataAction.DIRECT_TRANSFER return BridgeDataAction.ASSOCIATE_VIEW @@ -2929,7 +2968,10 @@ def _native_array_handle_argument_action_diagnostics( for name, actual, required in expected if actual is not required ) - projected = handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + projected = ( + handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + and handle.output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE + ) expected_codegen = CodegenAction.IN_PLACE_ARGUMENT if projected else CodegenAction.CALL_LOCAL_INPUT if plan.binding.codegen_action is not expected_codegen: diagnostics.append( @@ -2952,7 +2994,10 @@ def _native_array_handle_argument_ownership_diagnostics( ) expected_destruction = ( DestructionPolicy.CALLER - if handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + if ( + handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + and handle.output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE + ) else DestructionPolicy.NONE ) if plan.destruction_policy is not expected_destruction: @@ -3040,12 +3085,10 @@ def _native_array_default_handle_operation_diagnostics( roles = handle.default_handle.operation_roles required = { NativeArrayOperation.SHAPE, - NativeArrayOperation.ARRAY_ACTUAL, - NativeArrayOperation.DESCRIPTOR, NativeArrayOperation.DESTROY, } if handle.descriptor_kind is NativeArrayDescriptorKind.POINTER: - required.add(NativeArrayOperation.ASSOCIATE) + required.update({NativeArrayOperation.ASSOCIATE, NativeArrayOperation.DESCRIPTOR}) diagnostics = [] complete = len(set(operations)) == len(operations) and required.issubset(operations) if not complete: @@ -3064,7 +3107,6 @@ def _native_array_default_handle_storage_diagnostics( """Match persistent owner storage and descriptor ABI to construction.""" default = handle.default_handle expected_owner_role = { - NativeArrayDefaultConstruction.FACT_PACKED_EMPTY: None, NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR: True, }[default.construction] owner_role = True if default.owner_storage_role is not None else None @@ -3075,11 +3117,13 @@ def _native_array_default_handle_storage_diagnostics( owner_path, "inconsistent-default-handle-owner-storage-role", default.owner_storage_role ) ) - expected_abi = { - NativeArrayDefaultConstruction.FACT_PACKED_EMPTY: NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL, - NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR: NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR, - }[default.construction] - if handle.handoff.abi is not expected_abi: + # A default handle that owns a lazily created descriptor requires the + # direct handoff: the storage it attaches is what crosses. A result + # keeps its own owned storage instead and never attaches one. + if ( + default.construction is NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR + and handle.handoff.abi is not NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + ): diagnostics.append( self._diagnostic(owner_path, "inconsistent-default-handle-descriptor-abi", handle.handoff.abi) ) @@ -3130,11 +3174,17 @@ def _native_array_actual_diagnostics( if actual is None: return () array = plan.array - expected_sources = ( - NativeArraySourceKind.NDARRAY, + expected_sources = (NativeArraySourceKind.NDARRAY,) + handle_sources = { NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE, - ) + } + if handle_sources.intersection(actual.accepted_sources): + expected_sources = ( + *expected_sources, + NativeArraySourceKind.ALLOCATABLE_HANDLE, + NativeArraySourceKind.POINTER_HANDLE, + ) diagnostics = [ *self._native_array_actual_source_diagnostics(plan, expected_sources), *self._native_array_actual_shape_diagnostics(plan), @@ -3302,17 +3352,11 @@ def _native_descriptor_handoff_diagnostics( argument: ArgumentTransferPlan | None, ) -> tuple[WrapperPlanDiagnostic, ...]: """Validate descriptor ABI roles without reconstructing its policy.""" - handoff = handle.handoff rank = handle.array.rank diagnostics = [] if rank is None: return (self._diagnostic(owner_path, "missing-native-descriptor-rank", None),) - expected_counts = ( - len(handoff.lower_bound_roles), - len(handoff.extent_roles), - len(handoff.stride_multiplier_roles), - ) - diagnostics.extend(self._native_descriptor_abi_diagnostics(owner_path, handle, expected_counts)) + diagnostics.extend(self._native_descriptor_abi_diagnostics(owner_path, handle)) diagnostics.extend(self._native_descriptor_presence_diagnostics(owner_path, handle, argument)) diagnostics.extend(self._native_array_operation_diagnostics(owner_path, handle)) return tuple(diagnostics) @@ -3321,11 +3365,9 @@ def _native_descriptor_abi_diagnostics( self, owner_path: str, handle: NativeArrayHandlePlan, - expected_counts: tuple[int, int, int], ) -> tuple[WrapperPlanDiagnostic, ...]: """Dispatch exact role validation by typed descriptor ABI.""" handlers = { - NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL: self._fact_packed_descriptor_diagnostics, NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR: self._direct_descriptor_diagnostics, NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE: self._owned_descriptor_diagnostics, } @@ -3333,44 +3375,20 @@ def _native_descriptor_abi_diagnostics( handler = handlers[handle.handoff.abi] except KeyError: return (self._diagnostic(owner_path, "unknown-native-descriptor-handoff", handle.handoff.abi),) - return handler(owner_path, handle, expected_counts) - - def _fact_packed_descriptor_diagnostics( - self, - owner_path: str, - handle: NativeArrayHandlePlan, - expected_counts: tuple[int, int, int], - ) -> tuple[WrapperPlanDiagnostic, ...]: - """Validate every call-local descriptor fact role.""" - handoff = handle.handoff - diagnostics = [] - expected_rank = handle.array.rank - if expected_counts != (expected_rank, expected_rank, expected_rank): - diagnostics.append( - self._diagnostic(owner_path, "inconsistent-native-descriptor-axis-roles", expected_counts) - ) - if None in { - handoff.descriptor_pointer_role, - handoff.base_addr_role, - handoff.elem_len_role, - handoff.rank_role, - }: - diagnostics.append(self._diagnostic(owner_path, "missing-native-descriptor-fact-role", None)) - if handoff.owner_storage_role is not None: - diagnostics.append(self._diagnostic(owner_path, "fact-packed-has-owner-storage", None)) - return tuple(diagnostics) + return handler(owner_path, handle) def _direct_descriptor_diagnostics( self, owner_path: str, handle: NativeArrayHandlePlan, - expected_counts: tuple[int, int, int], ) -> tuple[WrapperPlanDiagnostic, ...]: """Validate one persistent projected standard-descriptor pointer.""" diagnostics = [] - if handle.handoff.descriptor_pointer_role is None or any(expected_counts): + if handle.handoff.descriptor_pointer_role is None: diagnostics.append(self._diagnostic(owner_path, "invalid-direct-native-descriptor-roles", None)) - if handle.output_projection is not NativeArrayOutputProjection.PROJECTED_HANDLE: + if handle.output_projection is not NativeArrayOutputProjection.PROJECTED_HANDLE and ( + handle.descriptor_kind not in {NativeArrayDescriptorKind.ALLOCATABLE, NativeArrayDescriptorKind.POINTER} + ): diagnostics.append(self._diagnostic(owner_path, "direct-descriptor-without-projection", None)) return tuple(diagnostics) @@ -3378,12 +3396,10 @@ def _owned_descriptor_diagnostics( self, owner_path: str, handle: NativeArrayHandlePlan, - expected_counts: tuple[int, int, int], ) -> tuple[WrapperPlanDiagnostic, ...]: """Validate persistent wrapper-owned result descriptor storage roles.""" handoff = handle.handoff - invalid = handoff.owner_storage_role is None or handoff.descriptor_pointer_role is not None - if invalid or any(expected_counts): + if handoff.owner_storage_role is None or handoff.descriptor_pointer_role is not None: return (self._diagnostic(owner_path, "invalid-owned-native-descriptor-roles", None),) return () @@ -3430,14 +3446,16 @@ def _native_array_operation_diagnostics( def _required_native_array_operations( handle: NativeArrayHandlePlan, ) -> set[NativeArrayOperation]: - """Return common operations required by the completed descriptor kind.""" - required = { - NativeArrayOperation.SHAPE, - NativeArrayOperation.ARRAY_ACTUAL, - NativeArrayOperation.DESCRIPTOR, - } - if handle.descriptor_kind is NativeArrayDescriptorKind.POINTER: - required.add(NativeArrayOperation.ASSOCIATE) + """Return common operations required by the completed descriptor kind. + + Every handle reports its shape. A pointer additionally reports the + descriptor it is associated with and accepts a new association, because + a pointer that has no storage of its own has nowhere else to record + what another pointer was pointed at. + """ + required = {NativeArrayOperation.SHAPE} + if handle.descriptor_kind is NativeArrayDescriptorKind.POINTER and handle.descriptor_inquiries: + required.update({NativeArrayOperation.ASSOCIATE, NativeArrayOperation.DESCRIPTOR}) return required def _array_action_diagnostics( @@ -3859,12 +3877,39 @@ def _array_layout_role_diagnostics( if array is None: return () return ( + *self._array_entrypoint_abi_diagnostics(plan), *self._array_order_diagnostics(plan), *self._array_axis_mode_diagnostics(plan), *self._array_stride_role_diagnostics(plan), *self._array_dense_actual_role_diagnostics(plan), ) + def _array_entrypoint_abi_diagnostics( + self, + plan: ArgumentTransferPlan, + ) -> tuple[WrapperPlanDiagnostic, ...]: + """Require the transport fields selected by completed array ABI policy.""" + array = plan.array + if array is None: + return () + diagnostics = [] + if array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR: + if plan.entrypoint.passing is not EntrypointPassingConvention.C_DESCRIPTOR_POINTER: + diagnostics.append( + self._diagnostic(plan.owner_path, "invalid-array-descriptor-passing", plan.entrypoint.passing.value) + ) + if plan.entrypoint.pass_array_metadata: + diagnostics.append(self._diagnostic(plan.owner_path, "unexpected-array-descriptor-metadata", None)) + if array.upper_bound_roles or array.stride_roles or array.dense_actual_role is not None: + diagnostics.append(self._diagnostic(plan.owner_path, "unexpected-array-descriptor-roles", None)) + elif array.entrypoint_abi is not ArrayEntrypointABI.RAW_ADDRESS: + diagnostics.append(self._diagnostic(plan.owner_path, "invalid-array-entrypoint-abi", array.entrypoint_abi)) + if array.signed_strides and ( + array.entrypoint_abi is not ArrayEntrypointABI.C_DESCRIPTOR or array.contiguous is True + ): + diagnostics.append(self._diagnostic(plan.owner_path, "invalid-array-signed-strides", None)) + return tuple(diagnostics) + def _array_dense_actual_role_diagnostics( self, plan: ArgumentTransferPlan, @@ -3873,7 +3918,13 @@ def _array_dense_actual_role_diagnostics( array = plan.array if array is None: return () - expected = f"{plan.owner_path}:dense-actual" if array.contiguous is False and array.rank is not None else None + expected = ( + f"{plan.owner_path}:dense-actual" + if array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS + and array.contiguous is False + and array.rank is not None + else None + ) if array.dense_actual_role != expected: return (self._diagnostic(plan.owner_path, "invalid-array-dense-actual-role", array.dense_actual_role),) return () @@ -3904,7 +3955,7 @@ def _array_axis_mode_diagnostics(self, plan: ArgumentTransferPlan) -> tuple[Wrap return () if array.contiguous is True and any(axis != "dense" for axis in array.axes): return (self._diagnostic(plan.owner_path, "invalid-array-axis-modes", array.axes),) - if array.contiguous is False and "strided" not in array.axes: + if array.contiguous is False and array.rank is not None and "strided" not in array.axes: return (self._diagnostic(plan.owner_path, "invalid-array-axis-modes", array.axes),) return () @@ -3913,7 +3964,7 @@ def _array_stride_role_diagnostics(self, plan: ArgumentTransferPlan) -> tuple[Wr array = plan.array if array is None: return () - if array.contiguous is False: + if array.contiguous is False and array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS: return ( *self._required_array_stride_role_diagnostics(plan), *self._array_stride_role_count_diagnostics(plan), diff --git a/prik/planning/entrypoints.py b/prik/planning/entrypoints.py index 43df37715..388ad1746 100644 --- a/prik/planning/entrypoints.py +++ b/prik/planning/entrypoints.py @@ -24,7 +24,6 @@ ModuleGetterAction, ModuleObjectAccessMechanism, NativeArrayDefaultConstruction, - NativeArrayDescriptorInterop, NativeArrayOperation, NativeDescriptorHandoffABI, ) @@ -50,31 +49,21 @@ ) -_FIELD_HANDLE_LOCAL_OPERATIONS = frozenset( +# Answered in the binding from the descriptor the handle's entry point +# supplies, so no Fortran procedure is planned for them. +_DESCRIPTOR_ANSWERED_OPERATIONS = frozenset( { - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, - NativeArrayOperation.TO_NUMPY, - NativeArrayOperation.ARRAY_ACTUAL, - } -) -_MODULE_HANDLE_LOCAL_OPERATIONS = frozenset( - { - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, + NativeArrayOperation.ALLOCATED, + NativeArrayOperation.ASSOCIATED, + NativeArrayOperation.CONTIGUOUS, + NativeArrayOperation.DESCRIPTOR, + NativeArrayOperation.ELEMENT_LENGTH, + NativeArrayOperation.SHAPE, NativeArrayOperation.TO_NUMPY, } ) _OWNED_HANDLE_ENTRYPOINT_OPERATIONS = frozenset( { - NativeArrayOperation.ALLOCATED, - NativeArrayOperation.ASSOCIATED, - NativeArrayOperation.CONTIGUOUS, - NativeArrayOperation.SHAPE, NativeArrayOperation.ASSOCIATE, NativeArrayOperation.DEALLOCATE, NativeArrayOperation.NULLIFY, @@ -236,6 +225,7 @@ def _operation( GeneratedSupportProcedureImplementationOwner.FORTRAN ), ) -> GeneratedSupportProcedureEntrypointPlan: + symbol_name = NativeSymbolNames.bounded(f"{owner_path}::{role}", symbol_name) return GeneratedSupportProcedureEntrypointPlan( key=f"{owner_path}::{role}", owner_path=owner_path, @@ -700,17 +690,19 @@ def _string_field_operations(self, owner, field, route, owner_path, owner_parame def _ordinary_array_field_operations(self, owner, field, route, owner_path, owner_parameter): owner_values = (self._opaque_parameter("owner", fortran_name="owner_address"),) if owner_parameter else () - callback = self._descriptor_callback_parameter( - semantic_type_name=field.semantic_type_name, - rank=field.array.rank, - descriptor_kind=None, + # A fixed field reports its base address and extents, the same shape a + # fixed module array uses. Its rank is fixed and its storage contiguous, + # so a descriptor would carry nothing the extents do not already give. + extents = tuple( + self._int64_parameter(f"extent_{axis}", reference=True, intent="out") for axis in range(field.array.rank) ) operations = [ self._operation( owner_path, f"field:{route}:get", self._field_symbol(owner, field, route, "get"), - (*owner_values, callback, self._opaque_parameter("context")), + (*owner_values, *extents), + self._opaque_result(), ) ] if field.setter_action is SetterAction.WRITE_THROUGH: @@ -730,9 +722,18 @@ def _field_handle_operations(self, owner, field, route, owner_path, owner_parame raise ValueError(f"Native handle field {field.owner_path!r} has no completed rank") owner_values = (self._opaque_parameter("owner", fortran_name="owner_address"),) if owner_parameter else () operations = [] - for operation in handle.operations: - if operation in _FIELD_HANDLE_LOCAL_OPERATIONS: - continue + # The descriptor entry point is what every inquiry runs through, so it + # is planned for the handle rather than for one of its capabilities. + planned = [NativeArrayOperation.DESCRIPTOR] if handle.descriptor_inquiries else [] + planned.extend( + operation + for operation in handle.operations + # The descriptor entry point is already planned, and a NumPy view is + # a Python object, which only the binding can build. + if operation not in {NativeArrayOperation.DESCRIPTOR, NativeArrayOperation.TO_NUMPY} + and (not handle.descriptor_inquiries or operation not in _DESCRIPTOR_ANSWERED_OPERATIONS) + ) + for operation in planned: signature = self._field_handle_signature(field, handle, operation, owner_values) operations.append( self._operation( @@ -758,7 +759,8 @@ def _field_handle_signature(self, field, handle, operation, owner_values): extents = tuple( self._int64_parameter(f"extent_{axis}", reference=True) for axis in range(handle.array.rank) ) - return NativeEntrypointSignaturePlan((*owner_values, *extents), self._void_result()) + result = self._bool_result() if not handle.descriptor_inquiries else self._void_result() + return NativeEntrypointSignaturePlan((*owner_values, *extents), result) if operation is NativeArrayOperation.DESCRIPTOR: callback = self._descriptor_callback_parameter( semantic_type_name=field.semantic_type_name, @@ -992,9 +994,18 @@ def _module_native_array_operations(self, variable): if handle is None or handle.array.rank is None: raise ValueError(f"Module handle {variable.owner_path!r} has no completed operation plan") operations = [] - for operation in handle.operations: - if operation in _MODULE_HANDLE_LOCAL_OPERATIONS: - continue + # The descriptor entry point is what every inquiry runs through, so it + # is planned for the handle rather than for one of its capabilities. + planned = [NativeArrayOperation.DESCRIPTOR] if handle.descriptor_inquiries else [] + planned.extend( + operation + for operation in handle.operations + # The descriptor entry point is already planned, and a NumPy view is + # a Python object, which only the binding can build. + if operation not in {NativeArrayOperation.DESCRIPTOR, NativeArrayOperation.TO_NUMPY} + and (not handle.descriptor_inquiries or operation not in _DESCRIPTOR_ANSWERED_OPERATIONS) + ) + for operation in planned: signature = self._module_native_array_signature(variable, handle, operation) if signature is None: continue @@ -1018,23 +1029,15 @@ def _module_native_array_signature(self, variable, handle, operation): return NativeEntrypointSignaturePlan((), self._bool_result()) if operation is NativeArrayOperation.ELEMENT_LENGTH: return NativeEntrypointSignaturePlan((), self._int64_result()) - if operation is NativeArrayOperation.ARRAY_ACTUAL: - if self._uses_module_allocatable_descriptor(variable): - return self._module_descriptor_callback_signature(variable, handle) - return NativeEntrypointSignaturePlan((), self._opaque_result()) if operation is NativeArrayOperation.SHAPE: extents = tuple( self._int64_parameter(f"extent_{axis}", reference=True, intent="out") for axis in range(handle.array.rank) ) - return NativeEntrypointSignaturePlan(extents, self._void_result()) + result = self._bool_result() if not handle.descriptor_inquiries else self._void_result() + return NativeEntrypointSignaturePlan(extents, result) if operation is NativeArrayOperation.DESCRIPTOR: - if self._uses_module_allocatable_descriptor(variable): - return self._module_descriptor_callback_signature(variable, handle) - if handle.descriptor_kind.value != "pointer": - return None - descriptor = self._descriptor_parameter("descriptor", handle, variable.semantic_type_name, intent="out") - return NativeEntrypointSignaturePlan((descriptor,), self._void_result()) + return self._module_descriptor_callback_signature(variable, handle) if operation is NativeArrayOperation.ASSOCIATE: source = self._descriptor_parameter("source", handle, variable.semantic_type_name, intent="in") return NativeEntrypointSignaturePlan((source,), self._void_result()) @@ -1053,14 +1056,6 @@ def _module_descriptor_callback_signature(self, variable, handle): ) return NativeEntrypointSignaturePlan((callback, self._opaque_parameter("context")), self._void_result()) - @staticmethod - def _uses_module_allocatable_descriptor(variable: ModuleVariablePlan) -> bool: - handle = variable.native_array_handle - return bool( - handle is not None - and handle.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR - ) - @staticmethod def _nullable_derived_module_proxy(variable: ModuleVariablePlan) -> bool: return bool( diff --git a/prik/planning/models.py b/prik/planning/models.py index b0d5cb33b..a9de7f462 100644 --- a/prik/planning/models.py +++ b/prik/planning/models.py @@ -34,6 +34,7 @@ ArgumentConversionPhase, ArgumentHandoffMode, ArrayLogicalABI, + ArrayEntrypointABI, ArrayPythonLayout, ArrayWritebackABI, BridgeDataAction, @@ -65,9 +66,11 @@ DirectResultABI, DeclarationCallableAction, ExternalDeclarationMode, + ModuleArrayAddressMechanism, ModuleGetterAction, ModuleObjectAccessMechanism, NativeArrayDescriptorInterop, + NativeArrayDescriptorAttribute, CharacterLocalRelease, NativeArrayDescriptorKind, NativeArrayDescriptorOwnership, @@ -478,6 +481,10 @@ class ArrayHandoffPlan(StageRecord): native_order: str | None contiguous: bool | None python_layout: ArrayPythonLayout + # How the dummy is reached, and whether an axis may run backwards. Both are + # completed in policy; a backend implements the mechanism they name. + entrypoint_abi: ArrayEntrypointABI + signed_strides: bool minimum_rank: int maximum_rank: int flatten_python_storage: bool @@ -509,7 +516,7 @@ class NativeArrayActualPlan(StageRecord): accepted_sources: tuple[NativeArraySourceKind, ...] dtype: str - rank: int + rank: int | None shape: tuple[str, ...] order: str | None writable: bool @@ -530,12 +537,6 @@ class NativeDescriptorHandoffPlan(StageRecord): abi: NativeDescriptorHandoffABI descriptor_pointer_role: str | None - base_addr_role: str | None - elem_len_role: str | None - rank_role: str | None - lower_bound_roles: tuple[str, ...] - extent_roles: tuple[str, ...] - stride_multiplier_roles: tuple[str, ...] presence_role: str | None owner_storage_role: str | None operation_roles: tuple[tuple[NativeArrayOperation, str], ...] @@ -569,6 +570,7 @@ class NativeArrayHandlePlan(StageRecord): """ descriptor_kind: NativeArrayDescriptorKind + descriptor_attribute: NativeArrayDescriptorAttribute handle_kind: NativeArrayHandleKind origin: NativeArrayHandleOrigin owner: OwnershipOwner @@ -585,6 +587,14 @@ class NativeArrayHandlePlan(StageRecord): destroy_behavior: NativeArrayDestroyBehavior extraction_action: NativeArrayExtractionAction descriptor_interop: NativeArrayDescriptorInterop + # A handle answers its inquiries -- shape, state, element width, + # contiguity, the NumPy view -- from the live descriptor its entry point + # supplies, unless its declaration cannot cross a bind(C) descriptor + # interface at all. A deferred-length character pointer is that case: the + # standard does not allow such a dummy in a bind(C) interface, and GNU + # Fortran mistranslates the descriptor rather than rejecting it, so those + # inquiries stay on generated Fortran procedures of their own. + descriptor_inquiries: bool nullable: bool optional_absent: bool storage_mode: StorageMode @@ -783,6 +793,10 @@ class ModuleVariablePlan(StageRecord): derived: DerivedModuleObjectPlan | None = None character_length: int | None = None docstring: str | None = None + # Present only for a borrowed fixed-array view. Both backends read it: the + # bridge to reach the address, the binding to define the C helper that one + # of the two mechanisms calls. It is therefore a shared fact, not a facet. + array_address: ModuleArrayAddressMechanism | None = None @dataclass diff --git a/prik/planning/planner.py b/prik/planning/planner.py index b681ef79f..02902d32d 100644 --- a/prik/planning/planner.py +++ b/prik/planning/planner.py @@ -21,6 +21,7 @@ from prik.policy.models import ( ArgumentConversionPhase, ArgumentHandoffMode, + ArrayEntrypointABI, ArrayHandoffPolicy, CallbackHandoffPolicy, CallbackResultPolicy, @@ -37,6 +38,7 @@ ModuleGetterAction, ModuleObjectAccessMechanism, ModuleVariablePolicy, + NativeArraySourceKind, OverloadPolicy, OptionalMode, ArgumentPolicy, @@ -1153,6 +1155,7 @@ def _module_variable_plan( native_assignment=policy.native_assignment, ), character_length=policy.character_length, + array_address=policy.array_address, array=self._array_plan(policy.array, policy.owner_path), native_array_handle=self._native_array_handle_plan(policy.native_array_handle, policy.owner_path), derived=( @@ -2235,6 +2238,7 @@ def _native_array_handle_plan( raise ValueError(f"Native array handle {owner_path!r} is missing its array data facet") return NativeArrayHandlePlan( descriptor_kind=policy.descriptor_kind, + descriptor_attribute=policy.descriptor_attribute, handle_kind=policy.handle_kind, origin=policy.origin, owner=policy.owner, @@ -2251,6 +2255,7 @@ def _native_array_handle_plan( destroy_behavior=policy.destroy_behavior, extraction_action=policy.extraction_action, descriptor_interop=policy.descriptor_interop, + descriptor_inquiries=policy.descriptor_inquiries, nullable=policy.nullable, optional_absent=policy.optional_absent, storage_mode=policy.storage_mode, @@ -2291,18 +2296,9 @@ def _native_descriptor_handoff_plan( operations, ) -> NativeDescriptorHandoffPlan: """Name descriptor facts once for binding, bridge, and lifecycle consumers.""" - fact_packed = policy.abi is NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL return NativeDescriptorHandoffPlan( abi=policy.abi, descriptor_pointer_role=self._native_descriptor_pointer_role(policy, owner_path), - base_addr_role=self._native_descriptor_fact_role(owner_path, "base-addr", fact_packed), - elem_len_role=self._native_descriptor_fact_role(owner_path, "elem-len", fact_packed), - rank_role=self._native_descriptor_fact_role(owner_path, "descriptor-rank", fact_packed), - lower_bound_roles=self._native_descriptor_axis_roles(owner_path, policy.rank, "lower-bound", fact_packed), - extent_roles=self._native_descriptor_axis_roles(owner_path, policy.rank, "descriptor-extent", fact_packed), - stride_multiplier_roles=self._native_descriptor_axis_roles( - owner_path, policy.rank, "stride-multiplier", fact_packed - ), presence_role=self._native_descriptor_presence_role(policy, owner_path), owner_storage_role=self._native_descriptor_owner_role(policy, owner_path), operation_roles=tuple((operation, f"{owner_path}:operation:{operation.value}") for operation in operations), @@ -2318,10 +2314,6 @@ def _native_descriptor_pointer_role( return None return f"{owner_path}:descriptor" - def _native_descriptor_fact_role(self, owner_path: str, label: str, enabled: bool) -> str | None: - """Name one fact-packed scalar descriptor field.""" - return f"{owner_path}:{label}" if enabled else None - def _native_descriptor_presence_role( self, policy: NativeDescriptorHandoffPolicy, @@ -2340,19 +2332,6 @@ def _native_descriptor_owner_role( return f"{owner_path}:owner-storage" return None - def _native_descriptor_axis_roles( - self, - owner_path: str, - rank: int, - label: str, - enabled: bool, - ) -> tuple[str, ...]: - """Name one standard-descriptor field role per declared axis.""" - if not enabled: - return () - return tuple(f"{owner_path}:{label}:{axis}" for axis in range(rank)) - - # Ordinary-array buffer and raw-address planning. def _array_plan( self, policy: ArrayHandoffPolicy | None, @@ -2383,6 +2362,8 @@ def _array_plan( native_order=policy.native_order, contiguous=policy.contiguous, python_layout=policy.python_layout, + entrypoint_abi=policy.entrypoint_abi, + signed_strides=policy.signed_strides, minimum_rank=policy.minimum_rank, maximum_rank=policy.maximum_rank, flatten_python_storage=policy.flatten_python_storage, @@ -2396,8 +2377,8 @@ def _array_plan( extent_callable_tokens=policy.extent_callable_references, extent_callable_roles=policy.extent_callable_roles, extent_evaluation=policy.extent_evaluation, - upper_bound_roles=self._array_layout_roles(owner_path, abi_rank, policy.contiguous, "upper-bound"), - stride_roles=self._array_layout_roles(owner_path, abi_rank, policy.contiguous, "stride"), + upper_bound_roles=self._array_layout_roles(policy, owner_path, abi_rank, "upper-bound"), + stride_roles=self._array_layout_roles(policy, owner_path, abi_rank, "stride"), dense_actual_role=self._array_dense_actual_role( policy, owner_path, @@ -2435,7 +2416,12 @@ def _array_dense_actual_role( enabled: bool, ) -> str | None: """Name the dense-view selector only for concrete strided inputs.""" - if not enabled or policy.rank is None or policy.contiguous is not False: + if ( + not enabled + or policy.entrypoint_abi is not ArrayEntrypointABI.RAW_ADDRESS + or policy.rank is None + or policy.contiguous is not False + ): return None return f"{owner_path}:dense-actual" @@ -2472,13 +2458,13 @@ def _array_itemsize_role(self, policy: ArrayHandoffPolicy, owner_path: str) -> s def _array_layout_roles( self, + policy: ArrayHandoffPolicy, owner_path: str, rank: int, - contiguous: bool | None, label: str, ) -> tuple[str, ...]: - """Name one ABI role per axis only for stride-aware layouts.""" - if contiguous is not False: + """Name per-axis metadata only when the raw ABI has to carry it.""" + if policy.entrypoint_abi is not ArrayEntrypointABI.RAW_ADDRESS or policy.contiguous is not False: return () return tuple(f"{owner_path}:{label}:{axis}" for axis in range(rank)) @@ -2586,10 +2572,42 @@ def _required_headers(self, namespaces: tuple[NamespacePlan, ...]) -> tuple[str, if handle is not None ) headers = list(self._native_array_headers(handles)) - if self._requires_derived_descriptor_header(namespaces): + if ( + self._requires_derived_descriptor_header(namespaces) + or self._accepts_array_handle_actual(namespaces) + or self._uses_array_descriptor_abi(namespaces) + ): headers.append(NATIVE_ARRAY_POINTER_C_DESCRIPTOR_HEADER) return tuple(dict.fromkeys(headers)) + @staticmethod + def _uses_array_descriptor_abi(namespaces: tuple[NamespacePlan, ...]) -> bool: + """Return whether an ordinary argument uses the standard descriptor ABI.""" + return any( + argument.array is not None and argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + for namespace in namespaces + for function in namespace.functions + for argument in function.arguments + ) + + @staticmethod + def _accepts_array_handle_actual(namespaces: tuple[NamespacePlan, ...]) -> bool: + """Return whether an ordinary array argument accepts an array handle. + + The storage such a handle names is reached through its descriptor, so a + module whose ordinary array dummies accept one needs the interop header + even when nothing else about the module does. Only a Fortran argument + accepts one, so no separate language test is needed here. + """ + accepts = {NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE} + return any( + argument.native_array_actual is not None + and accepts.intersection(argument.native_array_actual.accepted_sources) + for namespace in namespaces + for function in namespace.functions + for argument in function.arguments + ) + @staticmethod def _requires_derived_descriptor_header(namespaces: tuple[NamespacePlan, ...]) -> bool: """Return whether one derived field uses a standard C descriptor callback.""" diff --git a/prik/policy/completion.py b/prik/policy/completion.py index 0c397b22d..396e562ac 100644 --- a/prik/policy/completion.py +++ b/prik/policy/completion.py @@ -21,6 +21,7 @@ ObjectKind, SetterAction, default_ownership_policy, + declared_character_length, is_character_descriptor_update, ownership_context_for_argument, ) @@ -1374,12 +1375,38 @@ def _native_array_handle_policy( ) handle_kind = _native_array_handle_kind(descriptor_kind, context, optional_absent=optional_absent) blocker = _native_array_handle_blocker(descriptor_kind, handle_kind, decision) + if ( + blocker is None + and context.is_argument + and semantic_type.name == "String" + and declared_character_length(semantic_type.metadata) is not None + ): + blocker = ( + "fixed-width character allocatable and pointer array arguments have no interoperable descriptor interface" + ) + descriptor_inquiries = _native_array_descriptor_inquiries(descriptor_kind, semantic_type) + if not descriptor_inquiries and handle_kind not in { + "borrowed_module_descriptor", + "borrowed_field_descriptor", + }: + blocker = "deferred-length character pointer arrays cannot cross a bind(C) descriptor interface" descriptor_ownership = _native_array_descriptor_ownership(handle_kind) - to_numpy = _native_array_to_numpy_policy(descriptor_kind, handle_kind, decision, semantic_type) - operations = _native_array_handle_operations(descriptor_kind, handle_kind, context, semantic_type) + to_numpy = ( + _native_array_to_numpy_policy(descriptor_kind, handle_kind, decision, semantic_type) + if descriptor_inquiries + else "unsupported" + ) + operations = set(_native_array_handle_operations(descriptor_kind, handle_kind, context, semantic_type)) + if not descriptor_inquiries: + operations.difference_update({"allocate", "associate", "resize", "to_numpy"}) default_construction = _native_array_default_construction(handle_kind, context, semantic_type) return NativeArrayHandlePolicy( descriptor_kind=descriptor_kind, + descriptor_attribute=_native_array_descriptor_attribute( + descriptor_kind, + handle_kind, + semantic_type, + ), handle_kind=handle_kind, origin=_native_array_handle_origin(context), owner=_native_array_handle_owner(handle_kind), @@ -1389,7 +1416,9 @@ def _native_array_handle_policy( getter_behavior=_native_array_getter_behavior(handle_kind, context, blocker), python_setter=_native_array_python_setter(variable), native_setter=_native_array_native_setter(variable), - output_projection=_native_array_output_projection(descriptor_kind, handle_kind, context), + output_projection=( + _native_array_output_projection(descriptor_kind, handle_kind, context) if descriptor_inquiries else "none" + ), result_allocation=_native_array_result_allocation(descriptor_kind, handle_kind, context, semantic_type), release=_native_array_release_responsibility(handle_kind), target_lifetime=_native_array_target_lifetime(descriptor_kind, handle_kind, semantic_type, blocker), @@ -1399,17 +1428,19 @@ def _native_array_handle_policy( descriptor_kind, handle_kind, semantic_type, + descriptor_inquiries=descriptor_inquiries, ), nullable=bool(decision.nullable or optional_absent), optional_absent=optional_absent, storage_mode=decision.storage_mode.value, - operations=operations, + operations=tuple(sorted(operations)), blocker=blocker, default_construction=default_construction, default_descriptor_ownership="owned" if default_construction != "none" else "unknown", default_release="wrapper_dealloc" if default_construction != "none" else "none", default_destroy_behavior="handle_finalizer" if default_construction != "none" else "none", default_operations=(tuple(sorted({*operations, "destroy"})) if default_construction != "none" else ()), + descriptor_inquiries=descriptor_inquiries, ) @@ -1440,16 +1471,19 @@ def _native_array_default_construction( context: OwnershipContext, semantic_type: models.SemanticType, ) -> str: - """Complete how a runtime-constructed descriptor reaches one argument.""" + """Complete how a runtime-constructed descriptor reaches one argument. + + This pairs with the handoff ABI: an argument whose descriptor crosses + directly is handed a descriptor the Fortran runtime built, so a caller who + supplies a contract-default handle needs storage of its own to hand over. + """ if ( semantic_type.name == "String" or handle_kind not in {"argument_descriptor", "optional_absent_handle"} or not context.is_argument ): return "none" - if context.projects_result: - return "lazy_owned_descriptor" - return "fact_packed_empty" + return "lazy_owned_descriptor" def _native_array_handle_origin(context: OwnershipContext) -> str: @@ -1622,7 +1656,7 @@ def _native_array_to_numpy_policy( return _native_array_pointer_to_numpy_policy(semantic_type) if decision.is_blocked: return "unsupported" - if handle_kind == "borrowed_module_descriptor" and not semantic_type.metadata.get("aliased"): + if handle_kind == "borrowed_module_descriptor": return "descriptor_view" return "borrowed_view" @@ -1693,21 +1727,48 @@ def _native_array_descriptor_interop_requirement( descriptor_kind: str, handle_kind: str, semantic_type: models.SemanticType, + *, + descriptor_inquiries: bool, ) -> str: """Return the C-descriptor interop mechanism required by a supported handle.""" + if not descriptor_inquiries: + return "none" if descriptor_kind == "allocatable" and handle_kind == "owned_result_descriptor": return "owned_allocatable_c_descriptor" - if ( - descriptor_kind == "allocatable" - and handle_kind == "borrowed_module_descriptor" - and not semantic_type.metadata.get("aliased") - ): + if descriptor_kind == "allocatable" and handle_kind == "borrowed_module_descriptor": return "module_allocatable_c_descriptor" if descriptor_kind == "pointer" and handle_kind != "unsupported": return "pointer_c_descriptor" return "none" +def _native_array_descriptor_attribute( + descriptor_kind: str, + handle_kind: str, + semantic_type: models.SemanticType, +) -> str: + """Complete the attribute a handle's descriptor callback can supply. + + A fixed-width character module variable or field cannot associate with an + interoperable allocatable or pointer character dummy, because those dummies + have to declare deferred length. Its callback therefore supplies an + ordinary assumed-shape descriptor. The native entity remains allocatable + or pointer; only the callback projection has the ``other`` attribute. + """ + fixed_character = semantic_type.name == "String" and declared_character_length(semantic_type.metadata) is not None + if fixed_character and handle_kind in {"borrowed_module_descriptor", "borrowed_field_descriptor"}: + return "other" + return descriptor_kind + + +def _native_array_descriptor_inquiries( + descriptor_kind: str, + semantic_type: models.SemanticType, +) -> bool: + """Return whether the declaration has a legal bind(C) descriptor interface.""" + return not (descriptor_kind == "pointer" and semantic_type.metadata.get("fortran_character_length") == ":") + + def _pointer_policy_metadata(semantic_type: models.SemanticType) -> dict[str, object]: """Copy pointer-policy metadata into a safe mutable mapping, or return an empty mapping.""" policy = semantic_type.metadata.get(POINTER_POLICY_METADATA) diff --git a/prik/policy/construction.py b/prik/policy/construction.py index 86657f67d..a584630e9 100644 --- a/prik/policy/construction.py +++ b/prik/policy/construction.py @@ -93,6 +93,7 @@ CallbackThreadAction, CallbackGILAction, CallbackFatalAction, + ModuleArrayAddressMechanism, ModuleGetterAction, ModuleObjectAccessMechanism, DerivedFieldAccessMechanism, @@ -129,6 +130,7 @@ ClassSurfacePolicy, CharacterLocalPolicy, CharacterLocalRelease, + NativeArrayDescriptorAttribute, NativeArrayDescriptorKind, NativeArrayHandleKind, NativeDescriptorHandoffABI, @@ -148,6 +150,7 @@ NativeStatusErrorPolicy, ModuleVariablePolicy, LifecyclePolicy, + ArrayEntrypointABI, ArrayHandoffPolicy, ProcedurePrototypeArgumentPolicy, ProcedurePrototypeResultPolicy, @@ -1134,6 +1137,7 @@ def _ordinary_array_module_variable_policy( array: ArrayHandoffPolicy, ) -> ModuleVariablePolicy: """Build one borrowed ordinary module-array view policy.""" + address = _ordinary_array_module_address_mechanism(variable) blockers = _ordinary_array_module_variable_blockers(variable, getter, setter, array) return ModuleVariablePolicy( **_module_variable_policy_base(variable, module_name, owner_path), @@ -1148,9 +1152,25 @@ def _ordinary_array_module_variable_policy( supported=not blockers, blockers=tuple(blockers), array=array, + array_address=address, ) +def _ordinary_array_module_address_mechanism( + variable: models.SemanticVariable, +) -> ModuleArrayAddressMechanism: + """Select how the bridge obtains one fixed module array's base address. + + Addressable storage names itself directly. An ordinary declaration cannot, + so its whole array is handed to C through an assumed-size dummy, which + receives the bare base address. Both mechanisms borrow the same live + storage; only the route to its address differs. + """ + if variable.semantic_type.metadata.get("aliased"): + return ModuleArrayAddressMechanism.TARGET_ADDRESS + return ModuleArrayAddressMechanism.CAPTURED_ADDRESS + + def _constant_array_module_variable_policy( variable: models.SemanticVariable, module_name: str, @@ -1265,8 +1285,6 @@ def _ordinary_array_module_variable_blockers( blockers.append("ordinary module array requires one concrete fixed rank") if variable.semantic_type.name not in _PLAN_PRIMITIVE_SCALAR_TYPES | {"String"}: blockers.append("ordinary module array requires a primitive numeric element type") - if not variable.semantic_type.metadata.get("aliased"): - blockers.append("ordinary module array requires addressable Aliased target storage") expected_getter = ( ("owner", getter.owner, OwnershipOwner.NATIVE), ("transfer", getter.transfer, TransferMode.BORROWED_VIEW), @@ -1966,7 +1984,12 @@ def _complete_entrypoint_argument_route( ) ) ), - entrypoint_pass_array_metadata=(uses_adapter and argument.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER), + entrypoint_pass_array_metadata=( + uses_adapter + and argument.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER + and argument.array is not None + and argument.array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS + ), entrypoint_pass_descriptor_presence=(uses_adapter and argument.optional_mode is OptionalMode.DESCRIPTOR), entrypoint_pass_derived_transaction=(uses_adapter and argument.derived_call is not None), entrypoint_pass_callback_parameter=( @@ -2048,6 +2071,7 @@ def _argument_entrypoint_passing( function: models.SemanticFunction, argument: models.SemanticArgument, boundary: _ArgumentBoundaryPolicy, + array: ArrayHandoffPolicy | None, slot: NativeCallSlotPolicy | None, callback: CallbackHandoffPolicy | None, ) -> EntrypointPassingConvention: @@ -2064,6 +2088,12 @@ def _argument_entrypoint_passing( ) if boundary.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR: return EntrypointPassingConvention.C_DESCRIPTOR_POINTER + if ( + boundary.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER + and array is not None + and array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + ): + return EntrypointPassingConvention.C_DESCRIPTOR_POINTER if argument.optional: return EntrypointPassingConvention.NULLABLE_POINTER if direct_c_abi: @@ -2086,6 +2116,7 @@ def _argument_entrypoint_optionality( function: models.SemanticFunction, argument: models.SemanticArgument, boundary: _ArgumentBoundaryPolicy, + array: ArrayHandoffPolicy | None, slot: NativeCallSlotPolicy | None, ) -> EntrypointOptionalityAction: """Complete original native presence independently from the Python surface.""" @@ -2093,6 +2124,12 @@ def _argument_entrypoint_optionality( return EntrypointOptionalityAction.REQUIRED if boundary.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR: return EntrypointOptionalityAction.NULL_C_DESCRIPTOR_POINTER + if ( + boundary.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER + and array is not None + and array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + ): + return EntrypointOptionalityAction.NULL_C_DESCRIPTOR_POINTER if _argument_passes_by_value(argument, slot): return EntrypointOptionalityAction.ADAPTER_SIDE_FORTRAN_OMISSION if (function.origin.source_language == "fortran" and function.origin.native_abi == "c") or ( @@ -2508,8 +2545,9 @@ def slot_semantic_type(slot: NativeCallSlotPolicy) -> models.SemanticType | None semantic_argument = semantic_arguments_by_name.get(slot.python_name) return semantic_argument.semantic_type if semantic_argument is not None else None - parameters = tuple( - _direct_c_abi_type_policy( + def parameter_policy(slot: NativeCallSlotPolicy) -> DirectCABITypePolicy: + """Keep a descriptor array's standardized C declaration explicit.""" + policy = _direct_c_abi_type_policy( parameter_source[slot.native_position] if slot.native_position < len(parameter_source) and isinstance(parameter_source[slot.native_position], dict) else None, @@ -2527,8 +2565,16 @@ def slot_semantic_type(slot: NativeCallSlotPolicy) -> models.SemanticType | None ) ), ) - for slot in sorted(slots, key=lambda item: item.native_position) - ) + argument = argument_policies_by_name.get(slot.python_name or "") + if ( + argument is not None + and argument.array is not None + and argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + ): + return replace(policy, source_spelling="CFI_cdesc_t *", pointer_depth=1) + return policy + + parameters = tuple(parameter_policy(slot) for slot in sorted(slots, key=lambda item: item.native_position)) direct_result = next((result for result in results if result.source_kind == "direct_return"), None) result_source = source_abi.get("result") if isinstance(source_abi.get("result"), dict) else None result = ( @@ -2731,16 +2777,26 @@ def _direct_scalar_supported(argument: ArgumentPolicy) -> bool: def _direct_array_supported(argument: ArgumentPolicy) -> bool: - """Return whether one explicit or assumed-size array can use its C pointer.""" + """Return whether one ordinary array has the direct ABI its declaration names.""" + array = argument.array return bool( - argument.rank > 0 - and ( + ( argument.semantic_type_name in _PLAN_PRIMITIVE_SCALAR_TYPES or (argument.semantic_type_name == "String" and argument.character_length == 1) ) and argument.handoff_mode is ArgumentHandoffMode.ARRAY_BUFFER - and argument.array is not None - and argument.array.category in {"explicit_shape", "assumed_size"} + and array is not None + and ( + ( + array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS + and array.category in {"explicit_shape", "assumed_size"} + and argument.entrypoint_passing is EntrypointPassingConvention.POINTER_REFERENCE + ) + or ( + array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + and argument.entrypoint_passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER + ) + ) ) @@ -2981,7 +3037,10 @@ def _argument_policy( ) optional_mode = _optional_mode(argument, decision) callback = _callback_handoff_policy(argument) - array_policy = _array_handoff_policy(argument.semantic_type) + array_policy = _array_handoff_policy( + argument.semantic_type, + source_language=function.origin.source_language, + ) transformations, transformation_blockers = _argument_transformation_policies( argument, decision, @@ -3017,6 +3076,7 @@ def _argument_policy( function, argument, boundary, + array_policy, native_slot, callback, ) @@ -3024,6 +3084,7 @@ def _argument_policy( function, argument, boundary, + array_policy, native_slot, ) blockers = _completed_argument_blockers( @@ -3082,7 +3143,12 @@ def _argument_policy( character_length=_character_length(argument.semantic_type), character_local=_character_local_policy(argument.semantic_type, decision), array=array_policy, - native_array_actual=_native_array_actual_policy(argument, decision, array_policy), + native_array_actual=_native_array_actual_policy( + argument, + decision, + array_policy, + function.origin.source_language, + ), native_array_handle=_native_array_handle_wrapper_policy( argument.semantic_type, argument.metadata.get(models.RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA), @@ -3385,7 +3451,10 @@ def _direct_result_policy(context: _FunctionPolicyContext) -> _ResultPolicyCandi bridge_data_action=bridge_data_action, bridge_copy_reason=bridge_copy_reason, character_length=_character_length(return_type), - array=_array_handoff_policy(return_type), + array=_array_handoff_policy( + return_type, + source_language=context.function.origin.source_language, + ), native_array_handle=direct_handle, scalar_descriptor=scalar_descriptor, derived=derived, @@ -3607,7 +3676,10 @@ def _hidden_result_candidate( bridge_data_action=bridge_data_action, bridge_copy_reason=bridge_copy_reason, character_length=_character_length(argument.semantic_type), - array=_array_handoff_policy(argument.semantic_type), + array=_array_handoff_policy( + argument.semantic_type, + source_language=context.function.origin.source_language, + ), source_kind="hidden_output", python_returned=not argument.metadata.get(models.HIDDEN_NATIVE_OUTPUT_METADATA), native_name=mapping.native_name or argument.name, @@ -3731,6 +3803,7 @@ def _projected_native_call_slot_policy( python_position, visible_arguments, derived_types, + source_language=function.origin.source_language, ) @@ -3821,6 +3894,8 @@ def _projected_argument_native_call_slot_policy( python_position: int | None, visible_arguments: tuple[models.SemanticArgument, ...], derived_types: Mapping[tuple[str, str], DerivedTypePolicy], + *, + source_language: str | None, ) -> tuple[NativeCallSlotPolicy | None, int | None, tuple[str, ...]]: """Complete one Python argument projection after checking its position.""" if python_position is None: @@ -3842,6 +3917,7 @@ def _projected_argument_native_call_slot_policy( native_position, python_position, derived_types, + source_language=source_language, ) return slot, python_position, blockers @@ -3854,6 +3930,8 @@ def _projected_argument_slot( native_position: int, python_position: int, derived_types: Mapping[tuple[str, str], DerivedTypePolicy], + *, + source_language: str | None, ) -> tuple[NativeCallSlotPolicy, tuple[str, ...]]: """Construct one completed native slot for a visible projected argument.""" argument_path = f"{owner_path}.{argument.name}" @@ -3902,7 +3980,10 @@ def _projected_argument_slot( result_position=mapping.result_position, semantic_type_name=argument.semantic_type.name, character_length=_character_length(argument.semantic_type), - array=_array_handoff_policy(argument.semantic_type), + array=_array_handoff_policy( + argument.semantic_type, + source_language=source_language, + ), native_array_handle=_native_array_handle_wrapper_policy( argument.semantic_type, argument.metadata.get(models.RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA), @@ -4005,7 +4086,10 @@ def _hidden_result_native_call_slot_policy( result_position=mapping.result_position, semantic_type_name=argument.semantic_type.name, character_length=_character_length(argument.semantic_type), - array=_array_handoff_policy(argument.semantic_type), + array=_array_handoff_policy( + argument.semantic_type, + source_language=function.origin.source_language, + ), native_array_handle=_native_array_handle_wrapper_policy( argument.semantic_type, argument.metadata.get(models.RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA), @@ -4058,7 +4142,10 @@ def _hidden_result_native_call_slot_policy( result_position=mapping.result_position, semantic_type_name=argument.semantic_type.name, character_length=_character_length(argument.semantic_type), - array=_array_handoff_policy(argument.semantic_type), + array=_array_handoff_policy( + argument.semantic_type, + source_language=function.origin.source_language, + ), native_array_handle=_native_array_handle_wrapper_policy( argument.semantic_type, argument.metadata.get(models.RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA), @@ -4239,7 +4326,10 @@ def _implicit_native_call_slot_policies( array_copy_out=array_copy_out, semantic_type_name=argument.semantic_type.name, character_length=_character_length(argument.semantic_type), - array=_array_handoff_policy(argument.semantic_type), + array=_array_handoff_policy( + argument.semantic_type, + source_language=function.origin.source_language, + ), native_array_handle=_native_array_handle_wrapper_policy( argument.semantic_type, argument.metadata.get(models.RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA), @@ -5364,6 +5454,13 @@ def _runtime_semantic_validation_blockers( blockers.append(f"{label} has no runtime validators for semantic constraints {constraints}") if coercions: blockers.append(f"{label} has no wrapper conversion actions for semantic coercions {coercions}") + if semantic_type.metadata.get("fortran_protected"): + # Every generated accessor for a module array either allocates, + # deallocates or passes the variable to a dummy the callee may define. + # PROTECTED forbids all three outside the declaring module, so the + # bridge is refused here rather than emitted and left to fail in the + # Fortran compiler. + blockers.append(f"{label} is PROTECTED, so a generated accessor cannot define it outside its module") return tuple(blockers) @@ -6169,7 +6266,7 @@ def _native_array_handle_wrapper_policy( ) handle_kind = _native_array_enum(NativeArrayHandleKind, completed.handle_kind, owner_path, "handle kind") handoff = NativeDescriptorHandoffPolicy( - abi=_native_descriptor_handoff_abi(handle_kind, output_projection), + abi=_native_descriptor_handoff_abi(handle_kind), rank=int(semantic_type.rank or 0), optional_presence=completed.optional_absent, ) @@ -6179,26 +6276,19 @@ def _native_array_handle_wrapper_policy( owner_path, "descriptor interop", ) + descriptor_inquiries = completed.descriptor_inquiries operations = { _native_array_enum(NativeArrayOperation, item, owner_path, "operation") for item in completed.operations } - operations.update( - { - NativeArrayOperation.SHAPE, - NativeArrayOperation.ARRAY_ACTUAL, - NativeArrayOperation.DESCRIPTOR, - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, - } - ) + # Shape is what every handle is asked for. Everything an argument needs + # is read from the live descriptor in the binding, so no operation + # reports it; only a pointer still reports one, because a pointer that has + # no storage of its own has nowhere else to record what it was pointed at. + operations.add(NativeArrayOperation.SHAPE) + if descriptor == "pointer" and descriptor_inquiries: + operations.update({NativeArrayOperation.CONTIGUOUS, NativeArrayOperation.DESCRIPTOR}) if semantic_type.name == "String": operations.add(NativeArrayOperation.ELEMENT_LENGTH) - if semantic_type.metadata.get("fortran_character_length") == ":": - operations.difference_update({NativeArrayOperation.ALLOCATE, NativeArrayOperation.RESIZE}) - if descriptor == "pointer": - operations.add(NativeArrayOperation.CONTIGUOUS) if completed.destroy_behavior == NativeArrayDestroyBehavior.HANDLE_FINALIZER.value: operations.add(NativeArrayOperation.DESTROY) array = _array_handoff_policy(semantic_type) @@ -6211,6 +6301,12 @@ def _native_array_handle_wrapper_policy( owner_path, "descriptor kind", ), + descriptor_attribute=_native_array_enum( + NativeArrayDescriptorAttribute, + completed.descriptor_attribute, + owner_path, + "descriptor attribute", + ), handle_kind=handle_kind, origin=_native_array_enum(NativeArrayHandleOrigin, completed.origin, owner_path, "origin"), owner=_native_array_enum(OwnershipOwner, completed.owner, owner_path, "owner"), @@ -6257,6 +6353,7 @@ def _native_array_handle_wrapper_policy( "extraction action", ), descriptor_interop=interop, + descriptor_inquiries=descriptor_inquiries, nullable=completed.nullable, optional_absent=completed.optional_absent, storage_mode=_native_array_enum(StorageMode, completed.storage_mode, owner_path, "storage mode"), @@ -6312,13 +6409,9 @@ def _native_array_default_handle_policy( if operation in { NativeArrayOperation.SHAPE, - NativeArrayOperation.ARRAY_ACTUAL, NativeArrayOperation.DESCRIPTOR, - NativeArrayOperation.NATIVE_BYTE_ORDER, - NativeArrayOperation.ALIGNED, - NativeArrayOperation.WRITEABLE, - NativeArrayOperation.LAYOUT, NativeArrayOperation.CONTIGUOUS, + NativeArrayOperation.ELEMENT_LENGTH, } ) return NativeArrayDefaultHandlePolicy( @@ -6340,16 +6433,20 @@ def _native_array_default_handle_policy( ) -def _native_descriptor_handoff_abi( - handle_kind: NativeArrayHandleKind, - output_projection: NativeArrayOutputProjection, -) -> NativeDescriptorHandoffABI: - """Select one descriptor ABI from completed handle/result policy.""" +def _native_descriptor_handoff_abi(handle_kind: NativeArrayHandleKind) -> NativeDescriptorHandoffABI: + """Select one descriptor ABI from completed handle/result policy. + + Every descriptor a call receives is the Fortran runtime's to build: the + binding is handed one rather than establishing or filling a record of its + own. That holds for a pointer as much as an allocatable, and for an + optional argument as much as a required one, and it is what lets a callee + change an allocation or an association and have the caller's entity see it. + A result is the one exception, because there is no caller entity yet: the + wrapper owns storage the callee allocates into. + """ if handle_kind is NativeArrayHandleKind.OWNED_RESULT_DESCRIPTOR: return NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE - if output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE: - return NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR - return NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL + return NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR def _native_array_enum(enum_type, value: object, owner_path: str, label: str): @@ -6495,35 +6592,51 @@ def _native_array_assignment(value: str, owner_path: str) -> AssignmentMode: return _native_array_enum(AssignmentMode, value, owner_path, "native setter") +def _native_array_actual_dtype(argument: models.SemanticArgument) -> str | None: + """Return the NumPy dtype a handle must carry to stand in for this actual. + + A character actual is matched on its declared width as well as its kind, + because a handle whose elements are a different length describes different + storage. An assumed width is read from the live descriptor. + """ + if argument.semantic_type.name == "String": + length = _character_length(argument.semantic_type) + return "S" if length is None else f"S{length}" + return _NUMPY_DTYPE_NAMES.get(argument.semantic_type.name) + + def _native_array_actual_policy( argument: models.SemanticArgument, decision: OwnershipDecision, array: ArrayHandoffPolicy | None, + source_language: str | None = None, ) -> NativeArrayActualPolicy | None: - """Complete handle-as-array-actual acceptance for the Phase 6 buffer ABI.""" + """Complete handle-as-array-actual acceptance for the Phase 6 buffer ABI. + + A handle stands for storage a Fortran runtime owns, so a C dummy does not + accept one: there it is an object of the wrong type, like any other value + that is not an array. + """ if native_array_descriptor_kind(argument.semantic_type) is not None: return None if ( array is None or array.native_order != array.order - or array.rank is None - or argument.optional - or argument.semantic_type.name == "String" or decision.transfer is TransferMode.COPY_RETURN or decision.python_barrier_action is not PythonBarrierAction.ARRAY_STORAGE or decision.native_barrier_action is not NativeBarrierAction.PASS_ARRAY_BUFFER ): return None - try: - dtype = _NUMPY_DTYPE_NAMES[argument.semantic_type.name] - except KeyError: + dtype = _native_array_actual_dtype(argument) + if dtype is None: return None + handle_sources = ( + () + if source_language == "c" + else (NativeArraySourceKind.ALLOCATABLE_HANDLE, NativeArraySourceKind.POINTER_HANDLE) + ) return NativeArrayActualPolicy( - accepted_sources=( - NativeArraySourceKind.NDARRAY, - NativeArraySourceKind.ALLOCATABLE_HANDLE, - NativeArraySourceKind.POINTER_HANDLE, - ), + accepted_sources=(NativeArraySourceKind.NDARRAY, *handle_sources), dtype=dtype, rank=array.rank, shape=array.shape, @@ -6547,6 +6660,8 @@ def _native_array_module_variable_blockers( blockers = [] if variable.visibility != "public": blockers.append("native array module variable is not public") + # A descriptor handle hands out the same element-for-element view a fixed + # array does, so it carries the same width requirement. if handle.handle_kind is not NativeArrayHandleKind.BORROWED_MODULE_DESCRIPTOR: blockers.append(f"native array module handle kind {handle.handle_kind.value!r} is unsupported") if getter is None or getter.is_blocked or getter.kind is not ObjectKind.NUMPY_ARRAY: @@ -7162,17 +7277,12 @@ def _array_logical_argument_abi( semantic_type = argument.semantic_type if not is_boolean_semantic_type_name(semantic_type.name) or int(semantic_type.rank or 0) <= 0: return ArrayLogicalABI.NOT_APPLICABLE, None, False, False - source_type = _fortran_logical_native_type(argument) - if source_type is None: - if semantic_type.name in {"Bool", "Bool8"}: - return ArrayLogicalABI.C_BOOL_VIEW, "logical(c_bool)", False, False - copy_in = bool(getattr(argument, "_source_reads_argument", True)) - return ArrayLogicalABI.NATIVE_KIND_COPY, None, copy_in, decision.mutates_native - if "".join(source_type.casefold().split()) == "logical(kind=c_bool)": - return ArrayLogicalABI.C_BOOL_VIEW, "logical(c_bool)", False, False - copy_in = bool(getattr(argument, "_source_reads_argument", True)) - copy_out = decision.mutates_native - return ArrayLogicalABI.NATIVE_KIND_COPY, source_type, copy_in, copy_out + # The buffer is a NumPy integer of the element's own width, so the native + # pointer describes the caller's storage exactly and no directional copy is + # required for any logical kind. + # A spelling the source did not record is left unset; backend lowering then + # resolves the width from the semantic type itself. + return ArrayLogicalABI.C_BOOL_VIEW, _fortran_logical_native_type(argument), False, False def _logical_argument_bridge_action( @@ -7342,21 +7452,24 @@ def _array_writeback_abi( ) -> ArrayWritebackABI: """Complete mutable ordinary-array byte normalization before planning. - Exact-kind logical copies canonicalize bytes while copying out, so only a - direct ``c_bool`` view needs the separate low-bit normalization pass. + A Boolean array needs no more than any other kind. Its elements already + hold the zero or one a C ``_Bool`` is defined to hold, because the compiler + profiles request the option that guarantees it, so there is nothing left to + reduce. Reducing anyway could not help a translation unit built without + that option either: such a compiler represents false as the complement of + true, which no test applied here could tell from a true value. """ + del logical_abi if array is None or handoff_mode is not ArgumentHandoffMode.ARRAY_BUFFER or not decision.mutates_native: return ArrayWritebackABI.NOT_APPLICABLE - if is_boolean_semantic_type_name(semantic_type.name): - return ( - ArrayWritebackABI.LOGICAL_LOW_BIT_INT8 - if logical_abi is ArrayLogicalABI.C_BOOL_VIEW - else ArrayWritebackABI.NOT_APPLICABLE - ) return ArrayWritebackABI.NATIVE_ARRAY -def _array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandoffPolicy | None: +def _array_handoff_policy( + semantic_type: models.SemanticType, + *, + source_language: str | None = None, +) -> ArrayHandoffPolicy | None: """Copy structured buffer or raw-pointee facts into completed wrapper policy.""" if _is_raw_array_address_type(semantic_type): return _raw_array_handoff_policy(semantic_type) @@ -7376,6 +7489,12 @@ def _array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandoffPol minimum_rank, maximum_rank = _array_handoff_rank_bounds(rank, array.category, flatten_python_storage) order = _array_handoff_order(array.order, array.category) contiguous = _array_handoff_contiguous(array.contiguous, array.category) + entrypoint_abi = _array_entrypoint_abi( + array.category, + character=semantic_type.name == "String", + source_language=source_language, + ) + signed_strides = _array_handoff_signed_strides(entrypoint_abi, contiguous) return ArrayHandoffPolicy( rank=rank, shape=shape, @@ -7383,7 +7502,9 @@ def _array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandoffPol order=order, native_order=_array_handoff_native_order(array.order, array.copy_order, array.category), contiguous=contiguous, - python_layout=_array_handoff_python_layout(order, contiguous, rank), + python_layout=_array_handoff_python_layout(order, contiguous, rank, signed_strides), + entrypoint_abi=entrypoint_abi, + signed_strides=signed_strides, minimum_rank=minimum_rank, maximum_rank=maximum_rank, flatten_python_storage=flatten_python_storage, @@ -7429,30 +7550,80 @@ def _array_handoff_native_order( def _array_handoff_contiguous(contiguous: bool | None, category: str | None) -> bool | None: - """Complete the contiguity a contract asserts, leaving C runtime rank open. + """Complete the contiguity and section layout a contract asserts. ``None`` states that the contract asserts nothing about layout: the caller's - own strides reach the native call. Fortran assumed rank keeps its contiguous - descriptor default, and every C ``T[...]`` that did not spell ``Contiguous`` - stays stride-agnostic. + own strides reach the native call. A Fortran assumed-rank dummy receives a + descriptor, but a NumPy actual still has to be representable as a Fortran + array section, so it uses the same stride-aware layout as assumed shape. + Every C ``T[...]`` that did not spell ``Contiguous`` stays stride-agnostic. """ if contiguous is not None: return contiguous - if category in {SCALAR_STORAGE_CATEGORY, "assumed_rank"}: + if category == SCALAR_STORAGE_CATEGORY: return True + if category == "assumed_rank": + return False return None +def _array_entrypoint_abi( + category: str | None, + *, + character: bool, + source_language: str | None, +) -> ArrayEntrypointABI: + """Complete how one array dummy is reached, by asking the direct question. + + A ``bind(C)`` procedure with no bridge receives the address of the first + element and nothing more for an explicit-shape or assumed-size dummy, and + for a raw C pointer, whose rank the contract may leave to run time but + which is still only ever an address: the declaration already says how to + read what is there, so nothing is conveyed beside it. Every other form -- assumed-shape, + deferred-shape, assumed-rank, and a contract that names no Fortran category + because a bridge dummy will be generated for it -- is reached through a + ``CFI_cdesc_t *``, which carries an extent and a signed byte stride per + axis. + """ + if source_language == "c": + return ArrayEntrypointABI.RAW_ADDRESS + if character: + # GNU Fortran currently loses elem_len when CFI_section constructs a + # character view. Keep character arrays on the address-and-width ABI so + # every supported compiler observes the correct element length. + return ArrayEntrypointABI.RAW_ADDRESS + if category in {"explicit_shape", "assumed_size", "raw_address", "runtime_rank", SCALAR_STORAGE_CATEGORY}: + return ArrayEntrypointABI.RAW_ADDRESS + return ArrayEntrypointABI.C_DESCRIPTOR + + +def _array_handoff_signed_strides( + entrypoint_abi: ArrayEntrypointABI, + contiguous: bool | None, +) -> bool: + """Complete whether an axis of the actual may run backwards. + + Two things have to hold. The entrypoint must carry a descriptor, because a + bare address says nothing about which way an axis runs. And the dummy must + not require contiguous storage, which a reversed axis is not -- a + ``CONTIGUOUS`` dummy keeps its requirement whatever its calling convention + carries. + """ + return entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR and contiguous is not True + + def _array_handoff_python_layout( order: str | None, contiguous: bool | None, rank: int | None, + signed_strides: bool = False, ) -> ArrayPythonLayout: """Select the layout every accepted Python array actual must already have.""" if contiguous is None: return ArrayPythonLayout.ANY_STRIDED if contiguous is False: - return ArrayPythonLayout.POSITIVE_STRIDED_F + # The section rules are the same either way; only the sign is at stake. + return ArrayPythonLayout.SIGNED_STRIDED_F if signed_strides else ArrayPythonLayout.POSITIVE_STRIDED_F if order == "ORDER_C": return ArrayPythonLayout.C_CONTIGUOUS if order == "ORDER_F" or (rank is not None and rank > 1): @@ -7573,6 +7744,9 @@ def _raw_array_handoff_policy(semantic_type: models.SemanticType) -> ArrayHandof native_order=order, contiguous=True, python_layout=_array_handoff_python_layout(order, True, rank), + # A raw C address is the whole ABI here: nothing conveys a stride. + entrypoint_abi=ArrayEntrypointABI.RAW_ADDRESS, + signed_strides=False, minimum_rank=rank, maximum_rank=rank, itemsize=_character_length(semantic_type) if semantic_type.name == "String" else None, diff --git a/prik/policy/models.py b/prik/policy/models.py index 345682357..fe4cfae05 100644 --- a/prik/policy/models.py +++ b/prik/policy/models.py @@ -137,15 +137,42 @@ class ArrayPythonLayout(str, Enum): Policy selects the constraint; a backend only enforces it. ``ANY_STRIDED`` states that the contract constrains neither ordering nor contiguity, so the caller's own strides reach the native call unchanged. + + ``SIGNED_STRIDED_F`` is ``POSITIVE_STRIDED_F`` with the sign requirement + lifted: the axes must still be a Fortran array section -- ordered by + magnitude of stride, each a whole number of elements, none overlapping + another -- because that is what can be described to Fortran, but an axis + may run backwards. It is selected only where the entrypoint carries a + descriptor, since an address alone cannot say which way an axis runs. """ ANY_CONTIGUOUS = "any_contiguous" C_CONTIGUOUS = "c_contiguous" F_CONTIGUOUS = "f_contiguous" POSITIVE_STRIDED_F = "positive_strided_f" + SIGNED_STRIDED_F = "signed_strided_f" ANY_STRIDED = "any_strided" +class ArrayEntrypointABI(str, Enum): + """Completed shape in which one array actual reaches its native dummy. + + This is the answer to the project's direct-entrypoint question for arrays: + what would a ``bind(C)`` procedure with no bridge receive? An + ``assumed_shape`` or ``assumed_rank`` dummy is interoperable and receives a + ``CFI_cdesc_t *``, which carries a signed byte stride per axis. An + ``explicit_shape`` or ``assumed_size`` dummy receives the address of its + first element and nothing else, so nothing about its layout can be + conveyed, and only the layout the dummy already assumes is acceptable. + + A bridge implements whichever of these the direct route would have used; + it does not get to pick. + """ + + RAW_ADDRESS = "raw_address" + C_DESCRIPTOR = "c_descriptor" + + class ArgumentHandoffMode(str, Enum): """Completed binding-to-bridge ABI shape for one argument.""" @@ -311,6 +338,24 @@ class ModuleGetterAction(str, Enum): DERIVED_OBJECT = "derived_object" +class ModuleArrayAddressMechanism(str, Enum): + """Completed native mechanism that yields a fixed module array's base address. + + ``TARGET_ADDRESS`` applies to storage the declaration made addressable, where + ``c_loc`` names the array directly. ``CAPTURED_ADDRESS`` applies to an + ordinary array without that attribute: ``c_loc`` cannot name it, so the whole + array is handed to ``prik_capture_address``, a ``bind(C)`` primitive whose + assumed-type assumed-size dummy receives the bare base address. The + Fortran side forms no pointer and claims no target. The captured address is + valid for as long as the module variable keeps its storage, which the Fortran + standard does not guarantee across the program's lifetime; see the module + variable guide for the responsibility that carries. + """ + + TARGET_ADDRESS = "target_address" + CAPTURED_ADDRESS = "captured_address" + + class ModuleObjectAccessMechanism(str, Enum): """Completed native access path for one derived module value.""" @@ -725,6 +770,14 @@ class NativeArrayDescriptorKind(str, Enum): POINTER = "pointer" +class NativeArrayDescriptorAttribute(str, Enum): + """Attribute carried by the descriptor a handle backend supplies.""" + + ALLOCATABLE = "allocatable" + POINTER = "pointer" + OTHER = "other" + + class CharacterLocalRelease(str, Enum): """Completed release responsibility for one adapter-local character value. @@ -751,7 +804,6 @@ class NativeArrayHandleKind(str, Enum): class NativeDescriptorHandoffABI(str, Enum): """Binding-to-bridge descriptor representation.""" - FACT_PACKED_CALL_LOCAL = "fact_packed_call_local" DIRECT_STANDARD_DESCRIPTOR = "direct_standard_descriptor" OWNED_RESULT_STORAGE = "owned_result_storage" @@ -760,7 +812,6 @@ class NativeArrayDefaultConstruction(str, Enum): """Completed storage path for a runtime-constructed empty descriptor.""" NONE = "none" - FACT_PACKED_EMPTY = "fact_packed_empty" LAZY_OWNED_DESCRIPTOR = "lazy_owned_descriptor" @@ -864,13 +915,8 @@ class NativeArrayOperation(str, Enum): ASSOCIATED = "associated" SHAPE = "shape" ELEMENT_LENGTH = "element_length" - ARRAY_ACTUAL = "array_actual" DESCRIPTOR = "descriptor" TO_NUMPY = "to_numpy" - NATIVE_BYTE_ORDER = "native_byte_order" - ALIGNED = "aligned" - WRITEABLE = "writeable" - LAYOUT = "layout" CONTIGUOUS = "contiguous" ALLOCATE = "allocate" DEALLOCATE = "deallocate" @@ -936,6 +982,7 @@ class ModuleVariablePolicy: blockers: tuple[str, ...] = () character_length: int | None = None array: ArrayHandoffPolicy | None = None + array_address: ModuleArrayAddressMechanism | None = None native_array_handle: NativeArrayHandleWrapperPolicy | None = None derived: DerivedModuleObjectPolicy | None = None @@ -965,6 +1012,12 @@ class ArrayHandoffPolicy: native_order: str | None contiguous: bool | None python_layout: ArrayPythonLayout + # How this dummy is reached, and therefore what can be said about layout. + entrypoint_abi: ArrayEntrypointABI + # Whether an axis of the actual may run backwards. Only a descriptor can + # carry that, and only a dummy that does not require contiguous storage can + # accept it. + signed_strides: bool minimum_rank: int maximum_rank: int flatten_python_storage: bool = False @@ -1057,7 +1110,7 @@ class NativeArrayActualPolicy: accepted_sources: tuple[NativeArraySourceKind, ...] dtype: str - rank: int + rank: int | None shape: tuple[str, ...] order: str | None writable: bool @@ -1093,6 +1146,7 @@ class NativeArrayHandleWrapperPolicy: """Typed wrapper-facing projection of completed native handle policy.""" descriptor_kind: NativeArrayDescriptorKind + descriptor_attribute: NativeArrayDescriptorAttribute handle_kind: NativeArrayHandleKind origin: NativeArrayHandleOrigin owner: OwnershipOwner @@ -1109,6 +1163,14 @@ class NativeArrayHandleWrapperPolicy: destroy_behavior: NativeArrayDestroyBehavior extraction_action: NativeArrayExtractionAction descriptor_interop: NativeArrayDescriptorInterop + # A handle answers its inquiries -- shape, state, element width, + # contiguity, the NumPy view -- from the live descriptor its entry point + # supplies, unless its declaration cannot cross a bind(C) descriptor + # interface at all. A deferred-length character pointer is that case: the + # standard does not allow such a dummy in a bind(C) interface, and GNU + # Fortran mistranslates the descriptor rather than rejecting it, so those + # inquiries stay on generated Fortran procedures of their own. + descriptor_inquiries: bool nullable: bool optional_absent: bool storage_mode: StorageMode @@ -1411,6 +1473,8 @@ class FunctionWrapperPolicy: native_order="F", contiguous=True, python_layout=ArrayPythonLayout.F_CONTIGUOUS, + entrypoint_abi=ArrayEntrypointABI.RAW_ADDRESS, + signed_strides=False, minimum_rank=2, maximum_rank=2, ) diff --git a/prik/policy/native_array_handles.py b/prik/policy/native_array_handles.py index 8c7a4326c..cba8c9465 100644 --- a/prik/policy/native_array_handles.py +++ b/prik/policy/native_array_handles.py @@ -37,6 +37,7 @@ class NativeArrayHandlePolicy: """Completed post-IR policy for a native allocatable or pointer array handle.""" descriptor_kind: str + descriptor_attribute: str handle_kind: str origin: str owner: str @@ -53,6 +54,7 @@ class NativeArrayHandlePolicy: destroy_behavior: str to_numpy: str descriptor_interop: str + descriptor_inquiries: bool nullable: bool optional_absent: bool storage_mode: str @@ -387,6 +389,7 @@ def _variable_native_array_policy( mark_native_array_handle(example_type, "pointer") example_policy = NativeArrayHandlePolicy( descriptor_kind="pointer", + descriptor_attribute="pointer", handle_kind="pointer", origin="module", owner="native", @@ -403,6 +406,7 @@ def _variable_native_array_policy( destroy_behavior="nullify", to_numpy="borrowed_view", descriptor_interop="pointer_c_descriptor", + descriptor_inquiries=True, nullable=True, optional_absent=False, storage_mode="alias", diff --git a/prik/runtime/handles.py b/prik/runtime/handles.py index 458ec150e..ade6797f3 100644 --- a/prik/runtime/handles.py +++ b/prik/runtime/handles.py @@ -4,18 +4,31 @@ import ctypes import operator -from collections.abc import Callable, Mapping, Sequence +from collections.abc import Callable, Iterable, Sequence from contextlib import suppress -from dataclasses import dataclass from typing import Any import numpy as np -HandleOperation = Callable[..., Any] +HandleDispatcher = Callable[[str, tuple[Any, ...]], Any] _PRESENT_NATIVE_ARRAY_DESCRIPTOR_ARGUMENT = ctypes.c_int(1) _PRESENT_NATIVE_ARRAY_DESCRIPTOR_ARGUMENT_ADDRESS = ctypes.addressof(_PRESENT_NATIVE_ARRAY_DESCRIPTOR_ARGUMENT) +# What a pointer reports about the target it is associated with, and what +# another pointer is given to reproduce that association: +# +# (base address, element width, rank, then per axis lower bound, extent, +# stride in bytes) +# +# A generated handle reads these out of its live descriptor in C and a +# generated association writes them back into one, so they are exchanged in +# this flat form and never assembled into a descriptor in Python. A contract +# handle that has no native storage yet keeps its association here until it +# is given some. +_DESCRIPTOR_FACT_HEADER = 3 +_DESCRIPTOR_FACTS_PER_AXIS = 3 + class _OwnerRetainedNDArray(np.ndarray): """Internal ndarray view carrying a strong reference to native owner state.""" @@ -28,93 +41,84 @@ def _retain_numpy_owner(value: np.ndarray, owner: Any) -> np.ndarray: return retained -@dataclass(frozen=True) -class _NativeArrayHandoff: - """Internal opaque native pointer handoff returned by generated handle ops.""" - - address: int - owner: Any = None - - def __post_init__(self) -> None: - if isinstance(self.address, bool) or not isinstance(self.address, int): - raise TypeError("native array handoff address must be an integer") - if self.address <= 0: - raise ValueError("native array handoff address must be a non-null positive pointer value") +def _descriptor_facts(value: Any, rank: int) -> tuple[int, ...]: + """Validate one flat descriptor-fact tuple reported for a handle.""" + expected = _DESCRIPTOR_FACT_HEADER + _DESCRIPTOR_FACTS_PER_AXIS * int(rank) + try: + facts = tuple(operator.index(field) for field in value) + except TypeError: + raise TypeError("native array descriptor facts must be a sequence of integers") from None + if len(facts) != expected: + raise ValueError(f"native array descriptor facts must hold {expected} values; received {len(facts)}") + if facts[0] < 0: + raise ValueError(f"native array descriptor base address must be non-negative; received {facts[0]}") + if facts[2] != int(rank): + raise ValueError(f"native array descriptor rank {facts[2]} does not match declared rank {int(rank)}") + return facts + + +def _descriptor_facts_shape_and_strides(facts: tuple[int, ...]) -> tuple[tuple[int, ...], tuple[int, ...]]: + """Split one fact tuple into its extents and byte strides.""" + axes = range(facts[2]) + offsets = tuple(_DESCRIPTOR_FACT_HEADER + _DESCRIPTOR_FACTS_PER_AXIS * axis for axis in axes) + return ( + tuple(facts[offset + 1] for offset in offsets), + tuple(facts[offset + 2] for offset in offsets), + ) -@dataclass(frozen=True) -class _NativeArrayDescriptorHandoff: - """Internal opaque handoff for one versioned native-handle capsule.""" +def _empty_descriptor_facts(dtype: Any, rank: int) -> tuple[int, ...]: + """Return the facts of storage that is not there. - capsule: Any + Every axis is empty, so the bounds describe nothing and no claim is made + about an array that does not exist. + """ + itemsize = np.dtype(dtype).itemsize + return (0, itemsize, int(rank), *((0, 0, itemsize) * int(rank))) - def __post_init__(self) -> None: - if self.capsule is None: - raise TypeError("native array descriptor handoff capsule is required") +def _numpy_view_from_descriptor_facts(facts: tuple[int, ...], dtype: Any) -> np.ndarray | None: + """Build a NumPy view over the storage one fact tuple describes. -def _numpy_view_from_pointer_c_descriptor( - descriptor: Any, - *, - dtype: Any, - expected_rank: int | None = None, -) -> np.ndarray | None: - """Build a NumPy view from generated TS 29113 pointer descriptor fields.""" - base_addr = _pointer_descriptor_base_addr(descriptor) - if base_addr == 0: + Only a handle with no native storage of its own takes this route; every + generated handle builds its view in C, where the descriptor is. + """ + if facts[0] == 0: return None array_dtype = np.dtype(dtype) - _validate_pointer_descriptor_itemsize(descriptor, array_dtype) - shape, strides = _pointer_descriptor_shape_and_strides(descriptor) - if expected_rank is not None and len(shape) != int(expected_rank): + if facts[1] != array_dtype.itemsize: raise ValueError( - f"pointer descriptor rank {len(shape)} does not match declared handle rank {int(expected_rank)}" + f"native array element width {facts[1]} does not match NumPy dtype itemsize {array_dtype.itemsize}" ) - buffer_offset, buffer_nbytes, view_offset = _descriptor_view_buffer_window( - tuple(shape), - tuple(strides), - array_dtype.itemsize, - ) - buffer_addr = base_addr + buffer_offset - if buffer_addr < 0: - raise ValueError("pointer descriptor view buffer starts before address zero") - buffer_type = ctypes.c_char * buffer_nbytes - buffer = buffer_type.from_address(buffer_addr) - return np.ndarray(tuple(shape), dtype=array_dtype, buffer=buffer, strides=tuple(strides), offset=view_offset) + shape, strides = _descriptor_facts_shape_and_strides(facts) + offset, nbytes, view_offset = _descriptor_view_buffer_window(shape, strides, array_dtype.itemsize) + start = facts[0] + offset + if start < 0: + raise ValueError("native array view buffer starts before address zero") + buffer = (ctypes.c_char * nbytes).from_address(start) + return np.ndarray(shape, dtype=array_dtype, buffer=buffer, strides=strides, offset=view_offset) -def _native_array_handle_from_generated_ops( +def _native_array_handle_from_generated_dispatch( descriptor_kind: str, dtype: Any, rank: int, - ops: Mapping[str, HandleOperation], + invoke: HandleDispatcher, + capabilities: Iterable[str], owner: Any = None, descriptor_ownership: str = "borrowed", to_numpy_policy: str = "borrowed_view", + native_backend: Any = None, generation: int | None = None, ) -> NativeArrayHandleBase: - """Build a runtime handle from generated operation callables.""" + """Build a runtime handle from one generated operation dispatcher. + + ``native_backend`` is the capsule publishing the entity's native backend, + which a binding reads directly to reach the descriptor. It is carried, not + required: a handle created from a contract has none until it is given storage. + """ owned = descriptor_ownership == "owned" - normalized_ops = {} - for name, operation in ops.items(): - if name == "array_actual": - normalized = _generated_handoff_operation(operation, owner=owner, pass_owner=owned) - elif name == "descriptor" and owned: - normalized = _generated_owned_descriptor_operation(operation, owner) - elif name in {"shape", "to_numpy"} and owned: - normalized = _generated_owned_descriptor_record_operation(operation, owner) - elif name == "associate": - normalized = _generated_pointer_associate_operation( - operation, - owner=owner if owned else None, - ) - elif name in {"allocate", "resize"}: - normalized = _generated_shape_operation(operation, owner=owner if owned else None) - elif owned: - normalized = _generated_owned_handle_operation(operation, owner) - else: - normalized = _generated_handle_operation(operation) - normalized_ops[name] = normalized + normalized_capabilities = frozenset(capabilities) try: handle_cls = { "allocatable": AllocatableArray, @@ -123,19 +127,22 @@ def _native_array_handle_from_generated_ops( except KeyError: raise ValueError("generated native array handle kind must be 'allocatable' or 'pointer'") from None try: - return handle_cls( + handle = handle_cls( dtype=dtype, rank=rank, - ops=normalized_ops, + invoke=invoke, + capabilities=normalized_capabilities, owner=owner, descriptor_ownership=descriptor_ownership, to_numpy_policy=to_numpy_policy, generation=generation, ) + handle._native_backend = native_backend + return handle except BaseException: - if owned and "destroy" in normalized_ops: + if owned and "destroy" in normalized_capabilities: with suppress(Exception): - normalized_ops["destroy"](None) + invoke("destroy", (owner,) if owner is not None else ()) raise @@ -144,65 +151,61 @@ def _native_array_handle_from_contract( dtype: Any, rank: int, ) -> NativeArrayHandleBase: - """Create one owned, initially empty descriptor handle from a contract.""" - descriptor_state = { - "record": _empty_descriptor_record(dtype, rank), - "owner": None, - } + """Create one owned, initially empty descriptor handle from a contract. + + Such a handle has no native storage until a call gives it some, so it + answers from a fact tuple of its own instead of from a descriptor. That is + also where a pointer association taken before it was ever bound is kept, + so the association can be replayed once storage arrives. + """ + state: dict[str, Any] = {"facts": _empty_descriptor_facts(dtype, rank), "source": None} - def current_shape(_handle: NativeArrayHandleBase) -> tuple[int, ...] | None: - record = descriptor_state["record"] - if _pointer_descriptor_base_addr(record) == 0: + def current_shape() -> tuple[int, ...] | None: + if state["facts"][0] == 0: return None - shape, _strides = _pointer_descriptor_shape_and_strides(record) + shape, _strides = _descriptor_facts_shape_and_strides(state["facts"]) return shape - def current_array_actual(_handle: NativeArrayHandleBase) -> _NativeArrayHandoff | None: - address = _pointer_descriptor_base_addr(descriptor_state["record"]) - return _NativeArrayHandoff(address, owner=descriptor_state["owner"]) + def descriptor() -> tuple[int, ...]: + return state["facts"] - def descriptor(_handle: NativeArrayHandleBase) -> Mapping[str, Any]: - return descriptor_state["record"] + def present() -> bool: + return state["facts"][0] != 0 - def present(_handle: NativeArrayHandleBase) -> bool: - return _pointer_descriptor_base_addr(descriptor_state["record"]) != 0 + def current_view() -> np.ndarray | None: + return _numpy_view_from_descriptor_facts(state["facts"], dtype) - def current_view(_handle: NativeArrayHandleBase) -> np.ndarray | None: - return _numpy_view_from_pointer_c_descriptor( - descriptor_state["record"], - dtype=dtype, - expected_rank=rank, - ) + def clear() -> None: + state["facts"] = _empty_descriptor_facts(dtype, rank) + state["source"] = None - def clear(_handle: NativeArrayHandleBase) -> None: - descriptor_state["record"] = _empty_descriptor_record(dtype, rank) - descriptor_state["owner"] = None - - def associate_record( - _handle: NativeArrayHandleBase, - record: Mapping[str, Any], - owner: NativeArrayHandleBase, + def associate_facts( + facts: tuple[int, ...], + source: NativeArrayHandleBase, ) -> None: - descriptor_state["record"] = _copy_pointer_descriptor_record(record) - descriptor_state["owner"] = owner + state["facts"] = facts + state["source"] = source - common_ops = { + operations = { "shape": current_shape, - "array_actual": current_array_actual, "descriptor": descriptor, "to_numpy": current_view, "destroy": clear, + "allocated": present, + "associated": present, + "nullify": clear, + "_associate_facts": associate_facts, } + + def dispatch(operation: str, args: tuple[Any, ...]) -> Any: + return operations[operation](*args) + try: - handle_cls, descriptor_ops = { - "allocatable": (AllocatableArray, {"allocated": present}), + handle_cls, capabilities = { + "allocatable": (AllocatableArray, {"allocated", "shape", "descriptor", "to_numpy", "destroy"}), "pointer": ( PointerArray, - { - "associated": present, - "nullify": clear, - "_associate_record": associate_record, - }, + {"associated", "shape", "descriptor", "to_numpy", "destroy", "nullify", "_associate_facts"}, ), }[descriptor_kind] except KeyError: @@ -210,7 +213,8 @@ def associate_record( handle = handle_cls( dtype=dtype, rank=rank, - ops={**common_ops, **descriptor_ops}, + invoke=dispatch, + capabilities=capabilities, descriptor_ownership="owned", to_numpy_policy="borrowed_view", ) @@ -218,66 +222,29 @@ def associate_record( return handle -def _empty_descriptor_record(dtype: Any, rank: int) -> dict[str, Any]: - """Return canonical unallocated or unassociated descriptor facts.""" - array_dtype = np.dtype(dtype) - return { - "base_addr": 0, - "elem_len": array_dtype.itemsize, - "rank": int(rank), - "dim": [{"lower_bound": 0, "extent": 0, "sm": array_dtype.itemsize} for _axis in range(int(rank))], - } - - -def _copy_pointer_descriptor_record(descriptor: Mapping[str, Any]) -> dict[str, Any]: - """Copy validated standard descriptor facts for independent association state.""" - dimensions = _pointer_descriptor_dimensions(descriptor) - return { - "base_addr": _required_descriptor_int(descriptor, "base_addr"), - "elem_len": _required_descriptor_int(descriptor, "elem_len"), - "rank": _required_descriptor_int(descriptor, "rank"), - "dim": [ - { - "lower_bound": _required_descriptor_int(dimension, "lower_bound", field_owner=f"dim[{index}]"), - "extent": _required_descriptor_int(dimension, "extent", field_owner=f"dim[{index}]"), - "sm": _required_descriptor_int(dimension, "sm", field_owner=f"dim[{index}]"), - } - for index, dimension in enumerate(dimensions) - ], - } - - -def _pointer_descriptor_record_facts(descriptor: Mapping[str, Any]) -> tuple[int, ...]: - """Flatten standard descriptor facts for one generated association operation.""" - record = _copy_pointer_descriptor_record(descriptor) - fields = [ - record["base_addr"], - record["elem_len"], - record["rank"], - ] - for dimension in record["dim"]: - fields.extend( - ( - dimension["lower_bound"], - dimension["extent"], - dimension["sm"], - ) - ) - return tuple(fields) - - def _bind_contract_native_array_handle( handle: NativeArrayHandleBase, descriptor_kind: str, dtype: Any, rank: int, - ops: Mapping[str, HandleOperation], + invoke: HandleDispatcher, + capabilities: Iterable[str], owner: Any, descriptor_ownership: str, - to_numpy_policy: str, + to_numpy_policy: str | None, generation: int | None = None, + native_backend: Any = None, ) -> None: - """Attach generated persistent descriptor storage to a contract handle.""" + """Attach generated persistent descriptor storage to a contract handle. + + ``to_numpy_policy`` is ``None`` when the argument that supplied the storage + does not project a result. Such an argument gives the handle a descriptor + to hand over, but it does not define what the handle exposes, so the + handle keeps the exposure it was created with. + + ``native_backend`` is the backend over the attached storage, which every + later call reads directly from C. + """ if not isinstance(handle, NativeArrayHandleBase) or not handle._contract_default: raise TypeError("generated descriptor storage can attach only to a fresh contract handle") if handle.closed: @@ -288,239 +255,35 @@ def _bind_contract_native_array_handle( raise ValueError(f"{descriptor_kind} handle rank {handle.rank} does not match generated rank {int(rank)}") if not handle._dtype_matches(dtype): raise TypeError(f"{descriptor_kind} handle dtype {handle.dtype!r} does not match generated dtype {dtype!r}") - pending_pointer_descriptor = ( - handle._association_descriptor_record() if isinstance(handle, PointerArray) and handle.associated else None - ) - generated = _native_array_handle_from_generated_ops( + # An association taken before there was anywhere native to record it is + # replayed onto the storage that just arrived. + pending = handle._call_operation("descriptor") if isinstance(handle, PointerArray) and handle.associated else None + generated = _native_array_handle_from_generated_dispatch( descriptor_kind, dtype, rank, - ops, + invoke, + capabilities, owner=owner, descriptor_ownership=descriptor_ownership, - to_numpy_policy=to_numpy_policy, + to_numpy_policy=handle._to_numpy_policy if to_numpy_policy is None else to_numpy_policy, + native_backend=native_backend, generation=generation, ) - handle._ops = generated._ops + handle._invoke = generated._invoke + handle._capabilities = generated._capabilities handle._owner = generated._owner handle._descriptor_ownership = generated._descriptor_ownership handle._to_numpy_policy = generated._to_numpy_policy handle._generation = generated._generation + # The storage just attached is the wrapper's own and lives as long as the + # handle, so the handle can publish it the way a module array publishes + # its entity. Later calls then reach it from C without coming back here. + handle._native_backend = generated._native_backend handle._contract_default = False generated._closed = True - if pending_pointer_descriptor is not None: - handle._call_op("associate", pending_pointer_descriptor) - - -def _generated_handle_operation(operation: HandleOperation) -> HandleOperation: - """Adapt a generated operation callable to the handle operation protocol.""" - - def call(_handle: NativeArrayHandleBase, *args: Any) -> Any: - return operation(*args) - - return call - - -def _generated_owned_handle_operation(operation: HandleOperation, owner: Any) -> HandleOperation: - """Adapt an operation whose first argument is persistent native owner storage.""" - - def call(_handle: NativeArrayHandleBase, *args: Any) -> Any: - return operation(owner, *args) - - return call - - -def _generated_owned_descriptor_operation(operation: HandleOperation, owner: Any) -> HandleOperation: - """Adapt an owned standard-descriptor pointer operation to typed handoff.""" - - def call(_handle: NativeArrayHandleBase, *args: Any) -> _NativeArrayDescriptorHandoff: - value = operation(owner, *args) - return _native_array_descriptor_handoff_from_generated_result(value, owner=owner) - - return call - - -def _generated_owned_descriptor_record_operation(operation: HandleOperation, owner: Any) -> HandleOperation: - """Adapt owned descriptor facts and normalize compiler zero-extent sentinels.""" - - def call(_handle: NativeArrayHandleBase, *args: Any) -> Any: - value = operation(owner, *args) - if not isinstance(value, Mapping): - return value - dimensions = value.get("dim") - if not isinstance(dimensions, Sequence): - return value - normalized_dimensions = [] - for dimension in dimensions: - if not isinstance(dimension, Mapping) or dimension.get("extent") != -1: - normalized_dimensions.append(dimension) - continue - normalized_dimensions.append({**dimension, "extent": 0}) - return {**value, "dim": normalized_dimensions} - - return call - - -def _generated_pointer_associate_operation( - operation: HandleOperation, - *, - owner: Any = None, -) -> HandleOperation: - """Adapt pointer association to one generated standard-descriptor operation.""" - - def call(_handle: NativeArrayHandleBase, descriptor: Mapping[str, Any]) -> Any: - facts = _pointer_descriptor_record_facts(descriptor) - if owner is None: - return operation(facts) - return operation(owner, facts) - - return call - - -def _generated_shape_operation(operation: HandleOperation, *, owner: Any = None) -> HandleOperation: - """Adapt generated shape operations from one runtime shape tuple to scalar extents.""" - - def call(_handle: NativeArrayHandleBase, shape: Sequence[int]) -> Any: - extents = tuple(np.int64(extent) for extent in shape) - return operation(*extents) if owner is None else operation(owner, *extents) - - return call - - -def _generated_handoff_operation( - operation: HandleOperation, - *, - owner: Any, - pass_owner: bool = False, -) -> HandleOperation: - """Adapt a generated pointer-address operation to the runtime handoff protocol.""" - - def call(_handle: NativeArrayHandleBase, *args: Any) -> _NativeArrayHandoff: - value = operation(owner, *args) if pass_owner else operation(*args) - return _native_array_handoff_from_generated_result(value, owner=owner) - - return call - - -def _native_array_handoff_from_generated_result(value: Any, *, owner: Any = None) -> _NativeArrayHandoff: - """Normalize a generated pointer operation result into a native handoff.""" - if isinstance(value, _NativeArrayHandoff): - return value - if isinstance(value, ctypes.c_void_p): - value = value.value - if isinstance(value, bool) or not isinstance(value, int): - raise TypeError(f"generated native array handoff address must be an integer; received {type(value).__name__}") - return _NativeArrayHandoff(value, owner=owner) - - -def _native_array_descriptor_handoff_from_generated_result( - value: Any, - *, - owner: Any = None, -) -> _NativeArrayDescriptorHandoff: - """Normalize a generated native-handle capsule into a typed handoff.""" - if isinstance(value, _NativeArrayDescriptorHandoff): - return value - if owner is None: - raise TypeError("generated native array descriptor handoff requires an owner capsule") - if value is not owner: - raise TypeError("generated native array descriptor operation must return its owner capsule") - return _NativeArrayDescriptorHandoff(owner) - - -def _pointer_descriptor_base_addr(descriptor: Any) -> int: - base_addr = _required_descriptor_int(descriptor, "base_addr") - if base_addr < 0: - raise ValueError(f"pointer descriptor base_addr must be non-negative; received {base_addr}") - return base_addr - - -def _validate_pointer_descriptor_itemsize(descriptor: Any, array_dtype: np.dtype) -> None: - elem_len = _required_descriptor_int(descriptor, "elem_len") - if elem_len != array_dtype.itemsize: - raise ValueError( - f"pointer descriptor elem_len {elem_len} does not match NumPy dtype itemsize {array_dtype.itemsize}" - ) - - -def _pointer_descriptor_shape_and_strides(descriptor: Any) -> tuple[list[int], list[int]]: - dimensions = _pointer_descriptor_dimensions(descriptor) - shape: list[int] = [] - strides: list[int] = [] - for index, dimension in enumerate(dimensions): - extent, stride = _pointer_descriptor_dimension_extent_stride(index, dimension) - shape.append(extent) - strides.append(stride) - return shape, strides - - -def _pointer_descriptor_dimensions(descriptor: Any) -> Sequence[Any]: - rank = _pointer_descriptor_rank(descriptor) - dimensions = _required_descriptor_field(descriptor, "dim") - if not isinstance(dimensions, Sequence) or isinstance(dimensions, (str, bytes)): - raise TypeError("pointer descriptor field 'dim' must be a sequence of dimension records") - if rank != len(dimensions): - raise ValueError(f"pointer descriptor rank {rank} does not match {len(dimensions)} dimension records") - return dimensions - - -def _pointer_descriptor_rank(descriptor: Any) -> int: - rank = _required_descriptor_int(descriptor, "rank") - if rank < 0: - raise ValueError(f"pointer descriptor rank must be non-negative; received {rank}") - return rank - - -def _pointer_descriptor_dimension_extent_stride(index: int, dimension: Any) -> tuple[int, int]: - if not _is_pointer_descriptor_dimension_record(dimension): - raise TypeError(f"pointer descriptor dimension {index} must be a mapping or field-record object") - _required_descriptor_int(dimension, "lower_bound", field_owner=f"dim[{index}]") - extent = _required_descriptor_int(dimension, "extent", field_owner=f"dim[{index}]") - stride = _required_descriptor_int(dimension, "sm", field_owner=f"dim[{index}]") - if extent < 0: - raise ValueError(f"pointer descriptor dim[{index}].extent must be non-negative; received {extent}") - return extent, stride - - -def _required_descriptor_int( - descriptor: Any, - field: str, - *, - field_owner: str = "descriptor", -) -> int: - value = _required_descriptor_field(descriptor, field, field_owner=field_owner) - if isinstance(value, bool) or not isinstance(value, int): - raise TypeError(f"pointer {field_owner} field {field!r} must be an integer") - return value - - -def _required_descriptor_field( - record: Any, - field: str, - *, - field_owner: str = "descriptor", -) -> Any: - if isinstance(record, Mapping): - try: - return record[field] - except KeyError: - pass - else: - missing = object() - value = getattr(record, field, missing) - if value is not missing: - return value - raise TypeError(f"pointer {field_owner} field {field!r} is required") - - -def _is_pointer_descriptor_record(value: Any) -> bool: - fields = ("base_addr", "elem_len", "rank", "dim") - return isinstance(value, Mapping) or all(hasattr(value, field) for field in fields) - - -def _is_pointer_descriptor_dimension_record(value: Any) -> bool: - fields = ("lower_bound", "extent", "sm") - return isinstance(value, Mapping) or all(hasattr(value, field) for field in fields) + if pending is not None: + handle._call_operation("associate", pending) def _descriptor_view_buffer_window( @@ -539,9 +302,9 @@ def _descriptor_view_buffer_window( class NativeArrayHandleBase: - """Shared runtime state and operation dispatch for native array handles.""" + """Shared runtime state and single-call dispatch for native array handles.""" - _REQUIRED_DESCRIPTOR_OPS: frozenset[str] = frozenset() + _REQUIRED_CAPABILITIES: frozenset[str] = frozenset() _VALID_DESCRIPTOR_KINDS = frozenset({"allocatable", "pointer"}) _VALID_DESCRIPTOR_OWNERSHIP = frozenset({"borrowed", "owned"}) _VALID_TO_NUMPY_POLICIES = frozenset( @@ -558,7 +321,8 @@ def __init__( *, dtype: Any, rank: int, - ops: Mapping[str, HandleOperation], + invoke: HandleDispatcher, + capabilities: Iterable[str], owner: Any = None, descriptor_kind: str, descriptor_ownership: str, @@ -578,14 +342,19 @@ def __init__( ) self._dtype = None if dtype is None else np.dtype(dtype) self._rank = int(rank) - self._ops = self._normalize_ops(ops) + if not callable(invoke): + raise TypeError(f"native array handle dispatcher must be callable; received {type(invoke).__name__}") + self._invoke: HandleDispatcher | None = invoke + self._capabilities = self._normalize_capabilities(capabilities) self._owner = owner self._descriptor_kind = descriptor_kind self._descriptor_ownership = descriptor_ownership self._to_numpy_policy = to_numpy_policy self._generation = generation + # Optional capsule publishing this entity's native descriptor backend. + self._native_backend: Any = None self._contract_default = False - self._validate_required_ops() + self._validate_required_capabilities() self._closed = False @property @@ -596,20 +365,10 @@ def dtype(self) -> np.dtype: def _deferred_character_dtype(self) -> np.dtype: """Resolve one deferred character width from generated native state.""" - if "element_length" in self._ops: - length = operator.index(self._call_op("element_length")) - if length < 0: - raise ValueError("native character array element length must be non-negative") - return np.dtype(f"S{length}") - value = self._call_op("to_numpy") - if isinstance(value, np.ndarray) and value.dtype.kind == "S": - return value.dtype - if _is_pointer_descriptor_record(value): - length = _required_descriptor_int(value, "elem_len") - if length < 0: - raise ValueError("native character array element length must be non-negative") - return np.dtype(f"S{length}") - raise TypeError("deferred character handle cannot resolve its runtime element length") + length = operator.index(self._call_operation("element_length")) + if length < 0: + raise ValueError("native character array element length must be non-negative") + return np.dtype(f"S{length}") @property def rank(self) -> int: @@ -617,13 +376,15 @@ def rank(self) -> int: @property def shape(self) -> tuple[int, ...] | None: - if self._to_numpy_absent_state(): - return None - shape = self._call_op("shape") + """Return current extents, or ``None`` when the storage is not there. + + The inquiry reads the descriptor itself, so it reports absence rather + than being asked about it first, and the extents it returns are the + ones the compiler recorded. + """ + shape = self._call_operation("shape") if shape is None: return None - if _is_pointer_descriptor_record(shape): - shape, _strides = _pointer_descriptor_shape_and_strides(shape) normalized = self._normalize_shape(shape) if len(normalized) != self.rank: raise ValueError( @@ -667,177 +428,64 @@ def close(self) -> Any: """Release generated owner storage for an owned native descriptor handle.""" if self.closed or not self.owned: return None - operation = self._ops["destroy"] + invoke = self._invoke + if invoke is None: + return None try: - return operation(self) + return self._call_operation("destroy") finally: self._closed = True self._owner = None - self._ops = {} + self._native_backend = None + self._invoke = None + self._capabilities = frozenset() def __del__(self) -> None: with suppress(Exception): self.close() - def _array_actual_for_binding( - self, - *, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - expected_layout: str | None = None, - require_writeable: bool = False, - require_native_byte_order: bool = False, - require_aligned: bool = False, - require_contiguous: bool = False, - ) -> Any: - """Return the generated native array actual after validating handle state.""" - self._validate_array_actual_state() - if expected_rank is not None and self.rank != int(expected_rank): - raise ValueError( - f"{self.descriptor_kind} handle rank {self.rank} does not match expected rank {int(expected_rank)}" - ) - if expected_dtype is not None and not self._dtype_matches(expected_dtype): - raise TypeError( - f"{self.descriptor_kind} handle dtype {self.dtype!r} does not match expected dtype {expected_dtype!r}" - ) - shape = self.shape - if shape is None: - raise ValueError(f"{self.descriptor_kind} handle has no valid array actual") - self._validate_expected_shape(shape, expected_shape) - self._validate_expected_layout(expected_layout) - self._validate_writeable(require_writeable) - self._validate_native_byte_order(require_native_byte_order) - self._validate_aligned(require_aligned) - self._validate_contiguous(require_contiguous) - return self._required_handoff_result("array_actual", self._call_op("array_actual")) - - def _descriptor_for_binding( - self, - *, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - ) -> Any: - """Return the generated native descriptor after validating handle metadata.""" - if expected_rank is not None and self.rank != int(expected_rank): - raise ValueError( - f"{self.descriptor_kind} handle rank {self.rank} does not match expected rank {int(expected_rank)}" - ) - if expected_dtype is not None and not self._dtype_matches(expected_dtype): - raise TypeError( - f"{self.descriptor_kind} handle dtype {self.dtype!r} does not match expected dtype {expected_dtype!r}" - ) - shape = self.shape - if shape is not None: - self._validate_expected_shape(shape, expected_shape) - descriptor = self._call_op("descriptor") - if isinstance(descriptor, _NativeArrayDescriptorHandoff): - return descriptor - if _is_pointer_descriptor_record(descriptor): - if _pointer_descriptor_base_addr(descriptor) == 0: - return self._contiguous_descriptor_record(0, None) - _validate_pointer_descriptor_itemsize(descriptor, np.dtype(self.dtype)) - descriptor_shape, _ = _pointer_descriptor_shape_and_strides(descriptor) - if len(descriptor_shape) != self.rank: - raise ValueError( - f"{self.descriptor_kind} descriptor rank {len(descriptor_shape)} " - f"does not match declared rank {self.rank}" - ) - return descriptor - address = descriptor.address if isinstance(descriptor, _NativeArrayHandoff) else descriptor - if isinstance(address, ctypes.c_void_p): - address = address.value or 0 - if isinstance(address, bool) or not isinstance(address, int): - raise TypeError( - f"{self.descriptor_kind} handle descriptor operation must return descriptor fields " - f"or an integer data address; received {type(descriptor).__name__}" - ) - return self._contiguous_descriptor_record(address, shape) - - def _descriptor_record_for_binding(self) -> Any: - """Return standard descriptor fields for a fact-packed descriptor call.""" - descriptor = self._call_op("to_numpy") - if not _is_pointer_descriptor_record(descriptor): - raise TypeError( - f"{self.descriptor_kind} handle cannot expose standard descriptor fields for binding handoff" - ) - return descriptor - - def _contiguous_descriptor_record(self, address: int, shape: tuple[int, ...] | None) -> dict[str, Any]: - """Build standard descriptor fields for a contiguous native array actual.""" - dtype = np.dtype(self.dtype) - extents = (0,) * self.rank if shape is None else shape - strides = [] - stride = dtype.itemsize - for extent in extents: - strides.append(stride) - stride *= max(int(extent), 1) - return { - "base_addr": int(address), - "elem_len": dtype.itemsize, - "rank": self.rank, - "dim": [ - {"lower_bound": 0, "extent": int(extent), "sm": int(axis_stride)} - for extent, axis_stride in zip(extents, strides, strict=True) - ], - } - def to_numpy(self) -> Any: - """Return a live view of current native storage, or ``None``.""" - if self._to_numpy_absent_state(): - return None - if self.to_numpy_policy == "unsupported": + """Return a live view of current native storage, or ``None``. + + The extraction reads the descriptor itself, so it reports absent + storage rather than being asked about it first, and it builds the view + from the declared element type over the storage the descriptor names. + """ + policy = self._to_numpy_policy + if policy == "unsupported": + # There is nothing to expose either way when the storage is not + # there, so absence is reported before the refusal. + if not self._present(): + return None raise NotImplementedError( f"{self.descriptor_kind} handle to_numpy extraction is unsupported by completed policy" ) - if ( - self.to_numpy_policy == "contiguous_view" - and "contiguous" in self._ops - and not bool(self._call_op("contiguous")) - ): - raise ValueError(f"{self.descriptor_kind} handle to_numpy target must be contiguous") - value = self._call_op("to_numpy") + value = self._call_operation("to_numpy") if value is None: - raise TypeError( - f"{self.descriptor_kind} handle to_numpy operation returned None for present descriptor state" - ) - if _is_pointer_descriptor_record(value): - value = _numpy_view_from_pointer_c_descriptor(value, dtype=self.dtype, expected_rank=self.rank) - if value is None: - raise TypeError( - f"{self.descriptor_kind} handle extraction returned a null descriptor for present descriptor state" - ) - value = _retain_numpy_owner(value, self) + return None self._validate_numpy_result(value) - if self.to_numpy_policy == "contiguous_view": + if self.owned and value.base is not None and value.base is self._owner: + # The view was built over storage this handle releases when it is + # finalized, so the view has to keep the handle alive too, not just + # the record that holds the storage. + value = _retain_numpy_owner(value, self) + if policy == "contiguous_view": self._validate_contiguous_numpy_result(value) return value - def _call_op(self, name: str, *args: Any) -> Any: + def _call_operation(self, name: str, *args: Any) -> Any: if self.closed: raise ReferenceError(f"{self.descriptor_kind} handle is closed") - try: - operation = self._ops[name] - except KeyError: + if name not in self._capabilities: raise NotImplementedError(f"{self.descriptor_kind} handle operation {name!r} is not available") from None - return operation(self, *args) - - def _required_handoff_result(self, operation: str, value: Any) -> Any: - if not isinstance(value, _NativeArrayHandoff): - raise TypeError( - f"{self.descriptor_kind} handle {operation} operation must return a native handoff object; " - f"received {type(value).__name__}" - ) - return value - - def _validate_array_actual_state(self) -> None: - """Validate descriptor-specific presence before native array-actual handoff.""" - raise NotImplementedError(f"{self.descriptor_kind} handle array-actual validation is not available") - - def _to_numpy_absent_state(self) -> bool: - """Return whether descriptor state makes extraction produce ``None``.""" - return False + invoke = self._invoke + if invoke is None: + raise ReferenceError(f"{self.descriptor_kind} handle is closed") + if name in {"allocate", "resize"}: + args = tuple(np.int64(extent) for extent in args[0]) + if self.owned and self._owner is not None: + args = (self._owner, *args) + return invoke(name, args) def _validate_numpy_result(self, value: Any) -> None: if not isinstance(value, np.ndarray): @@ -860,108 +508,36 @@ def _validate_contiguous_numpy_result(self, value: np.ndarray) -> None: if not (value.flags.c_contiguous or value.flags.f_contiguous): raise ValueError(f"{self.descriptor_kind} handle to_numpy result must be contiguous") + def _present(self) -> bool: + """Report whether this handle currently stands for any storage.""" + return True + def _dtype_matches(self, expected_dtype: Any) -> bool: try: return np.dtype(self.dtype) == np.dtype(expected_dtype) except TypeError: return self.dtype == expected_dtype - def _validate_expected_shape( - self, - shape: tuple[int, ...], - expected_shape: Sequence[int | None] | int | None, - ) -> None: - """Validate a concrete expected shape before native array-actual handoff.""" - if expected_shape is None: - return - expected = self._normalize_expected_shape(expected_shape) - if len(expected) != len(shape): - raise ValueError( - f"{self.descriptor_kind} handle shape rank {len(shape)} does not match expected shape rank " - f"{len(expected)}" - ) - for axis, (actual, wanted) in enumerate(zip(shape, expected, strict=True)): - if wanted is not None and actual != wanted: - raise ValueError( - f"{self.descriptor_kind} handle shape {shape!r} does not match expected shape " - f"{expected!r} at axis {axis}" - ) - - def _validate_expected_layout(self, expected_layout: str | None) -> None: - """Validate a required native layout before native array-actual handoff.""" - if expected_layout is None: - return - required = self._normalize_expected_layout_name(expected_layout) - actual_layout = self._call_op("layout") - actual = self._normalize_actual_layout_name(actual_layout) - if actual != required: - raise ValueError( - f"{self.descriptor_kind} handle layout {actual_layout!r} does not match expected layout " - f"{expected_layout!r}" - ) - - def _validate_writeable(self, require_writeable: bool) -> None: - """Validate writeability before native array-actual handoff.""" - if not require_writeable: - return - if not bool(self._call_op("writeable")): - raise TypeError(f"{self.descriptor_kind} handle array actual must be writeable") - - def _validate_contiguous(self, require_contiguous: bool) -> None: - """Validate the data-buffer ABI's contiguous-storage requirement.""" - if not require_contiguous: - return - if "contiguous" in self._ops: - contiguous = bool(self._call_op("contiguous")) - elif "layout" in self._ops: - contiguous = self._normalize_actual_layout_name(self._call_op("layout")) in {"C", "F"} - else: - raise ValueError(f"{self.descriptor_kind} handle cannot prove contiguous array storage") - if not contiguous: - raise ValueError(f"{self.descriptor_kind} handle array actual must be contiguous") - - def _validate_native_byte_order(self, require_native_byte_order: bool) -> None: - """Validate native byte order before native array-actual handoff.""" - if not require_native_byte_order: - return - if not bool(self._call_op("native_byte_order")): - raise TypeError(f"{self.descriptor_kind} handle array actual must use native byte order") - - def _validate_aligned(self, require_aligned: bool) -> None: - """Validate native alignment before native array-actual handoff.""" - if not require_aligned: - return - if not bool(self._call_op("aligned")): - raise TypeError(f"{self.descriptor_kind} handle array actual must be aligned") - - def _validate_required_ops(self) -> None: - if "shape" not in self._ops: + def _validate_required_capabilities(self) -> None: + if "shape" not in self._capabilities: raise ValueError(f"{self.descriptor_kind} native array handle requires generated operation 'shape'") - if "array_actual" not in self._ops: - raise ValueError(f"{self.descriptor_kind} native array handle requires generated operation 'array_actual'") - if "descriptor" not in self._ops: - raise ValueError(f"{self.descriptor_kind} native array handle requires generated operation 'descriptor'") - for name in sorted(self._REQUIRED_DESCRIPTOR_OPS): - if name not in self._ops: + for name in sorted(self._REQUIRED_CAPABILITIES): + if name not in self._capabilities: raise ValueError(f"{self.descriptor_kind} native array handle requires generated operation {name!r}") - if self.to_numpy_policy != "unsupported" and "to_numpy" not in self._ops: + if self.to_numpy_policy != "unsupported" and "to_numpy" not in self._capabilities: raise ValueError( f"{self.descriptor_kind} native array handle with to_numpy_policy " f"{self.to_numpy_policy!r} requires generated operation 'to_numpy'" ) - if self.owned and "destroy" not in self._ops: + if self.owned and "destroy" not in self._capabilities: raise ValueError(f"{self.descriptor_kind} owned native array handle requires generated operation 'destroy'") @staticmethod - def _normalize_ops(ops: Mapping[str, HandleOperation]) -> dict[str, HandleOperation]: - normalized = dict(ops) - for name, operation in normalized.items(): + def _normalize_capabilities(capabilities: Iterable[str]) -> frozenset[str]: + normalized = frozenset(capabilities) + for name in normalized: if not isinstance(name, str): - raise TypeError(f"native array handle operation names must be strings; received {type(name).__name__}") - if not callable(operation): - raise TypeError( - f"native array handle operation {name!r} must be callable; received {type(operation).__name__}" - ) + raise TypeError(f"native array handle capability names must be strings; received {type(name).__name__}") return normalized @staticmethod @@ -975,45 +551,19 @@ def _normalize_shape(shape: Sequence[int] | int) -> tuple[int, ...]: raise ValueError(f"native array handle shape dimensions must be non-negative; received {normalized!r}") return normalized - @staticmethod - def _normalize_expected_shape(shape: Sequence[int | None] | int) -> tuple[int | None, ...]: - try: - normalized = (operator.index(shape),) - except TypeError: - normalized = tuple(None if dimension is None else int(dimension) for dimension in shape) - for dimension in normalized: - if dimension is not None and dimension < 0: - raise ValueError( - f"expected native array shape dimensions must be non-negative; received {normalized!r}" - ) - return normalized - - @staticmethod - def _normalize_expected_layout_name(layout: str) -> str: - required = layout.upper() - if required not in {"C", "F"}: - raise ValueError(f"unsupported expected NumPy array layout {layout!r}") - return required - - @staticmethod - def _normalize_actual_layout_name(layout: Any) -> str: - actual = str(layout).upper() - if actual not in {"C", "F"}: - raise ValueError(f"native array handle layout operation returned unsupported layout {layout!r}") - return actual - class AllocatableArray(NativeArrayHandleBase): """Runtime handle for a native allocatable array descriptor.""" - _REQUIRED_DESCRIPTOR_OPS = frozenset({"allocated"}) + _REQUIRED_CAPABILITIES = frozenset({"allocated"}) def __init__( self, *, dtype: Any, rank: int, - ops: Mapping[str, HandleOperation], + invoke: HandleDispatcher, + capabilities: Iterable[str], owner: Any = None, descriptor_ownership: str = "borrowed", to_numpy_policy: str = "borrowed_view", @@ -1022,7 +572,8 @@ def __init__( super().__init__( dtype=dtype, rank=rank, - ops=ops, + invoke=invoke, + capabilities=capabilities, owner=owner, descriptor_kind="allocatable", descriptor_ownership=descriptor_ownership, @@ -1032,33 +583,30 @@ def __init__( @property def allocated(self) -> bool: - return bool(self._call_op("allocated")) + return bool(self._call_operation("allocated")) - def _validate_array_actual_state(self) -> None: - if not self.allocated: - raise ValueError("allocatable handle is unallocated and cannot be passed as an array actual") - - def _to_numpy_absent_state(self) -> bool: - return not self.allocated + def _present(self) -> bool: + return self.allocated def deallocate(self) -> Any: - return self._call_op("deallocate") + return self._call_operation("deallocate") def resize(self, shape: Sequence[int] | int) -> Any: - return self._call_op("resize", self._normalize_shape(shape)) + return self._call_operation("resize", self._normalize_shape(shape)) class PointerArray(NativeArrayHandleBase): """Runtime handle for a native pointer array descriptor.""" - _REQUIRED_DESCRIPTOR_OPS = frozenset({"associated", "nullify"}) + _REQUIRED_CAPABILITIES = frozenset({"associated", "nullify"}) def __init__( self, *, dtype: Any, rank: int, - ops: Mapping[str, HandleOperation], + invoke: HandleDispatcher, + capabilities: Iterable[str], owner: Any = None, descriptor_ownership: str = "borrowed", to_numpy_policy: str = "borrowed_view", @@ -1067,7 +615,8 @@ def __init__( super().__init__( dtype=dtype, rank=rank, - ops=ops, + invoke=invoke, + capabilities=capabilities, owner=owner, descriptor_kind="pointer", descriptor_ownership=descriptor_ownership, @@ -1077,30 +626,23 @@ def __init__( @property def associated(self) -> bool: - return bool(self._call_op("associated")) + return bool(self._call_operation("associated")) - def _validate_array_actual_state(self) -> None: - if not self.associated: - raise ValueError("pointer handle is unassociated and cannot be passed as an array actual") - if "contiguous" in self._ops and not bool(self._call_op("contiguous")): - raise ValueError( - "pointer handle target is noncontiguous and cannot use the pointer/shape array-actual handoff" - ) + def _present(self) -> bool: + return self.associated - def _to_numpy_absent_state(self) -> bool: - return not self.associated + def _association_facts(self) -> tuple[int, ...]: + """Report what this pointer is associated with, as flat facts. - def _association_descriptor_record(self) -> dict[str, Any]: - """Return independent standard descriptor facts for pointer assignment.""" - descriptor = self._descriptor_for_binding( - expected_dtype=self.dtype, - expected_rank=self.rank, - ) - if isinstance(descriptor, _NativeArrayDescriptorHandoff): - descriptor = self._descriptor_record_for_binding() - record = _copy_pointer_descriptor_record(descriptor) - _validate_pointer_descriptor_itemsize(record, self.dtype) - return record + A pointer assignment copies the association as it stands now; it does + not make the target follow the source afterwards. Reading the facts + here is what makes that snapshot. + """ + facts = _descriptor_facts(self._call_operation("descriptor"), self.rank) + itemsize = np.dtype(self.dtype).itemsize + if facts[0] != 0 and facts[1] != itemsize: + raise ValueError(f"pointer handle element width {facts[1]} does not match NumPy dtype itemsize {itemsize}") + return facts def associate(self, other: PointerArray) -> Any: """Make this pointer's association match another pointer handle.""" @@ -1114,248 +656,42 @@ def associate(self, other: PointerArray) -> Any: raise ValueError(f"pointer handle rank {self.rank} does not match source rank {other.rank}") if not self._dtype_matches(other.dtype): raise TypeError(f"pointer handle dtype {self.dtype!r} does not match source dtype {other.dtype!r}") - descriptor = other._association_descriptor_record() + facts = other._association_facts() if self._contract_default: - return self._call_op("_associate_record", descriptor, other) - return self._call_op("associate", descriptor) + # No native storage yet: keep the association, and the handle it + # came from, until storage arrives and it can be replayed. + return self._call_operation("_associate_facts", facts, other) + return self._call_operation("associate", facts) def nullify(self) -> Any: - return self._call_op("nullify") + return self._call_operation("nullify") def allocate(self, shape: Sequence[int] | int) -> Any: - return self._call_op("allocate", self._normalize_shape(shape)) + return self._call_operation("allocate", self._normalize_shape(shape)) def deallocate(self) -> Any: - return self._call_op("deallocate") + return self._call_operation("deallocate") def resize(self, shape: Sequence[int] | int) -> Any: - return self._call_op("resize", self._normalize_shape(shape)) - - -def _native_array_actual_for_binding( - value: Any, - *, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - expected_layout: str | None = None, - require_writeable: bool = False, - require_native_byte_order: bool = False, - require_aligned: bool = False, - require_contiguous: bool = False, -) -> Any: - """Return an ndarray or generated native array actual for a normal array argument.""" - if isinstance(value, NativeArrayHandleBase): - return value._array_actual_for_binding( - expected_dtype=expected_dtype, - expected_rank=expected_rank, - expected_shape=expected_shape, - expected_layout=expected_layout, - require_writeable=require_writeable, - require_native_byte_order=require_native_byte_order, - require_aligned=require_aligned, - require_contiguous=require_contiguous, - ) - if isinstance(value, np.ndarray): - _validate_ndarray_array_actual( - value, - expected_dtype=expected_dtype, - expected_rank=expected_rank, - expected_shape=expected_shape, - expected_layout=expected_layout, - require_writeable=require_writeable, - require_native_byte_order=require_native_byte_order, - require_aligned=require_aligned, - require_contiguous=require_contiguous, - ) - return value - if value is None: - raise TypeError("normal array argument is required; received None") - raise TypeError(f"expected NumPy array or native array handle argument; received {type(value).__name__}") + return self._call_operation("resize", self._normalize_shape(shape)) -def _native_array_actual_argument_for_binding_positional( - value: Any, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - expected_layout: str | None = None, - require_writeable: bool = False, - require_native_byte_order: bool = False, - require_aligned: bool = False, - include_rank: bool = False, - include_itemsize: bool = False, - include_strides: bool = False, - require_contiguous: bool = False, - flatten_storage: bool = False, - flat_axis: int | None = None, -) -> tuple[int, ...]: - """Pack a normal array actual into generated Bind-C array descriptor fields.""" - strided_ndarray = include_strides and isinstance(value, np.ndarray) - if strided_ndarray: - _validate_ndarray_positive_strides(value) - actual = _native_array_actual_for_binding( - value, - expected_dtype=expected_dtype, - expected_rank=None if flatten_storage else expected_rank, - expected_shape=None if flatten_storage else expected_shape, - # Positive-stride validation below is the exact Fortran-order contract - # for a strided ndarray; NumPy's contiguous flag is intentionally false. - expected_layout=None if strided_ndarray else expected_layout, - require_writeable=bool(require_writeable), - require_native_byte_order=bool(require_native_byte_order), - require_aligned=bool(require_aligned), - require_contiguous=bool(require_contiguous), - ) - address, shape, itemsize = _normal_array_actual_abi_facts(value, actual, expected_dtype) - if flatten_storage: - shape = _flattened_storage_shape(shape, expected_shape, flat_axis) - fields = [address] - if include_rank: - fields.append(len(shape)) - if include_itemsize: - fields.append(itemsize) - fields.extend(shape) - if include_strides: - extents, upper_bounds, strides = _normal_array_actual_stride_facts(actual, shape, itemsize) - fields[-len(shape) :] = extents - fields.extend(upper_bounds) - fields.extend(strides) - return tuple(fields) - - -def _flattened_storage_shape( - shape: tuple[int, ...], - expected_shape: Sequence[int | None] | int | None, - flat_axis: int | None, -) -> tuple[int, ...]: - """Return native extents for a contiguous actual with one flat edge.""" - if not 1 <= len(shape) <= 15: - raise TypeError(f"Flat storage expects NumPy array rank 1 through 15; received rank {len(shape)}") - expected = ( - NativeArrayHandleBase._normalize_expected_shape(expected_shape) if expected_shape is not None else (None,) - ) - if len(shape) < len(expected): - raise TypeError(f"Flat storage expects NumPy array rank at least {len(expected)}; received rank {len(shape)}") - axis = 0 if flat_axis is None or int(flat_axis) < 0 else int(flat_axis) - if axis not in {0, len(expected) - 1}: - raise ValueError("Flat storage axis must be the first or final contract dimension") - if axis == 0: - return _leading_flattened_storage_shape(shape, expected) - return _final_flattened_storage_shape(shape, expected) - - -def _final_flattened_storage_shape(shape: tuple[int, ...], expected: tuple[int | None, ...]) -> tuple[int, ...]: - """Keep prefix extents and flatten all remaining axes into the final extent.""" - prefix_count = len(expected) - 1 - _validate_flat_expected_shape(shape[:prefix_count], expected[:prefix_count], offset=0) - return (*shape[:prefix_count], _extent_product(shape[prefix_count:])) - - -def _leading_flattened_storage_shape(shape: tuple[int, ...], expected: tuple[int | None, ...]) -> tuple[int, ...]: - """Flatten leading axes and keep suffix extents at the Python edge.""" - suffix_count = len(expected) - 1 - suffix_shape = shape[len(shape) - suffix_count :] if suffix_count else () - _validate_flat_expected_shape(suffix_shape, expected[1:], offset=len(shape) - suffix_count) - return (_extent_product(shape[: len(shape) - suffix_count]), *suffix_shape) - - -def _extent_product(shape: tuple[int, ...]) -> int: - """Return the element count covered by a flattened extent segment.""" - size = 1 - for extent in shape: - size *= int(extent) - return size - - -def _validate_flat_expected_shape( - actual: tuple[int, ...], - expected: tuple[int | None, ...], - *, - offset: int, -) -> None: - """Validate fixed non-flat dimensions for a flattened storage contract.""" - for axis, (actual_extent, wanted) in enumerate(zip(actual, expected, strict=True)): - if wanted is not None and actual_extent != wanted: - raise TypeError( - f"NumPy array has incompatible shape at axis {offset + axis}: " - f"received {actual!r}, expected {expected!r}" - ) - - -def _normal_array_actual_stride_facts( - actual: Any, - shape: tuple[int, ...], - itemsize: int, -) -> tuple[tuple[int, ...], tuple[int, ...], tuple[int, ...]]: - """Pack the positive-stride base extent and slice facts used by the bridge.""" - if isinstance(actual, _NativeArrayHandoff): - return shape, tuple(max(extent - 1, -1) for extent in shape), (1,) * len(shape) - if not isinstance(actual, np.ndarray): - raise TypeError(f"normal array actual operation returned unsupported value {type(actual).__name__}") - if actual.size == 0: - # NumPy may report zero strides for empty dimensions. No element can - # be addressed, so use the bridge's canonical empty-array facts. - return shape, tuple(max(extent - 1, -1) for extent in shape), (1,) * len(shape) - if any(stride <= 0 for stride in actual.strides): - raise ValueError("array actual strides must be positive") - - extents = [] - upper_bounds = [] - relative_strides = [] - base_product = 1 - element_strides = tuple(stride // itemsize for stride in actual.strides) - for axis, (logical_extent, element_stride) in enumerate(zip(shape, element_strides, strict=True)): - relative_stride = element_stride // base_product - relative_strides.append(relative_stride) - upper_bound = -1 if logical_extent == 0 else (logical_extent - 1) * relative_stride - upper_bounds.append(upper_bound) - base_extent = max(element_strides[axis + 1] // base_product, 1) if axis + 1 < len(shape) else upper_bound + 1 - extents.append(base_extent) - base_product *= base_extent - return tuple(extents), tuple(upper_bounds), tuple(relative_strides) - - -def _validate_ndarray_positive_strides(value: np.ndarray) -> None: - """Match the bridge's positive non-overlapping Fortran slice contract.""" - for axis, stride in enumerate(value.strides): - invalid = stride % value.itemsize != 0 or (value.size > 0 and value.shape[axis] > 1 and stride <= 0) - if axis: - invalid |= ( - value.size > 0 - and value.shape[axis - 1] > 0 - and (stride < value.strides[axis - 1] * value.shape[axis - 1]) - ) - if invalid: - raise TypeError("NumPy array actual has incompatible layout; expected ordering (F)") - - -def _normal_array_actual_abi_facts( - value: Any, - actual: Any, - expected_dtype: Any, -) -> tuple[int, tuple[int, ...], int]: - if isinstance(actual, _NativeArrayHandoff): - shape = value.shape - if shape is None: - raise ValueError("native array handle array actual must report shape for binding handoff") - dtype = expected_dtype if expected_dtype is not None else value.dtype - return actual.address, tuple(int(axis) for axis in shape), int(np.dtype(dtype).itemsize) - if isinstance(actual, np.ndarray): - return int(actual.ctypes.data), tuple(int(axis) for axis in actual.shape), int(actual.dtype.itemsize) - raise TypeError(f"normal array actual operation returned unsupported value {type(actual).__name__}") - - -def _native_array_descriptor_for_binding( +def _native_array_backend_for_binding( value: Any, *, descriptor_kind: str, expected_dtype: Any = None, expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - optional: bool = False, -) -> Any: - """Return a generated native descriptor for a handle-typed binding argument.""" + optional_absent: bool = False, + bind_default: Callable[..., Any] | None = None, +) -> tuple[Any | None, ...]: + """Return the backend capsule a descriptor argument hands over. + + Only a handle that has no storage yet reaches this: everything else + publishes a backend the binding reads without coming back here. Attaching + storage is what gives such a handle one, so the binder runs first and the + backend it published is what goes back. + """ try: expected_type = { "allocatable": AllocatableArray, @@ -1364,223 +700,54 @@ def _native_array_descriptor_for_binding( except KeyError: raise ValueError(f"unsupported native array descriptor kind: {descriptor_kind!r}") from None if value is None: - if optional: - return None + if optional_absent: + return (None, None) raise TypeError(f"{descriptor_kind} native array handle argument is required; received None") if not isinstance(value, expected_type): raise TypeError(f"expected {descriptor_kind} native array handle argument; received {type(value).__name__}") - return value._descriptor_for_binding( - expected_dtype=expected_dtype, - expected_rank=expected_rank, - expected_shape=expected_shape, - ) - - -def _native_array_descriptor_argument_for_binding( - value: Any, - *, - descriptor_kind: str, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - optional_absent: bool = False, -) -> tuple[Any, ...]: - """Pack standard descriptor fields for generated CPython binding code.""" - descriptor = _native_array_descriptor_for_binding( - value, - descriptor_kind=descriptor_kind, - expected_dtype=expected_dtype, - expected_rank=expected_rank, - expected_shape=expected_shape, - optional=optional_absent, - ) - if descriptor is None: - if expected_rank is None: - raise ValueError("optional absent native array descriptor arguments require an expected rank") - fields = (None,) * (3 + 3 * int(expected_rank)) - return (*fields, None) - if isinstance(descriptor, _NativeArrayDescriptorHandoff): - descriptor = value._descriptor_record_for_binding() - dimensions = _pointer_descriptor_dimensions(descriptor) - fields = [ - _required_descriptor_int(descriptor, "base_addr"), - _required_descriptor_int(descriptor, "elem_len"), - _required_descriptor_int(descriptor, "rank"), - ] - for index, dimension in enumerate(dimensions): - fields.extend( - [ - _required_descriptor_int(dimension, "lower_bound", field_owner=f"dim[{index}]"), - _required_descriptor_int(dimension, "extent", field_owner=f"dim[{index}]"), - _required_descriptor_int(dimension, "sm", field_owner=f"dim[{index}]"), - ] + if expected_rank is not None and value.rank != int(expected_rank): + raise ValueError( + f"{descriptor_kind} handle rank {value.rank} does not match expected rank {int(expected_rank)}" ) - if optional_absent: - fields.append(_PRESENT_NATIVE_ARRAY_DESCRIPTOR_ARGUMENT_ADDRESS) - return tuple(fields) - - -def _native_array_descriptor_argument_for_binding_positional( - value: Any, - descriptor_kind: str, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - optional_absent: bool = False, -) -> tuple[Any, ...]: - """Positional wrapper used by generated CPython binding code.""" - return _native_array_descriptor_argument_for_binding( - value, - descriptor_kind=str(descriptor_kind), - expected_dtype=None if expected_dtype is None else np.dtype(expected_dtype), - expected_rank=None if expected_rank is None else int(expected_rank), - expected_shape=expected_shape, - optional_absent=bool(optional_absent), - ) - - -def _native_array_descriptor_handoff_for_binding( - value: Any, - *, - descriptor_kind: str, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - optional_absent: bool = False, - bind_default: HandleOperation | None = None, -) -> tuple[Any | None, ...]: - """Pack a versioned native-handle capsule for projected descriptor mutation.""" - if isinstance(value, NativeArrayHandleBase) and value._contract_default: + if expected_dtype is not None and not value._dtype_matches(expected_dtype): + raise TypeError( + f"{descriptor_kind} handle dtype {value.dtype!r} does not match expected dtype {expected_dtype!r}" + ) + if value._contract_default: if bind_default is None: raise TypeError( f"writable {descriptor_kind} contract handle requires generated persistent descriptor storage" ) - _native_array_descriptor_for_binding( - value, - descriptor_kind=descriptor_kind, - expected_dtype=expected_dtype, - expected_rank=expected_rank, - expected_shape=expected_shape, - optional=optional_absent, - ) bind_default(value) - descriptor = _native_array_descriptor_for_binding( - value, - descriptor_kind=descriptor_kind, - expected_dtype=expected_dtype, - expected_rank=expected_rank, - expected_shape=expected_shape, - optional=optional_absent, - ) - if descriptor is None: - return (None, None) if optional_absent else (None,) - if not isinstance(descriptor, _NativeArrayDescriptorHandoff): + backend = value._native_backend + if backend is None: raise TypeError( - f"writable {descriptor_kind} descriptor argument requires a generated direct descriptor handoff" + f"writable {descriptor_kind} descriptor argument requires generated persistent descriptor storage" ) if optional_absent: - return descriptor.capsule, _PRESENT_NATIVE_ARRAY_DESCRIPTOR_ARGUMENT_ADDRESS - return (descriptor.capsule,) + return backend, _PRESENT_NATIVE_ARRAY_DESCRIPTOR_ARGUMENT_ADDRESS + return (backend,) -def _native_array_descriptor_handoff_for_binding_positional( +def _native_array_backend_for_binding_positional( value: Any, descriptor_kind: str, expected_dtype: Any = None, expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, optional_absent: bool = False, - bind_default: HandleOperation | None = None, + bind_default: Callable[..., Any] | None = None, ) -> tuple[Any | None, ...]: """Positional wrapper used by projected-handle CPython binding code.""" - return _native_array_descriptor_handoff_for_binding( + return _native_array_backend_for_binding( value, descriptor_kind=str(descriptor_kind), expected_dtype=None if expected_dtype is None else np.dtype(expected_dtype), expected_rank=None if expected_rank is None else int(expected_rank), - expected_shape=expected_shape, optional_absent=bool(optional_absent), bind_default=bind_default, ) -def _validate_ndarray_array_actual( - value: np.ndarray, - *, - expected_dtype: Any = None, - expected_rank: int | None = None, - expected_shape: Sequence[int | None] | int | None = None, - expected_layout: str | None = None, - require_writeable: bool = False, - require_native_byte_order: bool = False, - require_aligned: bool = False, - require_contiguous: bool = False, -) -> None: - _validate_ndarray_expected_rank(value, expected_rank) - _validate_ndarray_native_byte_order(value, require_native_byte_order) - _validate_ndarray_expected_dtype(value, expected_dtype) - _validate_ndarray_expected_shape(tuple(int(dimension) for dimension in value.shape), expected_shape) - _validate_ndarray_expected_layout(value, expected_layout) - _validate_ndarray_writeable(value, require_writeable) - _validate_ndarray_aligned(value, require_aligned) - _validate_ndarray_contiguous(value, require_contiguous) - - -def _validate_ndarray_expected_rank(value: np.ndarray, expected_rank: int | None) -> None: - if expected_rank is not None and value.ndim != int(expected_rank): - raise TypeError(f"NumPy array rank {value.ndim} does not match expected rank {int(expected_rank)}") - - -def _validate_ndarray_expected_dtype(value: np.ndarray, expected_dtype: Any) -> None: - if expected_dtype is not None and np.dtype(value.dtype) != np.dtype(expected_dtype): - raise TypeError(f"NumPy array dtype {value.dtype!r} does not match expected dtype {np.dtype(expected_dtype)!r}") - - -def _validate_ndarray_writeable(value: np.ndarray, require_writeable: bool) -> None: - if require_writeable and not value.flags.writeable: - raise TypeError("NumPy array actual must be writeable") - - -def _validate_ndarray_native_byte_order(value: np.ndarray, require_native_byte_order: bool) -> None: - if require_native_byte_order and not value.dtype.isnative: - raise TypeError("NumPy array actual must use native byte order") - - -def _validate_ndarray_aligned(value: np.ndarray, require_aligned: bool) -> None: - if require_aligned and not value.flags.aligned: - raise TypeError("NumPy array actual must be aligned") - - -def _validate_ndarray_contiguous(value: np.ndarray, require_contiguous: bool) -> None: - if require_contiguous and not (value.flags.c_contiguous or value.flags.f_contiguous): - raise TypeError("NumPy array actual must be contiguous") - - -def _validate_ndarray_expected_shape( - shape: tuple[int, ...], - expected_shape: Sequence[int | None] | int | None, -) -> None: - if expected_shape is None: - return - expected = NativeArrayHandleBase._normalize_expected_shape(expected_shape) - if len(expected) != len(shape): - raise TypeError(f"NumPy array shape rank {len(shape)} does not match expected shape rank {len(expected)}") - for axis, (actual, wanted) in enumerate(zip(shape, expected, strict=True)): - if wanted is not None and actual != wanted: - raise TypeError( - f"NumPy array has incompatible shape at axis {axis}: received {shape!r}, expected {expected!r}" - ) - - -def _validate_ndarray_expected_layout(value: np.ndarray, expected_layout: str | None) -> None: - if expected_layout is None: - return - required = NativeArrayHandleBase._normalize_expected_layout_name(expected_layout) - matches = value.flags.f_contiguous if required == "F" else value.flags.c_contiguous - if not matches: - raise TypeError(f"NumPy array actual has incompatible layout; expected ordering ({required})") - - __all__ = ( "AllocatableArray", "NativeArrayHandleBase", @@ -1589,25 +756,28 @@ def _validate_ndarray_expected_layout(value: np.ndarray, expected_layout: str | if __name__ == "__main__": - # Generated extensions supply small operation dictionaries like this one. - # The adapter turns their raw call signatures into the stable handle API. + # Generated extensions supply one dispatcher and its completed capabilities. state = {"array": np.array([1.0, 2.0, 3.0], dtype=np.float64)} def resize(*extents: np.int64) -> None: state["array"] = np.zeros(tuple(int(extent) for extent in extents), dtype=np.float64) - array = _native_array_handle_from_generated_ops( + operations = { + "allocated": lambda: True, + "shape": lambda: state["array"].shape, + "to_numpy": lambda: state["array"], + "resize": resize, + } + + def invoke(operation: str, args: tuple[Any, ...]) -> Any: + return operations[operation](*args) + + array = _native_array_handle_from_generated_dispatch( "allocatable", np.float64, 1, - { - "allocated": lambda: True, - "array_actual": lambda: state["array"].ctypes.data, - "descriptor": lambda: state["array"].ctypes.data, - "shape": lambda: state["array"].shape, - "to_numpy": lambda: state["array"], - "resize": resize, - }, + invoke, + operations, ) print(f"Runtime handle: {type(array).__name__}") diff --git a/prik/runtime/native_support/prik_binding.h b/prik/runtime/native_support/prik_binding.h index b4f96011f..3c50c6981 100644 --- a/prik/runtime/native_support/prik_binding.h +++ b/prik/runtime/native_support/prik_binding.h @@ -8,7 +8,9 @@ #include #include #include +#include #include +#include #include #include @@ -23,11 +25,32 @@ #include #include -#define PRIK_NATIVE_ARRAY_HANDLE_ABI_VERSION 1u -#define PRIK_NATIVE_ARRAY_HANDLE_CAPSULE_NAME "prik.native_array_handle.v1" -#define PRIK_NATIVE_ARRAY_HANDLE_MAGIC UINT64_C(0x583250594e414831) +/* + * One capsule publishes everything a generated binding needs from another + * extension's array handle, and its name is derived from the record's own + * layout rather than from a version anyone maintains by hand. + * + * The problem it solves: a capsule carries an address, and C has no runtime + * types, so a reader has to decide what is at that address using offsets its + * own compiler baked in. Two extensions built from different snapshots of this + * header disagree about those offsets while agreeing about everything they can + * name. Comparing a version *field* cannot settle it -- reading the field + * already assumes the layout in question -- and it goes wrong worst on + * `context`, `with_descriptor` and `release`, which are opaque addresses no + * reader can sanity-check before calling one. + * + * The capsule name carries both a semantic ABI version and a layout tag. + * PyCapsule_GetPointer compares names before it returns the pointer, so a + * producer with a different callback contract or record layout is refused + * before any field is read. Bump the version when field meanings or callback + * behavior change without changing the record layout. + */ +#define PRIK_NATIVE_ARRAY_BACKEND_CAPSULE_PREFIX "prik.native_array_backend.v1" #define PRIK_NATIVE_ARRAY_KIND_ALLOCATABLE 1u #define PRIK_NATIVE_ARRAY_KIND_POINTER 2u +#define PRIK_NATIVE_ARRAY_ATTRIBUTE_ALLOCATABLE 1u +#define PRIK_NATIVE_ARRAY_ATTRIBUTE_POINTER 2u +#define PRIK_NATIVE_ARRAY_ATTRIBUTE_OTHER 3u #if defined(_MSC_VER) #define PRIK_NO_INLINE __declspec(noinline) @@ -37,115 +60,259 @@ #define PRIK_NO_INLINE #endif -typedef void (*prik_native_array_release_fn)(void *descriptor); +#ifdef PRIK_BINDING_CAPTURE_ADDRESS +/* + * Report the address a caller already passed by reference. + * + * Fortran's `c_loc` can only name a variable that is a target or a pointer, so + * an ordinary declaration -- a module array without `target`, say -- has no way + * to state its own address. Handing the whole object to a `bind(C)` procedure + * does: an assumed-type assumed-size dummy is passed as the bare base address, + * so the parameter below already is where that object lives and only has to be + * handed back. It does the job of `c_loc` exactly where `c_loc` is not + * available, for any type and any rank, because it never inspects what it is + * given. Nothing on the Fortran side forms a pointer or claims a target. + * + * This needs external linkage for the generated bridge to call it, so it is + * defined only in the translation unit that opts in with this macro. + */ +void *prik_capture_address(void *base) +{ + return base; +} +#endif + +/* + * Consumer for one descriptor that is valid only while it runs. + * The descriptor stays a compiler-owned representation here, as everywhere + * else in this header, so this signature does not depend on the Fortran + * interop header and stays usable from a C-only extension. + */ +typedef void (*prik_native_array_descriptor_fn)(void *descriptor, void *context); /* - * Versioned cross-extension record for one persistent Fortran array - * descriptor. The descriptor representation remains compiler-owned; this - * record only makes its metadata, ownership, and validation ABI common to - * independently generated prik extensions. + * Enter the native entity and run `consumer` while its descriptor is live. + * An ordinary projection may return without calling `consumer` when its + * allocatable or pointer entity has no storage. `context` is whatever that + * entity needs to be reached; see the backend record below. + */ +typedef void (*prik_native_array_with_descriptor_fn)( + void *context, + prik_native_array_descriptor_fn consumer, + void *consumer_context); + +/* Release the storage a backend owns. NULL when the backend borrows it. */ +typedef void (*prik_native_array_release_fn)(void *context); + +/* + * Bridges a generated descriptor bridge, whose consumer takes the compiler's + * descriptor type, to a backend consumer that takes it as void *. Forwarding + * through this record avoids casting between function pointer types. + */ +typedef struct { + prik_native_array_descriptor_fn consumer; + void *context; +} prik_native_array_descriptor_forward; + +/* + * Versioned cross-extension backend for one array handle. + * + * `with_descriptor(context, ...)` is the single entry point: it produces a + * live descriptor and runs the consumer on it. `context` is the address that + * entity needs -- the parent object for a derived-type field, the wrapper's + * own descriptor storage for an owned handle, NULL for a module variable -- + * and is resolved once when the handle is built, so reaching the entity costs + * one indirect call instead of a Python attribute lookup per operation. + * + * A borrowed backend enters Fortran, which supplies the plan-selected + * descriptor for the call; the descriptor is gone when the consumer returns + * and must never be retained. An owned backend hands over the persistent + * storage it allocated, which stays valid for the handle's life. Consumers + * cannot tell the two ownership forms apart, and must not try to. + * + * The metadata refuses an incompatible producer before any descriptor is + * interpreted. The record's own layout is attested by the capsule name, so + * nothing here restates it; what remains is what the name cannot know: + * - descriptor_size attests the producer's CFI_CDESC_T(rank) layout, which + * neither this record nor the descriptor itself can be read to establish; + * - descriptor_kind identifies the native entity while descriptor_attribute + * identifies what `with_descriptor` supplies. Together with rank, cfi_type + * and element_size, they let a reader refuse a mismatch without entering + * Fortran at all. + * element_size is 0 when the element width is only known at run time, as for + * a deferred-length character array; such a reader takes it from the live + * descriptor's elem_len instead. */ typedef struct { - uint64_t magic; - uint32_t abi_version; - uint32_t struct_size; uint32_t descriptor_kind; + uint32_t descriptor_attribute; uint32_t rank; + uint32_t descriptor_size; int32_t cfi_type; - uint32_t reserved; size_t element_size; - size_t descriptor_size; - void *descriptor; + void *context; + prik_native_array_with_descriptor_fn with_descriptor; prik_native_array_release_fn release; -} prik_native_array_handle; - -#define PRIK_MAX_ARRAY_RANK 15 +} prik_native_array_backend; -#ifdef PRIK_BINDING_NATIVE_ARRAY_ACTUAL - -/* Mechanical result of the normal-array native-handle slow path. */ -typedef struct { - void *data; - int64_t rank; - int64_t itemsize; - int64_t extents[PRIK_MAX_ARRAY_RANK]; - int64_t upper_bounds[PRIK_MAX_ARRAY_RANK]; - int64_t strides[PRIK_MAX_ARRAY_RANK]; -} prik_array_actual; -#endif +/* + * Describe one field for the layout tag: its name, where it starts, how wide + * it is. All three come from the same token, so they cannot disagree. + */ +#define PRIK_NATIVE_ARRAY_BACKEND_FIELD(member) \ + {#member, offsetof(prik_native_array_backend, member), sizeof(((prik_native_array_backend *)0)->member)} -/* Release descriptor payload and storage at most once while retaining the record. */ -/* Build a Python string from caller-supplied status-message storage. +/* + * Fold this record into one tag. + * + * Every field contributes its name, its offset and its width, in declaration + * order, and the total size goes in first. A reorder, a widening, an + * insertion, a removal and a rename all change the result. FNV-1a is used + * because the mixing has to be order-dependent -- XOR-ing the offsets would + * give the same tag for two fields exchanged. + * + * Field names are folded too, so renaming a field changes the tag even when its + * offset and width stay the same. The semantic version covers contract changes + * that do not alter or rename a field. + */ +static inline uint64_t prik_native_array_backend_layout_tag(void) +{ + static const struct { + const char *name; + size_t offset; + size_t width; + } layout[] = { + PRIK_NATIVE_ARRAY_BACKEND_FIELD(descriptor_kind), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(descriptor_attribute), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(rank), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(descriptor_size), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(cfi_type), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(element_size), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(context), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(with_descriptor), + PRIK_NATIVE_ARRAY_BACKEND_FIELD(release), + }; + uint64_t tag = UINT64_C(14695981039346656037); + size_t index; + const char *character; + + tag = (tag ^ (uint64_t)sizeof(prik_native_array_backend)) * UINT64_C(1099511628211); + for (index = 0; index < sizeof(layout) / sizeof(layout[0]); ++index) { + for (character = layout[index].name; *character != '\0'; ++character) { + tag = (tag ^ (uint64_t)(unsigned char)*character) * UINT64_C(1099511628211); + } + tag = (tag ^ (uint64_t)layout[index].offset) * UINT64_C(1099511628211); + tag = (tag ^ (uint64_t)layout[index].width) * UINT64_C(1099511628211); + } + return tag; +} - The read never passes ``capacity`` because a native writer is not obliged to - terminate. When it did terminate, the bytes are taken exactly as written; - when it did not, the storage is fixed-length padded (Fortran blank-pads - ``character(len=n)``), so trailing blanks and NULs are dropped. */ -static inline PyObject *prik_status_message_text(const char *bytes, Py_ssize_t capacity) +/* + * Name the capsule this extension publishes and accepts. + * + * Both sides build this string from their own header, so two extensions agree + * on it exactly when they agree on the record. The name is what + * PyCapsule_GetPointer compares, which is why a disagreement is reported + * before the pointer is handed over rather than after something has been read + * through it. + */ +static inline const char *prik_native_array_backend_capsule_name(void) { - const char *terminator = (const char *)memchr(bytes, 0, (size_t)capacity); - Py_ssize_t length = capacity; - if (terminator != NULL) { - return PyUnicode_FromStringAndSize(bytes, (Py_ssize_t)(terminator - bytes)); - } - while (length > 0 && (bytes[length - 1] == ' ' || bytes[length - 1] == '\0')) { - length -= 1; + static char name[80]; + + if (name[0] == '\0') { + /* Every caller computes the same bytes, so a race writes them twice. */ + snprintf( + name, + sizeof(name), + PRIK_NATIVE_ARRAY_BACKEND_CAPSULE_PREFIX ".%016llx", + (unsigned long long)prik_native_array_backend_layout_tag()); } - return PyUnicode_FromStringAndSize(bytes, length); + return name; } +/* + * Hand over a descriptor the wrapper itself owns. + * + * A handle the caller created has no native entity behind it: the binding + * allocated its descriptor when the handle was first bound and keeps it for + * the handle's lifetime. There is no call-scoped window to stay inside, so + * the consumer runs on that storage directly. Publishing it through the same + * backend lets such a handle reach a call the way a module array does. + */ +static inline void prik_native_array_owned_with_descriptor( + void *context, + prik_native_array_descriptor_fn consumer, + void *consumer_context) +{ + consumer(context, consumer_context); +} -static inline void prik_native_array_handle_release(prik_native_array_handle *handle) +/* + * Release owned storage exactly once. + * + * `release` is non-NULL only when `context` is storage this extension + * allocated, so a borrowed backend -- a module variable's, a field's -- never + * reaches the free below and Python never releases native storage it does not + * own. Clearing `context` makes the release idempotent, so an explicit + * close() and finalization can both run. + */ +static inline void prik_native_array_backend_release(prik_native_array_backend *backend) { - void *descriptor; + void *context; - if (handle == NULL || handle->descriptor == NULL) { + if (backend == NULL || backend->release == NULL || backend->context == NULL) { return; } - descriptor = handle->descriptor; - handle->descriptor = NULL; - if (handle->release != NULL) { - handle->release(descriptor); - } - free(descriptor); + context = backend->context; + backend->context = NULL; + backend->release(context); + free(context); } -/* Finalize one native handle record owned by a Python capsule. */ -static inline void prik_native_array_handle_capsule_destructor(PyObject *capsule) +/* Finalize one backend record owned by a Python capsule. */ +static inline void prik_native_array_backend_capsule_destructor(PyObject *capsule) { PyObject *error_type = NULL; PyObject *error_value = NULL; PyObject *error_traceback = NULL; - prik_native_array_handle *handle; + prik_native_array_backend *backend; PyErr_Fetch(&error_type, &error_value, &error_traceback); - handle = (prik_native_array_handle *)PyCapsule_GetPointer( - capsule, PRIK_NATIVE_ARRAY_HANDLE_CAPSULE_NAME); - if (handle == NULL) { + backend = (prik_native_array_backend *)PyCapsule_GetPointer( + capsule, prik_native_array_backend_capsule_name()); + if (backend == NULL) { PyErr_Clear(); } else { - prik_native_array_handle_release(handle); - handle->magic = 0; - free(handle); + prik_native_array_backend_release(backend); + free(backend); } PyErr_Restore(error_type, error_value, error_traceback); } /* - * Create a capsule that takes descriptor ownership only on success. The - * caller remains responsible for descriptor cleanup when this function - * returns NULL. + * Publish a per-handle backend. + * + * A handle whose entity needs a context address, or whose storage this + * extension owns, cannot share one file-scope record, so its backend is built + * when the handle is and released with the capsule that carries it. A module + * variable needs neither, and publishes a file-scope record directly. + * + * Ownership of `context` transfers only on success: when this returns NULL the + * caller is still responsible for releasing it. */ -static inline PyObject *prik_native_array_handle_capsule_new( +static inline PyObject *prik_native_array_backend_capsule_new( uint32_t descriptor_kind, + uint32_t descriptor_attribute, uint32_t rank, + uint32_t descriptor_size, int cfi_type, size_t element_size, - size_t descriptor_size, - void *descriptor, + void *context, + prik_native_array_with_descriptor_fn with_descriptor, prik_native_array_release_fn release) { - prik_native_array_handle *handle; + prik_native_array_backend *backend; PyObject *capsule; if (descriptor_kind != PRIK_NATIVE_ARRAY_KIND_ALLOCATABLE @@ -153,250 +320,327 @@ static inline PyObject *prik_native_array_handle_capsule_new( PyErr_SetString(PyExc_ValueError, "invalid prik native array descriptor kind"); return NULL; } - if (descriptor == NULL || descriptor_size == 0 || element_size == 0 || release == NULL) { - PyErr_SetString(PyExc_ValueError, "incomplete prik native array handle storage"); + if (descriptor_attribute != PRIK_NATIVE_ARRAY_ATTRIBUTE_ALLOCATABLE + && descriptor_attribute != PRIK_NATIVE_ARRAY_ATTRIBUTE_POINTER + && descriptor_attribute != PRIK_NATIVE_ARRAY_ATTRIBUTE_OTHER) { + PyErr_SetString(PyExc_ValueError, "invalid prik native array descriptor attribute"); return NULL; } - handle = (prik_native_array_handle *)calloc(1, sizeof(*handle)); - if (handle == NULL) { + if (with_descriptor == NULL) { + PyErr_SetString(PyExc_ValueError, "prik native array backend needs a descriptor entry point"); + return NULL; + } + if (release != NULL && context == NULL) { + PyErr_SetString(PyExc_ValueError, "prik native array backend has no storage to release"); + return NULL; + } + backend = (prik_native_array_backend *)calloc(1, sizeof(*backend)); + if (backend == NULL) { PyErr_NoMemory(); return NULL; } - handle->magic = PRIK_NATIVE_ARRAY_HANDLE_MAGIC; - handle->abi_version = PRIK_NATIVE_ARRAY_HANDLE_ABI_VERSION; - handle->struct_size = (uint32_t)sizeof(*handle); - handle->descriptor_kind = descriptor_kind; - handle->rank = rank; - handle->cfi_type = (int32_t)cfi_type; - handle->element_size = element_size; - handle->descriptor_size = descriptor_size; - handle->descriptor = descriptor; - handle->release = release; + backend->descriptor_kind = descriptor_kind; + backend->descriptor_attribute = descriptor_attribute; + backend->rank = rank; + backend->descriptor_size = descriptor_size; + backend->cfi_type = (int32_t)cfi_type; + backend->element_size = element_size; + backend->context = context; + backend->with_descriptor = with_descriptor; + backend->release = release; capsule = PyCapsule_New( - handle, - PRIK_NATIVE_ARRAY_HANDLE_CAPSULE_NAME, - prik_native_array_handle_capsule_destructor); + backend, prik_native_array_backend_capsule_name(), prik_native_array_backend_capsule_destructor); if (capsule == NULL) { - handle->descriptor = NULL; - handle->magic = 0; - free(handle); + backend->context = NULL; + free(backend); } return capsule; } -/* Validate and unwrap one cross-extension native array handle capsule. */ -static inline prik_native_array_handle *prik_native_array_handle_from_capsule( - PyObject *capsule, - uint32_t expected_kind, - uint32_t expected_rank, - int expected_cfi_type, - size_t expected_element_size, - size_t expected_descriptor_size) +/* + * Unwrap a backend capsule and check what makes its record usable at all. + * + * A backend that owns its storage and has released it is closed; a borrowed + * one has no storage of its own and its NULL context means only that its + * entity needs no address. + */ +static inline prik_native_array_backend *prik_native_array_backend_from_capsule(PyObject *capsule) { - prik_native_array_handle *handle; + prik_native_array_backend *backend; - if (!PyCapsule_IsValid(capsule, PRIK_NATIVE_ARRAY_HANDLE_CAPSULE_NAME)) { - PyErr_SetString(PyExc_TypeError, "incompatible prik native array handle capsule"); + backend = (prik_native_array_backend *)PyCapsule_GetPointer( + capsule, prik_native_array_backend_capsule_name()); + if (backend == NULL) { return NULL; } - handle = (prik_native_array_handle *)PyCapsule_GetPointer( - capsule, PRIK_NATIVE_ARRAY_HANDLE_CAPSULE_NAME); - if (handle == NULL) { + if (backend->with_descriptor == NULL) { + PyErr_SetString(PyExc_TypeError, "incompatible prik native array backend record"); return NULL; } - if (handle->magic != PRIK_NATIVE_ARRAY_HANDLE_MAGIC - || handle->abi_version != PRIK_NATIVE_ARRAY_HANDLE_ABI_VERSION - || handle->struct_size != sizeof(*handle)) { - PyErr_SetString(PyExc_TypeError, "incompatible prik native array handle ABI"); - return NULL; - } - if (handle->descriptor_kind != expected_kind) { - PyErr_SetString(PyExc_TypeError, "prik native array descriptor kind does not match"); + if (backend->release != NULL && backend->context == NULL) { + PyErr_SetString(PyExc_ReferenceError, "prik native array handle is closed"); return NULL; } - if (handle->rank != expected_rank) { - PyErr_SetString(PyExc_ValueError, "prik native array descriptor rank does not match"); + return backend; +} + +/* + * Read a backend for a descriptor dummy. + * + * Such a dummy is declared allocatable or pointer, so the handle's own kind + * has to be the declared one; everything else about the storage must match the + * declaration too. `expected_element_size` is 0 for a dummy that takes its + * width from the actual. + */ +static inline prik_native_array_backend *prik_native_array_backend_for_descriptor( + PyObject *capsule, + uint32_t expected_descriptor_kind, + uint32_t expected_rank, + uint32_t expected_descriptor_size, + int expected_cfi_type, + size_t expected_element_size) +{ + prik_native_array_backend *backend; + uint32_t expected_descriptor_attribute; + + backend = prik_native_array_backend_from_capsule(capsule); + if (backend == NULL) { return NULL; } - if (handle->cfi_type != expected_cfi_type) { - PyErr_SetString(PyExc_TypeError, "prik native array element type does not match"); + if (backend->descriptor_size != expected_descriptor_size) { + PyErr_SetString(PyExc_TypeError, "incompatible Fortran descriptor storage size"); return NULL; } - if (expected_element_size != 0 && handle->element_size != expected_element_size) { - PyErr_SetString(PyExc_TypeError, "prik native array element size does not match"); + expected_descriptor_attribute = expected_descriptor_kind == PRIK_NATIVE_ARRAY_KIND_POINTER + ? PRIK_NATIVE_ARRAY_ATTRIBUTE_POINTER + : PRIK_NATIVE_ARRAY_ATTRIBUTE_ALLOCATABLE; + if (backend->descriptor_attribute != expected_descriptor_attribute) { + PyErr_SetString( + PyExc_TypeError, + "native array handle does not expose the descriptor attribute required by the dummy argument"); return NULL; } - if (handle->descriptor_size != expected_descriptor_size) { - PyErr_SetString(PyExc_TypeError, "incompatible Fortran descriptor storage size"); + if (backend->descriptor_kind != expected_descriptor_kind || backend->rank != expected_rank + || backend->cfi_type != expected_cfi_type + || (expected_element_size != 0 && backend->element_size != expected_element_size)) { + PyErr_SetString(PyExc_TypeError, "native array handle does not match the declared dummy argument"); return NULL; } - if (handle->descriptor == NULL) { - PyErr_SetString(PyExc_ReferenceError, "prik native array handle is closed"); + return backend; +} + +/* + * Take the descriptor a backend owns. + * + * `prik_native_array_owned_with_descriptor` hands its consumer the context + * itself, so an owned backend's context *is* persistent descriptor storage and + * may be held for as long as the handle lives. Only the operations published + * on an owned handle -- allocate, resize, deallocate, destroy -- ask for it, + * and only ever about their own handle's storage; a borrowed backend has + * nothing of the kind, and saying so here keeps a mis-wired one an error + * instead of a null descriptor handed to CFI_allocate. + * + * `release` is non-NULL exactly for storage this extension owns, which is the + * same fact and the one that survives being read from another extension, where + * the inline entry point above is a different function. + */ +static inline void *prik_native_array_backend_owned_descriptor(prik_native_array_backend *backend) +{ + if (backend->release == NULL) { + PyErr_SetString( + PyExc_TypeError, + "native array handle operation needs storage the handle owns; this one borrows its descriptor"); return NULL; } - return handle; + return backend->context; } /* - * Execute the Python native-handle handoff once per slow-path call site. - * The completed wrapper plan supplies every contract selector; this helper - * only performs reference management and decodes the returned ABI fields. + * Read a backend for an ordinary array actual. + * + * An ordinary array dummy takes the storage behind a handle, not the handle's + * descriptor kind: an allocatable and a pointer are equally acceptable there, + * so the kind is not compared. Everything that decides whether the storage + * matches the dummy -- rank, element type and element size -- still is, and a + * character dummy that takes its width from the actual passes 0 for the size. */ -#ifdef PRIK_BINDING_NATIVE_ARRAY_ACTUAL -PRIK_NO_INLINE static int prik_array_actual_unpack( - PyObject *value, - const char *dtype, - int expected_rank, - PyObject *expected_shape, - const char *expected_layout, - int require_writeable, - int require_native_byte_order, - int require_aligned, - int include_rank, - int include_itemsize, - int include_strides, - int require_contiguous, - int flatten_storage, - int flat_axis, - prik_array_actual *actual) -{ - PyObject *runtime = NULL; - PyObject *helper = NULL; - PyObject *layout = NULL; - PyObject *packed = NULL; - PyObject *item; - Py_ssize_t expected_fields; - Py_ssize_t position; - int axis; +static inline prik_native_array_backend *prik_native_array_backend_for_actual( + PyObject *capsule, + uint32_t minimum_rank, + uint32_t maximum_rank, + int expected_cfi_type, + size_t expected_element_size, + const char *dtype_name, + const char *argument_name) +{ + prik_native_array_backend *backend; - if (expected_shape == NULL || actual == NULL) { - PyErr_SetString(PyExc_RuntimeError, "prik generated an incomplete native array actual"); - return -1; + backend = prik_native_array_backend_from_capsule(capsule); + if (backend == NULL) { + return NULL; } - if (expected_rank < 1 || expected_rank > PRIK_MAX_ARRAY_RANK) { - PyErr_SetString(PyExc_RuntimeError, "prik generated an invalid native array rank"); - return -1; + if (backend->rank < minimum_rank || backend->rank > maximum_rank + || backend->cfi_type != expected_cfi_type + || (expected_element_size != 0 && backend->element_size != expected_element_size)) { + PyErr_Format( + PyExc_TypeError, + "%s handle of rank %u with %zu-byte elements does not match expected dtype %s for argument %s", + backend->descriptor_kind == PRIK_NATIVE_ARRAY_KIND_POINTER ? "pointer" : "allocatable", + (unsigned)backend->rank, + backend->element_size, + dtype_name, + argument_name); + return NULL; } + return backend; +} - actual->data = NULL; - actual->rank = 0; - actual->itemsize = 0; - for (axis = 0; axis < PRIK_MAX_ARRAY_RANK; axis++) { - actual->extents[axis] = 0; - actual->upper_bounds[axis] = 0; - actual->strides[axis] = 1; - } +#define PRIK_MAX_ARRAY_RANK 15 - if (expected_layout == NULL) { - layout = Py_None; - Py_INCREF(layout); - } else { - layout = PyUnicode_FromString(expected_layout); - if (layout == NULL) { - return -1; - } +#ifdef PRIK_BINDING_NATIVE_ARRAY_ACTUAL + +/* Mechanical result of reading a live handle descriptor for an ordinary array. */ +typedef struct { + void *data; + int64_t rank; + int64_t itemsize; + int64_t extents[PRIK_MAX_ARRAY_RANK]; + int64_t upper_bounds[PRIK_MAX_ARRAY_RANK]; + int64_t strides[PRIK_MAX_ARRAY_RANK]; +} prik_array_actual; +#endif + +/* Build a Python string from caller-supplied status-message storage. + + The read never passes ``capacity`` because a native writer is not obliged to + terminate. When it did terminate, the bytes are taken exactly as written; + when it did not, the storage is fixed-length padded (Fortran blank-pads + ``character(len=n)``), so trailing blanks and NULs are dropped. */ +static inline PyObject *prik_status_message_text(const char *bytes, Py_ssize_t capacity) +{ + const char *terminator = (const char *)memchr(bytes, 0, (size_t)capacity); + Py_ssize_t length = capacity; + if (terminator != NULL) { + return PyUnicode_FromStringAndSize(bytes, (Py_ssize_t)(terminator - bytes)); } - runtime = PyImport_ImportModule("prik.runtime.handles"); - if (runtime == NULL) { - Py_DECREF(layout); - return -1; + while (length > 0 && (bytes[length - 1] == ' ' || bytes[length - 1] == '\0')) { + length -= 1; } - helper = PyObject_GetAttrString(runtime, "_native_array_actual_argument_for_binding_positional"); - Py_DECREF(runtime); - if (helper == NULL) { - Py_DECREF(layout); - return -1; + return PyUnicode_FromStringAndSize(bytes, length); +} + + +/* Completed selectors for compact ordinary NumPy-array validation. */ +#define PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS 0 +#define PRIK_ARRAY_LAYOUT_C_CONTIGUOUS 1 +#define PRIK_ARRAY_LAYOUT_F_CONTIGUOUS 2 +#define PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F 3 +#define PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F 4 +#define PRIK_ARRAY_LAYOUT_ANY_STRIDED 5 + +/* + * Report why one strided array cannot be described to Fortran. + * + * A Fortran array section runs over a contiguous parent: each axis advances by + * a whole number of elements, the axes are ordered by how far they step, and no + * axis steps back into another's span. A view that breaks those rules -- one + * NumPy made by broadcasting, or by overlapping itself -- has no parent to be a + * section of, whatever its strides say, so it cannot be handed over without + * copying. An axis that merely runs backwards breaks none of them, and is + * refused only where the entrypoint takes an address and so has nowhere to say + * so. + */ +static inline int prik_array_refuse_section( + const char *argument_name, + int axis, + int signed_strides) +{ + PyErr_Format( + PyExc_TypeError, + "Argument %s has a layout at axis %d that is not a Fortran array section: " + "each axis must step a whole number of elements%s, in increasing order of step, without overlapping", + argument_name, + axis, + signed_strides ? "" : " forward"); + return -1; +} + +/* + * Validate one axis of an F-ordered strided array. + * + * ``signed_strides`` says whether an axis may run backwards, which is exactly + * whether the entrypoint carries a descriptor to record it in. Everything else + * is required either way, because it is what makes the view a section at all. + */ +static inline int prik_array_validate_strided_axis( + PyArrayObject *array, + int axis, + int signed_strides, + const char *argument_name) +{ + int previous_axis; + npy_intp stride = PyArray_STRIDE(array, axis); + npy_intp itemsize = PyArray_ITEMSIZE(array); + npy_intp previous_extent; + npy_intp previous; + + if (itemsize <= 0 || (stride % itemsize) != 0) { + /* A step that is not a whole element has no Fortran spelling at all. */ + return prik_array_refuse_section(argument_name, axis, signed_strides); } - packed = PyObject_CallFunction( - helper, - "OsiOOiiiiiiiii", - value, - dtype, - expected_rank, - expected_shape, - layout, - require_writeable, - require_native_byte_order, - require_aligned, - include_rank, - include_itemsize, - include_strides, - require_contiguous, - flatten_storage, - flat_axis); - Py_DECREF(helper); - Py_DECREF(layout); - if (packed == NULL) { - return -1; + if (PyArray_SIZE(array) == 0 || PyArray_DIM(array, axis) <= 1) { + /* One element cannot step anywhere, and no element cannot either. */ + return 0; } - - expected_fields = 1 + include_rank + include_itemsize + expected_rank; - if (include_strides) { - expected_fields += 2 * expected_rank; + if (stride == 0) { + /* A repeated element: NumPy broadcasting, which Fortran has no form for. */ + return prik_array_refuse_section(argument_name, axis, signed_strides); } - if (!PyTuple_Check(packed) || PyTuple_GET_SIZE(packed) != expected_fields) { - PyErr_SetString(PyExc_RuntimeError, "prik native array handoff returned invalid ABI fields"); - Py_DECREF(packed); - return -1; + if (stride == NPY_MIN_INTP) { + /* Its magnitude is not representable by the signed descriptor index type. */ + return prik_array_refuse_section(argument_name, axis, signed_strides); } - - position = 0; - actual->data = PyLong_AsVoidPtr(PyTuple_GET_ITEM(packed, position++)); - if (actual->data == NULL && PyErr_Occurred()) { - Py_DECREF(packed); + if (!signed_strides && stride < 0) { + PyErr_Format( + PyExc_TypeError, + "Argument %s runs backwards along axis %d, and this entrypoint receives only an address, " + "which cannot record a direction", + argument_name, + axis); return -1; } - if (include_rank) { - actual->rank = (int64_t)PyLong_AsLongLong(PyTuple_GET_ITEM(packed, position++)); - if (PyErr_Occurred()) { - Py_DECREF(packed); - return -1; - } + previous_axis = axis - 1; + while (previous_axis >= 0 && PyArray_DIM(array, previous_axis) <= 1) { + previous_axis -= 1; } - if (include_itemsize) { - actual->itemsize = (int64_t)PyLong_AsLongLong(PyTuple_GET_ITEM(packed, position++)); - if (PyErr_Occurred()) { - Py_DECREF(packed); - return -1; + if (previous_axis >= 0) { + npy_intp current = stride < 0 ? -stride : stride; + previous = PyArray_STRIDE(array, previous_axis); + if (previous == NPY_MIN_INTP) { + return prik_array_refuse_section(argument_name, previous_axis, signed_strides); } - } - for (axis = 0; axis < expected_rank; axis++) { - item = PyTuple_GET_ITEM(packed, position++); - actual->extents[axis] = (int64_t)PyLong_AsLongLong(item); - if (PyErr_Occurred()) { - Py_DECREF(packed); + previous = previous < 0 ? -previous : previous; + previous_extent = PyArray_DIM(array, previous_axis); + if (previous != 0 && previous > current / previous_extent) { + /* + * This axis steps less far than the one before it covers, so the + * axes are either in the wrong order for Fortran or they overlap. + * Both are the same measurement, and ordering is what a caller can + * actually act on. + */ + PyErr_Format( + PyExc_TypeError, + "Argument %s has incompatible layout; expected ordering (F)", + argument_name); return -1; } - } - if (include_strides) { - for (axis = 0; axis < expected_rank; axis++) { - item = PyTuple_GET_ITEM(packed, position++); - actual->upper_bounds[axis] = (int64_t)PyLong_AsLongLong(item); - if (PyErr_Occurred()) { - Py_DECREF(packed); - return -1; - } - } - for (axis = 0; axis < expected_rank; axis++) { - item = PyTuple_GET_ITEM(packed, position++); - actual->strides[axis] = (int64_t)PyLong_AsLongLong(item); - if (PyErr_Occurred()) { - Py_DECREF(packed); - return -1; - } + if (previous == 0 || (current % previous) != 0) { + /* CFI_section needs an integral step relative to its contiguous parent. */ + return prik_array_refuse_section(argument_name, axis, signed_strides); } } - Py_DECREF(packed); return 0; } -#endif - -/* Completed selectors for compact ordinary NumPy-array validation. */ -#define PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS 0 -#define PRIK_ARRAY_LAYOUT_C_CONTIGUOUS 1 -#define PRIK_ARRAY_LAYOUT_F_CONTIGUOUS 2 -#define PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F 3 -#define PRIK_ARRAY_LAYOUT_ANY_STRIDED 4 /* * Validate mechanics shared by every ordinary NumPy-array argument. The @@ -438,23 +682,10 @@ static inline int prik_array_validate_ndarray( } if (layout == PRIK_ARRAY_LAYOUT_ANY_STRIDED) { /* The plan accepts whatever strides the caller's array already has. */ - } else if (layout == PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F) { + } else if (layout == PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F || layout == PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F) { + int signed_strides = layout == PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F; for (axis = 0; axis < rank; axis++) { - npy_intp stride = PyArray_STRIDE(array, axis); - if ((stride % PyArray_ITEMSIZE(array)) != 0 - || (PyArray_SIZE(array) > 0 && PyArray_DIM(array, axis) > 1 && stride <= 0)) { - PyErr_Format( - PyExc_TypeError, - "Argument %s has incompatible layout; expected ordering (F)", - argument_name); - return -1; - } - if (axis > 0 && PyArray_SIZE(array) > 0 && PyArray_DIM(array, axis - 1) > 0 - && stride < PyArray_STRIDE(array, axis - 1) * PyArray_DIM(array, axis - 1)) { - PyErr_Format( - PyExc_TypeError, - "Argument %s has incompatible layout; expected ordering (F)", - argument_name); + if (prik_array_validate_strided_axis(array, axis, signed_strides, argument_name) < 0) { return -1; } } @@ -532,17 +763,9 @@ static inline int prik_array_validate( * * A wrapper needs two things from an array argument: the raw pointer handed to * the native entrypoint, and one extent per contract axis. Obtaining them takes - * two routes. A NumPy array is validated and read directly, which is the route - * every ordinary call takes. Anything else is a native array handle returned - * earlier by generated code, whose Fortran-owned descriptor is resolved through - * prik.runtime.handles; that route also produces the diagnostics for an - * argument that is neither. - * - * Both routes live here so the emitted wrapper carries one call instead of the - * whole sequence. Every parameter is a selector already decided by the - * completed wrapper plan; this helper makes no interoperability decision of - * its own, and each is passed directly rather than through a descriptor struct - * so the values arrive in registers instead of behind a pointer. + * two routes. This helper validates and reads NumPy arrays. Generated wrappers + * read native handles through their versioned backend capsule before falling + * back here for the NumPy route and the common wrong-type diagnostic. * * object the Python argument to bind * numpy_type NPY_* element selector the plan chose for this array @@ -556,18 +779,11 @@ static inline int prik_array_validate( * require_contiguous non-zero when the plan requires contiguous storage * require_writeable non-zero when the plan may write through this argument * python_type public dtype name used in diagnostics, "numpy.float64" - * dtype_name handoff dtype name for the handle route, "float64" * argument_name public argument name used in diagnostics - * order handoff ordering for the handle route, "F", "C", or NULL * flatten_axis contract axis that absorbs every trailing runtime axis; * `rank - 1` when the plan does not flatten - * actual_* the nine handle-route selectors passed straight through - * to prik_array_actual_unpack * fixed one entry per contract axis: the required extent, or - * -1 when the axis is free. A required extent is checked - * on the direct route and becomes the expected-shape - * entry on the handle route; a free axis is neither - * checked nor constrained + * -1 when the axis is free * data receives the pointer passed to the native entrypoint * extents receives one extent per contract axis * @@ -584,19 +800,8 @@ PRIK_NO_INLINE static int prik_bind_array( int require_contiguous, int require_writeable, const char *python_type, - const char *dtype_name, const char *argument_name, - const char *order, int flatten_axis, - int actual_writable, - int actual_native_byte_order, - int actual_aligned, - int actual_runtime_rank, - int actual_itemsize, - int actual_strides, - int actual_contiguous, - int actual_flatten, - int actual_flat_axis, const long long *fixed, void **data, int64_t *extents) @@ -629,41 +834,15 @@ PRIK_NO_INLINE static int prik_bind_array( } return 0; } - { - PyObject *shape = PyTuple_New(rank); - prik_array_actual actual; - if (shape == NULL) { - return -1; - } - for (axis = 0; axis < rank; ++axis) { - PyObject *item; - if (fixed[axis] >= 0) { - item = PyLong_FromLongLong(fixed[axis]); - } else { - Py_INCREF(Py_None); - item = Py_None; - } - if (item == NULL) { - Py_DECREF(shape); - return -1; - } - PyTuple_SET_ITEM(shape, axis, item); - } - if (prik_array_actual_unpack( - object, dtype_name, rank, shape, order, - actual_writable, actual_native_byte_order, actual_aligned, - actual_runtime_rank, actual_itemsize, actual_strides, - actual_contiguous, actual_flatten, actual_flat_axis, &actual) < 0) { - Py_DECREF(shape); - return -1; - } - Py_DECREF(shape); - *data = actual.data; - for (axis = 0; axis < rank; ++axis) { - extents[axis] = actual.extents[axis]; - } - return 0; - } + /* A generated wrapper takes an accepted handle route before calling this + ndarray-only helper. */ + PyErr_Format( + PyExc_TypeError, + "Expected a compatible numpy.ndarray of dtype %s for argument %s. Received ", + python_type, + argument_name, + Py_TYPE(object)->tp_name); + return -1; } #endif diff --git a/prik/semantics/fortran2ir.py b/prik/semantics/fortran2ir.py index f6fe7327a..8d269ccfc 100644 --- a/prik/semantics/fortran2ir.py +++ b/prik/semantics/fortran2ir.py @@ -468,6 +468,8 @@ def _convert_variable_type( if getattr(var, "target", False): metadata["aliased"] = True metadata["fortran_target"] = True + if getattr(var, "_fortran_protected", False): + metadata["fortran_protected"] = True if getattr(var, "pointer", False): metadata["fortran_pointer"] = True metadata["fortran_pointer_association"] = "runtime" diff --git a/tests/c/_support/cli.py b/tests/c/_support/cli.py index fa2ecb5d7..b18f6113e 100644 --- a/tests/c/_support/cli.py +++ b/tests/c/_support/cli.py @@ -2,6 +2,7 @@ from __future__ import annotations +import sys import types import prik.cli as prik_cli @@ -82,4 +83,7 @@ def error(self, message): parser = FakeParser() monkeypatch.setattr(prik_cli, "_parser_for_argv", lambda argv: (parser, argv)) + # main() falls back to sys.argv when called without an argv, and an empty + # command line prints help instead of dispatching. + monkeypatch.setattr(sys, "argv", ["prik", "input.c"]) return parser diff --git a/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py b/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py index d2fdf705a..9341ade1f 100644 --- a/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py +++ b/tests/c/primitive_pointers/codegen/test_runtime_rank_pointer_lowering.py @@ -4,7 +4,7 @@ from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ArrayPythonLayout +from prik.policy.models import ArrayEntrypointABI, ArrayPythonLayout, NativeArraySourceKind from prik.semantics.c2ir import c_file_to_semantic_module @@ -18,11 +18,14 @@ def test_direct_c_binding_keeps_pointer_abi_and_uses_completed_runtime_rank_boun binding = next(source.text for source in generated.sources if source.path.suffix == ".c") function = plan.namespaces[0].functions[0] array = function.arguments[0].array + actual = function.arguments[0].native_array_actual assert plan.bridge is None assert plan.entrypoint.native_languages == ("c",) assert array.rank is None assert (array.minimum_rank, array.maximum_rank) == (0, 15) + assert array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS + assert actual.accepted_sources == (NativeArraySourceKind.NDARRAY,) assert "double native_read(const double * input);" in binding assert array.python_layout is ArrayPythonLayout.ANY_STRIDED assert ("prik_array_validate(bound_input_obj, NPY_FLOAT64, 0, 15, PRIK_ARRAY_LAYOUT_ANY_STRIDED, 0, 0") in binding diff --git a/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py b/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py index e5b4ec126..47bc1bb26 100644 --- a/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py +++ b/tests/c/primitive_pointers/policy/test_runtime_rank_pointer_policy.py @@ -2,7 +2,12 @@ from prik.pipeline.pyi import pyi_text_to_semantic_module from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ArrayPythonLayout, EntrypointPassingConvention, EntrypointProjectionAction +from prik.policy.models import ( + ArrayEntrypointABI, + ArrayPythonLayout, + EntrypointPassingConvention, + EntrypointProjectionAction, +) from prik.semantics.native_contract import validate_pyi_native_contract @@ -28,6 +33,9 @@ def scale(values: Float64[...]) -> None: ... assert array.native_order == "ORDER_C" assert array.contiguous is None assert array.python_layout is ArrayPythonLayout.ANY_STRIDED + assert array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS + assert policy.native_call_slots[1].array.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS + assert policy.arguments[0].entrypoint_passing is EntrypointPassingConvention.POINTER_REFERENCE assert size_slot.semantic_type_name == "SizeT" assert size_slot.projection_action is EntrypointProjectionAction.COMPUTED_SIZE assert size_slot.entrypoint_passing is EntrypointPassingConvention.C_VALUE diff --git a/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py b/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py index a0adafbbc..022fba66b 100644 --- a/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py +++ b/tests/c/primitive_scalars/codegen/test_exact_native_scalar_lowering.py @@ -123,4 +123,4 @@ def update(values: {annotation}[:]) -> None: ... assert f"void update({c_type} * values);" in binding assert f"prik_bind_array(bound_values_obj, {numpy_macro}," in binding assert f'"{numpy_name}", ' in binding - assert '"values", NULL,' in binding + assert '"values", ' in binding diff --git a/tests/fortran/_support/native_array_handles.py b/tests/fortran/_support/native_array_handles.py index d14311abd..df4f158cf 100644 --- a/tests/fortran/_support/native_array_handles.py +++ b/tests/fortran/_support/native_array_handles.py @@ -3,62 +3,63 @@ import numpy as np -from prik.runtime.handles import ( - _NativeArrayHandoff, -) - - class _ArrayState: def __init__(self, *, shape=None, value=None): self.shape = shape self.value = value -def _handoff(address=1): - return _NativeArrayHandoff(address) - - -def _required_handoff_ops(): - return { - "array_actual": lambda _handle: _handoff(101), - "descriptor": lambda _handle: _handoff(102), - } - - def _common_ops(state: _ArrayState): + """Return the operations every generated handle supplies. + + A generated handle answers both of these from its live descriptor, so each + reports absence itself rather than being asked about it first. + """ return { - **_required_handoff_ops(), "shape": lambda _handle: state.shape, "to_numpy": lambda _handle: state.value if state.shape is not None else None, } -def _pointer_descriptor_for_array(value: np.ndarray): - return { - "base_addr": int(value.ctypes.data), - "elem_len": int(value.dtype.itemsize), - "rank": value.ndim, - "dim": [ - { - "lower_bound": 1, - "extent": int(extent), - "sm": int(stride), - } - for extent, stride in zip(value.shape, value.strides, strict=True) - ], - } +def _handle_dispatch(operations): + """Adapt concise operation test doubles to the runtime dispatcher contract.""" + + def invoke(operation, args): + if operation in {"allocate", "resize"}: + args = (args,) + return operations[operation](None, *args) + + return {"invoke": invoke, "capabilities": operations} + +def _generated_handle_dispatch(operations): + """Adapt generated-call-shaped test doubles to one dispatcher callable.""" -class _DescriptorFieldRecord: - def __init__(self, **fields): - self.__dict__.update(fields) + def invoke(operation, args): + return operations[operation](*args) + return invoke -def _pointer_descriptor_record_for_array(value: np.ndarray): - descriptor = _pointer_descriptor_for_array(value) - return _DescriptorFieldRecord( - base_addr=descriptor["base_addr"], - elem_len=descriptor["elem_len"], - rank=descriptor["rank"], - dim=[_DescriptorFieldRecord(**dimension) for dimension in descriptor["dim"]], + +def _descriptor_facts_for_array(value: np.ndarray, *, lower_bound: int = 1): + """Return the flat facts a generated pointer reports for one array. + + The layout is the one the generated consumer writes: base address, element + width, rank, then a lower bound, extent and byte stride per axis. + """ + return ( + int(value.ctypes.data), + int(value.dtype.itemsize), + value.ndim, + *( + field + for extent, stride in zip(value.shape, value.strides, strict=True) + for field in (lower_bound, int(extent), int(stride)) + ), ) + + +def _absent_descriptor_facts(dtype, rank: int): + """Return the flat facts a generated handle reports for absent storage.""" + itemsize = int(np.dtype(dtype).itemsize) + return (0, itemsize, rank, *((0, 0, itemsize) * rank)) diff --git a/tests/fortran/_support/ownership_policy.py b/tests/fortran/_support/ownership_policy.py index 6d9cb300f..c87b6c8ce 100644 --- a/tests/fortran/_support/ownership_policy.py +++ b/tests/fortran/_support/ownership_policy.py @@ -122,15 +122,18 @@ def _hidden_output_context(**kwargs) -> OwnershipContext: def _native_array_policy( *, descriptor_kind: str = "allocatable", + descriptor_attribute: str | None = None, handle_kind: str = "borrowed_module_descriptor", to_numpy: str = "borrowed_view", descriptor_interop: str = "none", + descriptor_inquiries: bool = True, nullable: bool = False, optional_absent: bool = False, operations: tuple[str, ...] = ("allocated", "to_numpy"), ) -> NativeArrayHandlePolicy: return NativeArrayHandlePolicy( descriptor_kind=descriptor_kind, + descriptor_attribute=descriptor_attribute or descriptor_kind, handle_kind=handle_kind, origin="module_variable", owner="native", @@ -147,6 +150,7 @@ def _native_array_policy( destroy_behavior="none", to_numpy=to_numpy, descriptor_interop=descriptor_interop, + descriptor_inquiries=descriptor_inquiries, nullable=nullable, optional_absent=optional_absent, storage_mode="alias", diff --git a/tests/fortran/_support/wrapper_build.py b/tests/fortran/_support/wrapper_build.py index 584e11f27..3fe6c0834 100644 --- a/tests/fortran/_support/wrapper_build.py +++ b/tests/fortran/_support/wrapper_build.py @@ -452,8 +452,18 @@ def _result_dtype(expected): return np.asarray(expected).dtype +def _array_element_dtype(expected): + """Return the dtype one Fortran array element reports to Python. + + A `logical(c_bool)` element is one byte holding zero or one, which is what + `numpy.bool_` describes, so it needs no adjustment. Only kinds wider than a + byte report an integer instead, and no fixture here exercises one. + """ + return _result_dtype(expected) + + def _array_argument(value, size: int, *, strided: bool): - dtype = np.asarray(value).dtype + dtype = _array_element_dtype(value) if strided: storage = np.zeros(2 * size, dtype=dtype) array = storage[::2] @@ -464,7 +474,7 @@ def _array_argument(value, size: int, *, strided: bool): def _array_result(expected, size: int, *, strided: bool): - dtype = _result_dtype(expected) + dtype = _array_element_dtype(expected) if strided: storage = np.zeros(2 * size, dtype=dtype) return storage[1::2] diff --git a/tests/fortran/allocatables/codegen/test_allocatable_lowering.py b/tests/fortran/allocatables/codegen/test_allocatable_lowering.py index 883b0ad5e..a621d9f36 100644 --- a/tests/fortran/allocatables/codegen/test_allocatable_lowering.py +++ b/tests/fortran/allocatables/codegen/test_allocatable_lowering.py @@ -6,6 +6,7 @@ from prik.policy.completion import complete_semantic_policies from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner +from prik.policy.models import NativeArrayDescriptorKind, NativeDescriptorHandoffABI def _allocatable_plan(): @@ -44,9 +45,15 @@ def test_plain_module_allocatable_uses_standard_descriptor_callback_without_copy bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") assert "void (*callback)(CFI_cdesc_t *, void *)" in c_source - assert "prik_module_allocatable_module_handles_plain_allocatable_descriptor_callback" in c_source - assert "descriptor->base_addr" in c_source - assert "Py_BuildValue" in c_source + assert "prik_module_allocatable_module_handles_plain_allocatable_descriptor_callback_with_descriptor" in c_source + # Every inquiry runs one shared consumer over the descriptor the bridge + # supplies; nothing copies the descriptor out to be read in Python. + assert "prik_native_array_read_shape(void * descriptor, void * context)" in c_source + assert "source->base_addr" in c_source + # The capability tuple is the only Python object the module builds; no + # descriptor field is packed into Python values for the handle to read back. + built = [line.strip() for line in c_source.splitlines() if "Py_BuildValue" in line] + assert built and all("build_capabilities" in line for line in built) assert "subroutine bind_c_plain_allocatable_descriptor(" in bridge_source assert 'bind(c, name="bind_c_plain_allocatable_descriptor")' in bridge_source assert "type(c_funptr), value :: callback_address" in bridge_source @@ -73,9 +80,6 @@ def test_allocated_direct_result_assigns_then_moves_into_owned_descriptor(): assert "deallocate(result)" in procedure assert "call prik_collect_allocatable_array_result(native_make(n), result)" not in procedure assert "result = result_value" not in procedure - assert "function bind_c_owned_result_" in bridge_source - assert "_allocated(" in bridge_source - assert "real(c_double), allocatable, dimension(:), intent(in) :: result" in bridge_source assert "subroutine bind_c_owned_result_" in bridge_source assert "_deallocate(" in bridge_source assert "real(c_double), allocatable, dimension(:), intent(inout) :: result" in bridge_source @@ -117,3 +121,80 @@ def invalid_argument(values: Annotated[Allocatable[Float64[:]], MaybeUnallocated with pytest.raises(ValueError, match="MaybeUnallocated metadata"): complete_semantic_policies(module) + + +def _allocatable_argument_plan(): + module = parse_pyi_text( + """ +from prik.contracts import Allocatable, Float64, native_call, nogil + +@nogil +@native_call([]) +def total(values: Allocatable[Float64[:]]) -> Float64: ... + +plain_allocatable: Allocatable[Float64[:]] +""", + module_name="allocatable_actuals", + ) + complete_semantic_policies(module) + return WrapperPlanner().build(module) + + +def test_no_generated_binding_establishes_an_allocated_allocatable_descriptor(): + """CFI_establish reserves the allocatable descriptor for the Fortran runtime. + + F2018 18.5.5.6 requires a null ``base_addr`` when the attribute is + ``CFI_attribute_allocatable``: an allocatable established from C must start + unallocated. Pairing that attribute with a real address describes an + already-allocated allocatable, which ifx rejects with + ``CFI_ERROR_BASE_ADDR_NOT_NULL`` while gfortran silently accepts it. + """ + artifacts = WrapperGenerator().generate(_allocatable_argument_plan()) + c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") + + forged = [ + line.strip() + for line in c_source.splitlines() + if "CFI_establish(" in line and "CFI_attribute_allocatable" in line and ", NULL," not in line + ] + assert forged == [] + + +def test_allocatable_argument_uses_the_descriptor_the_runtime_built(): + """The binding passes on the runtime's descriptor rather than a record of its own. + + A C descriptor is the Fortran runtime's to build, so the binding neither + establishes one for an allocatable actual nor copies the one it is handed: + the call is made inside the consumer holding it, and only that pointer + crosses. + """ + plan = _allocatable_argument_plan() + functions = {function.binding.python_name: function for function in plan.namespaces[0].functions} + argument = functions["total"].arguments[0] + handle = argument.native_array_handle + + assert handle is not None + assert handle.descriptor_kind is NativeArrayDescriptorKind.ALLOCATABLE + assert handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + + artifacts = WrapperGenerator().generate(plan) + c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") + assert "with_descriptor(" in c_source + copied = [line.strip() for line in c_source.splitlines() if "memcpy(" in line and "CFI_CDESC_T" in line] + assert copied == [] + + +def test_nogil_releases_only_while_the_descriptor_consumer_calls_fortran(): + c_source = next( + source.text + for source in WrapperGenerator().generate(_allocatable_argument_plan()).sources + if source.path.suffix == ".c" + ) + start = c_source.index("static void wrap_total_call_with_descriptor_0(") + end = c_source.index("\n}\n", start) + consumer = c_source[start:end] + + begin = consumer.index("Py_BEGIN_ALLOW_THREADS") + call = consumer.index("bind_c_total(") + finish = consumer.index("Py_END_ALLOW_THREADS") + assert begin < call < finish diff --git a/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py b/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py index b172a4876..afa0099bc 100644 --- a/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py +++ b/tests/fortran/allocatables/end_to_end/test_allocatable_cross_extension.py @@ -1,5 +1,6 @@ """Cross-extension allocatable descriptor and native-memory evidence.""" +import ctypes import shutil import subprocess import sys @@ -109,6 +110,45 @@ def test_caller_created_allocatable_crosses_separately_built_extensions(tmp_path assert values.closed is True +def test_a_backend_capsule_from_another_producer_is_refused_not_interpreted(tmp_path: Path): + """A reader refuses a capsule with another ABI name before reading it.""" + module = _build_text_and_import( + ALLOCATABLE_CROSS_A_SOURCE, + "fallocatable_cross_a.f90", + tmp_path, + { + "bind_c_fallocatable_cross_a_wrapper.f90", + "fallocatable_cross_a_wrapper.c", + "fallocatable_cross_a_wrapper.h", + }, + ) + values = Allocatable[Float64[:]]() + module.select_a(values) + assert module.total_a(values) == np.float64(3.0) + + capsule_new = ctypes.pythonapi.PyCapsule_New + capsule_new.restype = ctypes.py_object + capsule_new.argtypes = (ctypes.c_void_p, ctypes.c_char_p, ctypes.c_void_p) + capsule_get = ctypes.pythonapi.PyCapsule_GetPointer + capsule_get.restype = ctypes.c_void_p + capsule_get.argtypes = (ctypes.py_object, ctypes.c_char_p) + capsule_name = ctypes.pythonapi.PyCapsule_GetName + capsule_name.restype = ctypes.c_char_p + capsule_name.argtypes = (ctypes.py_object,) + + published = capsule_name(values._native_backend) + assert published.startswith(b"prik.native_array_backend.v1.") + address = capsule_get(values._native_backend, published) + assert address + stranger = published[: published.rindex(b".")] + b".0000000000000000" + values._native_backend = capsule_new(address, stranger, None) + + # Only the capsule name differs; the reader must reject it before using the + # live backend address. + with pytest.raises(ValueError, match="PyCapsule_GetPointer called with incorrect name"): + module.total_a(values) + + @pytest.mark.skipif(shutil.which("valgrind") is None, reason="Valgrind is required for native ownership checks") def test_allocatable_replacement_has_no_native_memory_errors( pyi_parity_build_mode: str, diff --git a/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py b/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py index 656d9d96e..5b240a95e 100644 --- a/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py +++ b/tests/fortran/allocatables/end_to_end/test_allocatable_handles.py @@ -11,6 +11,7 @@ from tests.fortran._support.wrapper_build import ( _build_source_or_generated_pyi_and_import, _build_text_and_import, + _compiler, _compile_native_object, _import_from_build_dir, _require_maybe_unallocated_function_result_support, @@ -24,6 +25,8 @@ ALLOCATABLE_VIEW_F90_SOURCE = FIXTURES / "native" / "fallocatable_views_f90.f90" CONTRACT_FIXTURES = FIXTURES / "contracts" pytestmark = pytest.mark.fortran_end_to_end + + PLAIN_ALLOCATABLE_MODULE_SOURCE = """\ module fallocatable_plain_f90 implicit none @@ -91,6 +94,7 @@ def _plain_allocatable_module(build_mode: str, tmp_path: Path): native_object = _compile_native_object(source, tmp_path / "native") result = build_pyi_extension( contract_dir / "__init__.pyi", + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "pyi_build", @@ -318,7 +322,7 @@ def test_plain_allocatable_module_array_exposes_current_live_view( module, wrapper_source_text = _plain_allocatable_module(pyi_parity_build_mode, tmp_path) assert "void (*callback)(CFI_cdesc_t *, void *)" in wrapper_source_text - assert "descriptor->base_addr" in wrapper_source_text + assert "source->base_addr" in wrapper_source_text handle = module.values assert isinstance(handle, AllocatableArray) @@ -359,3 +363,309 @@ def test_plain_allocatable_module_array_exposes_current_live_view( assert handle.allocated is False assert handle.shape is None assert handle.to_numpy() is None + + +LOWER_BOUND_SOURCE = """ +module falloc_lower_bounds_f90 + use iso_fortran_env, only: int32, real64 + implicit none + real(real64), allocatable :: plain_a(:) + real(real64), allocatable, target :: tgt_a(:) + real(real64), allocatable, target :: defaulted(:) + character(len=5), allocatable, target :: fixed_words(:) + character(len=5), allocatable :: missing_words(:) + character(len=5), pointer :: missing_pointer(:) => null() + character(len=:), allocatable, target :: deferred_words(:) +contains + function lower_bound_of(x) result(bound) + real(real64), allocatable, intent(in) :: x(:) + integer(int32) :: bound + bound = lbound(x, 1) + end function lower_bound_of + + function element_at(x, index) result(value) + real(real64), allocatable, intent(in) :: x(:) + integer(int32), intent(in) :: index + real(real64) :: value + value = x(index) + end function element_at + + function deferred_word_bound_and_width(x) result(packed) + character(len=:), allocatable, intent(in) :: x(:) + integer(int32) :: packed + packed = 100 * lbound(x, 1) + len(x) + end function deferred_word_bound_and_width + + subroutine setup() + allocate(plain_a(5:8)) + plain_a = 1.0d0 + allocate(tgt_a(5:8)) + tgt_a = 2.0d0 + allocate(defaulted(4)) + defaulted = 3.0d0 + allocate(character(len=5) :: fixed_words(5:8)) + fixed_words = 'aaaaa' + allocate(character(len=6) :: deferred_words(5:8)) + deferred_words = 'bbbbbb' + end subroutine setup +end module falloc_lower_bounds_f90 +""" + + +def test_module_allocatable_reports_its_real_lower_bound_with_or_without_target(tmp_path: Path): + """A module allocatable reports the bounds it actually has, either way. + + `target` allows `c_loc` on the variable, but that yields only a base address: + the bounds, strides and element length then have to come from somewhere else. + Reconstructing them hardcoded a lower bound of zero, which is wrong for every + Fortran array — the default is one — and further wrong for a declared `(5:8)`. + Both declarations read the descriptor, so both report 5. + """ + module = _build_text_and_import( + LOWER_BOUND_SOURCE, + "falloc_lower_bounds_f90.f90", + tmp_path, + { + "bind_c_falloc_lower_bounds_f90_wrapper.f90", + "falloc_lower_bounds_f90_wrapper.c", + "falloc_lower_bounds_f90_wrapper.h", + }, + ) + module.setup() + + # The bound is not a reported fact but part of the value: an allocatable + # dummy adopts the bounds of the descriptor it is given, so a wrong one + # makes the callee index the wrong elements. + for name, bound in (("plain_a", 5), ("tgt_a", 5), ("defaulted", 1)): + handle = getattr(module, name) + assert module.lower_bound_of(handle) == np.int32(bound), name + assert module.element_at(handle, np.int32(bound)) == handle.to_numpy()[0], name + # The Python view is unaffected: NumPy indexing stays zero-based. + assert handle.to_numpy().shape == (4,) + + # A character allocatable carries the same bounds, and its element length + # comes from the array rather than from a width the binding assumed. + for name, width in (("fixed_words", 5), ("deferred_words", 6)): + assert getattr(module, name).dtype == np.dtype(f"S{width}"), name + # A deferred-length actual reaches an allocatable dummy carrying both, so + # the bound and the width are read back out of the array itself. + assert module.deferred_word_bound_and_width(module.deferred_words) == np.int32(506) + + +def test_fixed_character_projection_reports_absence(tmp_path: Path): + module = _build_text_and_import( + LOWER_BOUND_SOURCE, + "falloc_lower_bounds_f90.f90", + tmp_path, + { + "bind_c_falloc_lower_bounds_f90_wrapper.f90", + "falloc_lower_bounds_f90_wrapper.c", + "falloc_lower_bounds_f90_wrapper.h", + }, + ) + module.setup() + + missing = module.missing_words + assert missing.allocated is False + assert missing.shape is None + assert missing.to_numpy() is None + assert missing.dtype == np.dtype("S5") + + missing_pointer = module.missing_pointer + assert missing_pointer.associated is False + assert missing_pointer.shape is None + missing_pointer.associate(missing_pointer) + assert missing_pointer.associated is False + + fixed = module.fixed_words + assert fixed.allocated is True + assert fixed.to_numpy().tolist() == [b"aaaaa"] * 4 + with pytest.raises(TypeError, match="descriptor attribute required by the dummy"): + module.deferred_word_bound_and_width(fixed) + + +BORROWED_DESCRIPTOR_SOURCE = """\ +module fallocatable_borrowed_f90 + implicit none + type :: box + real(8), allocatable :: field(:) + end type box + real(8), allocatable :: modvar(:) + type(box) :: thebox +contains + subroutine grow(values) + real(8), allocatable, intent(inout) :: values(:) + + if (allocated(values)) deallocate(values) + allocate(values(6)) + values = 9.0_8 + end subroutine grow + + function total(values) result(sum_out) + real(8), allocatable, intent(in) :: values(:) + real(8) :: sum_out + + sum_out = sum(values) + end function total + + function make(n) result(values) + integer(4), intent(in) :: n + real(8), allocatable :: values(:) + integer(4) :: i + + allocate(values(n)) + values = [(1.0_8 * i, i = 1, n)] + end function make +end module fallocatable_borrowed_f90 +""" + + +def test_every_allocatable_handle_kind_reaches_a_read_only_allocatable_dummy(tmp_path: Path): + """An allocatable actual borrows the descriptor the Fortran runtime built. + + A read-only allocatable dummy requires an allocatable actual, and C may not + establish one: F2018 18.5.5.6 reserves that descriptor for the runtime. The + binding therefore copies the descriptor handed to its callback rather than + rebuilding one from facts, so module, derived-field and result handles all + reach the dummy on every compiler instead of only where an invalid + descriptor happens to be tolerated. + """ + workdir = tmp_path / "borrowed" + workdir.mkdir(parents=True) + module = _build_text_and_import( + BORROWED_DESCRIPTOR_SOURCE, + "fallocatable_borrowed_f90.f90", + workdir, + { + "bind_c_fallocatable_borrowed_f90_wrapper.f90", + "fallocatable_borrowed_f90_wrapper.c", + "fallocatable_borrowed_f90_wrapper.h", + }, + ) + namespace = _sole_native_module(module) + + namespace.modvar.resize(4) + namespace.modvar.to_numpy()[:] = [1.0, 2.0, 3.0, 4.0] + namespace.thebox.field.resize(4) + namespace.thebox.field.to_numpy()[:] = [1.0, 2.0, 3.0, 4.0] + + assert namespace.total(namespace.modvar) == np.float64(10.0) + assert namespace.total(namespace.thebox.field) == np.float64(10.0) + assert namespace.total(namespace.make(np.int32(4))) == np.float64(10.0) + + # The copy is remade per call, so reallocating the native entity between + # calls cannot leave the previous descriptor behind. + namespace.modvar.resize(3) + namespace.modvar.to_numpy()[:] = [100.0, 200.0, 300.0] + assert namespace.total(namespace.modvar) == np.float64(600.0) + + +def test_a_writable_allocatable_dummy_reaches_the_callers_entity(tmp_path: Path): + """A callee that reallocates an ``intent(inout)`` dummy updates the caller. + + The descriptor a module array or field hands out exists only while the + consumer holding it runs, so a callee handed a copy would reallocate the + copy and leave the caller's entity naming released storage. The call is + made inside that consumer instead, which is what lets the new allocation + reach the entity. A handle owning its descriptor hands that over directly + and needs no such arrangement. + """ + workdir = tmp_path / "writable" + workdir.mkdir(parents=True) + module = _build_text_and_import( + BORROWED_DESCRIPTOR_SOURCE, + "fallocatable_borrowed_f90.f90", + workdir, + { + "bind_c_fallocatable_borrowed_f90_wrapper.f90", + "fallocatable_borrowed_f90_wrapper.c", + "fallocatable_borrowed_f90_wrapper.h", + }, + ) + namespace = _sole_native_module(module) + + # A module array: the callee replaces the allocation, and the module + # variable names the new one afterwards. + namespace.modvar.resize(2) + namespace.modvar.to_numpy()[:] = [1.0, 2.0] + namespace.grow(namespace.modvar) + assert namespace.modvar.shape == (6,) + assert namespace.modvar.to_numpy().tolist() == [9.0] * 6 + + # A derived-type field reaches its entity the same way, through its parent. + namespace.thebox.field.resize(2) + namespace.thebox.field.to_numpy()[:] = [1.0, 2.0] + namespace.grow(namespace.thebox.field) + assert namespace.thebox.field.shape == (6,) + assert namespace.thebox.field.to_numpy().tolist() == [9.0] * 6 + + # A handle owning its descriptor hands that over directly. + owned = namespace.make(np.int32(2)) + namespace.grow(owned) + assert owned.shape == (6,) + assert owned.to_numpy().tolist() == [9.0] * 6 + + +EMPTY_ACTUAL_SOURCE = """\ +module fallocatable_empty_actual_f90 + implicit none + +contains + + subroutine fill_empty(values) + real(8), allocatable, intent(inout) :: values(:) + + if (allocated(values)) deallocate(values) + allocate(values(0)) + end subroutine fill_empty + + subroutine fill_three(values) + real(8), allocatable, intent(inout) :: values(:) + + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = [1.0_8, 2.0_8, 3.0_8] + end subroutine fill_three + + ! An ordinary explicit-shape dummy: it receives an address and an extent. + function total(values, n) result(sum_values) + integer, intent(in) :: n + real(8), intent(in) :: values(n) + real(8) :: sum_values + + sum_values = sum(values) + end function total + +end module fallocatable_empty_actual_f90 +""" + + +def test_zero_sized_allocatable_handle_reaches_an_ordinary_array_dummy(tmp_path: Path): + """An allocated but empty array is present, and its extent is zero. + + A compiler may describe an empty dimension with an extent of -1, so the + extent an ordinary dummy receives has to be normalised. Passing the raw + value makes a zero-sized actual look like a shape mismatch. + """ + build_dir = tmp_path / "build" + build_dir.mkdir(parents=True) + module = _build_text_and_import( + EMPTY_ACTUAL_SOURCE, + "fallocatable_empty_actual_f90.f90", + build_dir, + { + "bind_c_fallocatable_empty_actual_f90_wrapper.f90", + "fallocatable_empty_actual_f90_wrapper.c", + "fallocatable_empty_actual_f90_wrapper.h", + }, + ) + + handle = Allocatable[Float64[:]]() + module.fill_empty(handle) + assert handle.allocated is True + assert handle.shape == (0,) + assert module.total(handle, np.int32(0)) == np.float64(0.0) + + module.fill_three(handle) + assert handle.shape == (3,) + assert module.total(handle, np.int32(3)) == np.float64(6.0) diff --git a/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py b/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py index 4184c58c1..e22ea634d 100644 --- a/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py +++ b/tests/fortran/allocatables/policy/test_allocatable_handle_policy.py @@ -87,6 +87,13 @@ def replace_values( def test_aliased_does_not_change_allocatable_live_view_semantics(): + """A module allocatable is reached through its descriptor either way. + + `Aliased` would allow `c_loc` on the variable, but that yields only a base + address: lower bounds, strides and element length would then have to be + assumed rather than read, and a non-default lower bound makes the assumption + wrong. Both declarations therefore complete to the same descriptor policy. + """ module = parse_pyi_text( """ values: Allocatable[Float64[:]] @@ -100,12 +107,10 @@ def test_aliased_does_not_change_allocatable_live_view_semantics(): values = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] shared_values = module.variables[1].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] - assert values.to_numpy == "descriptor_view" - assert shared_values.to_numpy == "borrowed_view" + assert values.to_numpy == shared_values.to_numpy == "descriptor_view" + assert values.descriptor_interop == shared_values.descriptor_interop == "module_allocatable_c_descriptor" assert values.owner == shared_values.owner == "native" assert values.borrowed is shared_values.borrowed is True - assert values.descriptor_interop == "module_allocatable_c_descriptor" - assert shared_values.descriptor_interop == "none" def test_owned_allocatable_result_records_local_standard_c_descriptor_build_requirement(): diff --git a/tests/fortran/allocatables/runtime/test_allocatable_array_actual_abi.py b/tests/fortran/allocatables/runtime/test_allocatable_array_actual_abi.py deleted file mode 100644 index fbb199ca8..000000000 --- a/tests/fortran/allocatables/runtime/test_allocatable_array_actual_abi.py +++ /dev/null @@ -1,72 +0,0 @@ -"""Allocatable handle handoff to ordinary native array arguments.""" - -import numpy as np -import pytest -from prik.runtime.handles import ( - AllocatableArray, - _native_array_actual_argument_for_binding_positional, -) -from tests.fortran._support.native_array_handles import ( - _ArrayState, - _handoff, -) - - -def test_allocatable_array_actual_hook_requires_allocated_state_without_to_numpy(): - actual = _handoff(201) - calls = [] - state = _ArrayState() - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(202), - "shape": lambda _handle: state.shape, - "allocated": lambda _handle: state.shape is not None, - "to_numpy": lambda _handle: pytest.fail("array-actual handoff must not call to_numpy"), - "array_actual": lambda _handle: calls.append("array_actual") or actual, - }, - ) - - with pytest.raises(ValueError, match="unallocated"): - handle._array_actual_for_binding(expected_dtype=np.float64, expected_rank=1) - - state.shape = (4,) - - assert handle._array_actual_for_binding(expected_dtype="float64", expected_rank=1) is actual - assert calls == ["array_actual"] - - -def test_array_actual_argument_abi_packer_uses_allocatable_native_array_actual_without_numpy_conversion(): - actual = _handoff(246) - calls = [] - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(247), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "layout": lambda _handle: "F", - "writeable": lambda _handle: True, - "native_byte_order": lambda _handle: True, - "aligned": lambda _handle: True, - "to_numpy": lambda _handle: pytest.fail("array-actual ABI packing must not call to_numpy"), - "array_actual": lambda _handle: calls.append("array_actual") or actual, - }, - ) - - assert _native_array_actual_argument_for_binding_positional( - handle, - "float64", - 1, - (2,), - "F", - True, - True, - True, - True, - True, - True, - ) == (actual.address, 1, np.dtype(np.float64).itemsize, 2, 1, 1) - assert calls == ["array_actual"] diff --git a/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py b/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py index b20032092..5e0600c72 100644 --- a/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py +++ b/tests/fortran/allocatables/runtime/test_allocatable_contract_handles.py @@ -7,20 +7,28 @@ from prik.runtime.handles import ( AllocatableArray, _bind_contract_native_array_handle, - _native_array_descriptor_argument_for_binding, - _native_array_descriptor_handoff_for_binding, + _native_array_backend_for_binding, ) +from tests.fortran._support.native_array_handles import _generated_handle_dispatch, _handle_dispatch -def test_fresh_contract_handle_supplies_present_empty_read_only_descriptor_facts(): +def test_fresh_contract_handle_has_no_descriptor_to_hand_over_on_its_own(): + """A handle the caller made owns no descriptor until the wrapper gives it one. + + Nothing rebuilds a descriptor from reported fields any more, so the only + thing such a handle can supply is storage a generated binder attached to + it. Reaching the call without that is a wrapper bug, not a caller error, + so it is refused rather than papered over with an empty descriptor. + """ handle = contracts.Allocatable[contracts.Float64[:]]() - assert _native_array_descriptor_argument_for_binding( - handle, - descriptor_kind="allocatable", - expected_dtype=np.float64, - expected_rank=1, - ) == (0, 8, 1, 0, 0, 8) + with pytest.raises(TypeError, match="requires generated persistent descriptor storage"): + _native_array_backend_for_binding( + handle, + descriptor_kind="allocatable", + expected_dtype=np.float64, + expected_rank=1, + ) def test_contract_default_allocatable_constructor_preserves_dtype_rank_and_empty_state(): @@ -60,12 +68,12 @@ def test_non_array_allocatable_annotations_are_not_factories(factory, message: s lambda: AllocatableArray( dtype="float64", rank=1, - ops={ - "shape": lambda _handle: None, - "array_actual": lambda _handle: None, - "descriptor": lambda _handle: None, - "allocated": lambda _handle: False, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "allocated": lambda _handle: False, + } + ), to_numpy_policy="unsupported", ), "float64", @@ -99,12 +107,14 @@ def test_generated_storage_rejects_incompatible_allocatable_contract_handles( handle = prepare() with pytest.raises(error, match=message): + operations = {} _bind_contract_native_array_handle( handle, "allocatable", dtype, rank, - {}, + _generated_handle_dispatch(operations), + operations, object(), "owned", "unsupported", @@ -112,29 +122,36 @@ def test_generated_storage_rejects_incompatible_allocatable_contract_handles( def test_writable_contract_handle_adopts_generated_storage_and_closes_once(): + """Binding attaches storage, and the backend over it is what goes to the call. + + An owned handle's backend and its owner are the same capsule: the record + holds the descriptor the binder allocated, and every later call reads it + without coming back through Python. + """ handle = contracts.Allocatable[contracts.Float64[:]]() calls = [] owner = object() def bind_default(value): + operations = { + "shape": lambda received_owner: calls.append(("shape", received_owner)) or None, + "allocated": lambda received_owner: False, + "destroy": lambda received_owner: calls.append(("destroy", received_owner)), + } _bind_contract_native_array_handle( value, "allocatable", "float64", 1, - { - "shape": lambda received_owner: calls.append(("shape", received_owner)) or None, - "array_actual": lambda received_owner: 0x5678, - "descriptor": lambda received_owner: received_owner, - "allocated": lambda received_owner: False, - "destroy": lambda received_owner: calls.append(("destroy", received_owner)), - }, + _generated_handle_dispatch(operations), + operations, owner, "owned", "unsupported", + native_backend=owner, ) - assert _native_array_descriptor_handoff_for_binding( + assert _native_array_backend_for_binding( handle, descriptor_kind="allocatable", expected_dtype=np.float64, @@ -153,12 +170,14 @@ def test_generated_storage_rejects_a_closed_contract_handle(): handle.close() with pytest.raises(ReferenceError, match="handle is closed"): + operations = {} _bind_contract_native_array_handle( handle, "allocatable", "float64", 1, - {}, + _generated_handle_dispatch(operations), + operations, object(), "owned", "unsupported", diff --git a/tests/fortran/allocatables/runtime/test_allocatable_descriptor_abi.py b/tests/fortran/allocatables/runtime/test_allocatable_descriptor_abi.py deleted file mode 100644 index 77c093878..000000000 --- a/tests/fortran/allocatables/runtime/test_allocatable_descriptor_abi.py +++ /dev/null @@ -1,31 +0,0 @@ -"""Allocatable descriptor handoff through the runtime ABI.""" - -import numpy as np -import pytest -from prik.runtime.handles import AllocatableArray -from tests.fortran._support.native_array_handles import _handoff - - -def test_allocatable_descriptor_hook_accepts_unallocated_descriptor_without_numpy_conversion(): - descriptor = _handoff(234) - calls = [] - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - "to_numpy": lambda _handle: pytest.fail("descriptor handoff must not call to_numpy"), - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "descriptor": lambda _handle: calls.append("descriptor") or descriptor, - }, - to_numpy_policy="unsupported", - ) - - assert handle._descriptor_for_binding(expected_dtype="float64", expected_rank=1) == { - "base_addr": descriptor.address, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 0, "sm": 8}], - } - assert calls == ["descriptor"] diff --git a/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py b/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py index 0b12b6abe..b58b89787 100644 --- a/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py +++ b/tests/fortran/allocatables/runtime/test_allocatable_handle_protocol.py @@ -9,7 +9,7 @@ from tests.fortran._support.native_array_handles import ( _ArrayState, _common_ops, - _required_handoff_ops, + _handle_dispatch, ) @@ -26,7 +26,7 @@ def test_allocatable_handle_uses_common_metadata_shape_owner_and_numpy_dispatch( handle = AllocatableArray( dtype="float64", rank=2, - ops=ops, + **_handle_dispatch(ops), owner=owner, descriptor_ownership="borrowed", generation=7, @@ -47,16 +47,22 @@ def test_allocatable_handle_uses_common_metadata_shape_owner_and_numpy_dispatch( assert handle.allocated is True -def test_allocatable_to_numpy_short_circuits_unallocated_state_before_generated_extraction(): +def test_allocatable_extraction_reports_unallocated_state_as_no_view(): + """Absence is reported by the extraction, not asked about beforehand. + + The generated extraction reads the descriptor, which is where whether the + storage exists is recorded, so nothing has to test allocation first. + """ handle = AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: pytest.fail("unallocated handles must not call generated extraction"), - "allocated": lambda _handle: False, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "to_numpy": lambda _handle: None, + "allocated": lambda _handle: pytest.fail("extraction must not need the allocation state"), + } + ), ) assert handle.to_numpy() is None @@ -70,7 +76,7 @@ def test_allocatable_handle_reports_absent_state_and_routes_resize_deallocate(): "deallocate": lambda _handle: setattr(state, "shape", None), "resize": lambda _handle, shape: setattr(state, "shape", shape), } - handle = AllocatableArray(dtype="float64", rank=1, ops=ops) + handle = AllocatableArray(dtype="float64", rank=1, **_handle_dispatch(ops)) assert handle.allocated is False assert handle.shape is None @@ -91,12 +97,14 @@ def test_allocatable_to_numpy_policy_returns_mutable_borrowed_view(): handle = AllocatableArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_common_ops(state), - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, + **_handle_dispatch( + { + **_common_ops(state), + "allocated": lambda _handle: True, + "deallocate": lambda _handle: None, + "resize": lambda _handle, _shape: None, + } + ), to_numpy_policy="borrowed_view", ) @@ -114,12 +122,14 @@ def test_allocatable_to_numpy_explicit_copy_is_independent(): handle = AllocatableArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_common_ops(state), - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, + **_handle_dispatch( + { + **_common_ops(state), + "allocated": lambda _handle: True, + "deallocate": lambda _handle: None, + "resize": lambda _handle, _shape: None, + } + ), to_numpy_policy="descriptor_view", ) @@ -138,23 +148,25 @@ def test_allocatable_handle_requires_generated_allocated_operation(): AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "to_numpy": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "to_numpy": lambda _handle: None, + } + ), ) -def test_allocatable_operations_are_gated_by_the_completed_ops_table(): +def test_allocatable_operations_are_gated_by_completed_capabilities(): handle = AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "allocated": lambda _handle: False, + } + ), to_numpy_policy="unsupported", ) @@ -169,11 +181,12 @@ def test_close_is_a_noop_for_a_borrowed_allocatable_handle(): handle = AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "allocated": lambda _handle: False, + } + ), owner=owner, descriptor_ownership="borrowed", to_numpy_policy="unsupported", diff --git a/tests/fortran/arrays/codegen/test_array_buffer_lowering.py b/tests/fortran/arrays/codegen/test_array_buffer_lowering.py index 4c7c36390..01f2900cb 100644 --- a/tests/fortran/arrays/codegen/test_array_buffer_lowering.py +++ b/tests/fortran/arrays/codegen/test_array_buffer_lowering.py @@ -16,7 +16,12 @@ TransferMode, ) from prik.policy.completion import complete_semantic_policies -from prik.policy.models import ArgumentHandoffMode, BridgeDataAction +from prik.policy.models import ( + ArgumentHandoffMode, + ArrayEntrypointABI, + BridgeDataAction, + EntrypointPassingConvention, +) from prik.pipeline.wrapper import WrapperGenerator from prik.planning import ArrayHandoffPlan, WrapperPlanner from prik.planning.models import DatatypeFamily @@ -59,6 +64,9 @@ def test_required_array_buffer_has_one_printable_editable_handoff_plan(): assert argument.array.shape == (":",) assert argument.array.axes == ("dense",) assert argument.array.contiguous is True + assert argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + assert argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER + assert argument.entrypoint.pass_array_metadata is False assert argument.array.flatten_python_storage is False assert argument.array.flat_axis is None assert argument.array.data_role == argument.entrypoint.handoff_role @@ -73,24 +81,13 @@ def test_required_array_buffer_dispatches_through_named_binding_and_bridge_metho c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert "double bind_c_sum_values(void * values, int64_t values_extent_0);" in c_source - # One shared binder call carries every completed selector: dtype, rank - # bounds, layout, contiguity, writeability, diagnostic names, and the nine - # selectors the native-handle route consumes. - assert ( - "prik_bind_array(bound_values_obj, NPY_FLOAT64, 1, 1, 1, PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS, " - '1, 1, "numpy.float64", "float64", "values", NULL, 0, 1, 1, 1, 0, 0, 0, 1, 0, -1, ' - "bound_values_bind_fixed, &bound_values, bound_values_bind_extents)" - ) in c_source - assert c_source.count("prik_bind_array(bound_values_obj") == 1 - assert "bound_values_bind_fixed[0] = -1;" in c_source - assert "bound_values_extent_0 = bound_values_bind_extents[0];" in c_source - assert "result = bind_c_sum_values(bound_values, bound_values_extent_0);" in c_source - - assert "type(c_ptr), value :: bound_values" in bridge_source - assert "integer(c_int64_t), value :: values_extent_0" in bridge_source - assert "real(c_double), pointer, contiguous, dimension(:) :: values" in bridge_source - assert "call c_f_pointer(bound_values, values, [values_extent_0])" in bridge_source + assert "double bind_c_sum_values(CFI_cdesc_t * values);" in c_source + assert "prik_describe_numpy_array((CFI_cdesc_t *)&bound_values_parent" in c_source + assert "call->result = bind_c_sum_values(call->descriptor_0);" in c_source + assert "bound_values_bind_fixed" not in c_source + + assert "real(c_double), dimension(:) :: values" in bridge_source + assert "call c_f_pointer(bound_values" not in bridge_source assert "result = native_sum_values(values)" in bridge_source diff --git a/tests/fortran/arrays/codegen/test_array_output_identity.py b/tests/fortran/arrays/codegen/test_array_output_identity.py index 9080447a5..51c5eb82b 100644 --- a/tests/fortran/arrays/codegen/test_array_output_identity.py +++ b/tests/fortran/arrays/codegen/test_array_output_identity.py @@ -86,35 +86,44 @@ def test_projected_array_lowering_increfs_original_objects_and_reuses_tuple_aggr assert "PyTuple_SET_ITEM(result_obj, 1, result_1_obj)" in c_source -def test_mutable_bool_array_writeback_normalizes_the_aliased_numpy_buffer_in_place(): +def test_mutable_bool_array_writeback_needs_no_normalization(): + """A Boolean array is written back like any other element type. + + Its elements already hold the zero or one a C `_Bool` is defined to hold, + because the compiler profiles request the option that guarantees it, so the + callee leaves nothing behind that has to be reduced afterwards. + """ plan = _logical_output_plan() - arguments = plan.namespaces[0].functions[0].arguments - values, out = arguments[1:] + values, out = plan.namespaces[0].functions[0].arguments[1:] - assert values.array_writeback_abi is ArrayWritebackABI.LOGICAL_LOW_BIT_INT8 - assert out.array_writeback_abi is ArrayWritebackABI.LOGICAL_LOW_BIT_INT8 + assert values.array_writeback_abi is ArrayWritebackABI.NATIVE_ARRAY + assert out.array_writeback_abi is ArrayWritebackABI.NATIVE_ARRAY artifacts = WrapperGenerator().generate(plan) bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert "integer(c_int8_t), pointer, dimension(:) :: out_logical_bytes" in bridge_source assert "call native_invert_flags(n, values, out)" in bridge_source - assert "call c_f_pointer(bound_out, out_logical_bytes, [out_extent_0])" in bridge_source - assert "out_logical_bytes = iand(out_logical_bytes, 1_c_int8_t)" in bridge_source + assert "_logical_bytes" not in bridge_source + assert "iand(" not in bridge_source + +def test_high_rank_bool_array_bridge_stays_inside_the_fortran_line_limit(): + """Free-form Fortran caps a line at 132 columns, whatever the rank. -def test_high_rank_bool_array_writeback_wraps_the_flattened_shape_product(): + A rank-15 descriptor dummy is the longest ordinary array declaration PRIK + emits and is where continuation would first be missed. + """ artifacts = WrapperGenerator().generate(_high_rank_logical_output_plan()) bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert "values_extent_0 * &" in bridge_source - assert "& values_extent_14])" in bridge_source + assert "dimension(:, :, :, :, :, :, :, :, :, :, :, :, :, :, :) :: values" in bridge_source assert max(map(len, bridge_source.splitlines())) <= 132 -def test_generator_rejects_a_non_normalized_mutable_bool_array_writeback_abi(): +def test_generator_rejects_a_normalized_mutable_bool_array_writeback_abi(): + """An edited plan cannot reintroduce a normalization pass that is not needed.""" plan = _logical_output_plan() - plan.namespaces[0].functions[0].arguments[-1].array_writeback_abi = ArrayWritebackABI.NATIVE_ARRAY + plan.namespaces[0].functions[0].arguments[-1].array_writeback_abi = ArrayWritebackABI.LOGICAL_LOW_BIT_INT8 with pytest.raises(ValueError, match="invalid-array-writeback-abi"): WrapperGenerator().generate(plan) diff --git a/tests/fortran/arrays/codegen/test_array_result_lowering.py b/tests/fortran/arrays/codegen/test_array_result_lowering.py index ca852a2e3..5b10e3541 100644 --- a/tests/fortran/arrays/codegen/test_array_result_lowering.py +++ b/tests/fortran/arrays/codegen/test_array_result_lowering.py @@ -113,9 +113,10 @@ def test_array_property_results_reuse_input_array_extent_roles_in_both_backends( assert "npy_intp result_obj_dims[] = {bound_values_extent_0};" in c_source assert "npy_intp result_obj_dims[] = {bound_values_extent_0 * bound_values_extent_1};" in c_source - assert "real(c_double), dimension(values_extent_0) :: result_value" in bridge_source - assert "dimension(values_extent_0 * values_extent_1) :: result_value" in bridge_source - assert "real(c_double), dimension(values_extent_1) :: result_value" in bridge_source + assert bridge_source.count("real(c_double), allocatable, dimension(:) :: result_value") == 3 + assert "allocate(result_value(size(values, 1)))" in bridge_source + assert "allocate(result_value(size(values, 1) * size(values, 2)))" in bridge_source + assert "allocate(result_value(size(values, 2)))" in bridge_source @pytest.mark.parametrize( diff --git a/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py b/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py index a308a2811..58f492838 100644 --- a/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py +++ b/tests/fortran/arrays/codegen/test_dense_array_shape_lowering.py @@ -181,12 +181,12 @@ def test_dense_array_lowering_uses_planned_shape_checks_and_bridge_orientation() assert "bound_values_bind_fixed[0] = -1;" in c_source assert ( "prik_bind_array(bound_values_obj, NPY_FLOAT64, 1, 1, 15, PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS, " - '1, 1, "numpy.float64", "float64", "values", NULL, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, ' + '1, 1, "numpy.float64", "values", 0, ' "bound_values_bind_fixed, &bound_values, bound_values_bind_extents)" ) in c_source assert ( "prik_bind_array(bound_values_obj, NPY_FLOAT64, 2, 2, 15, PRIK_ARRAY_LAYOUT_F_CONTIGUOUS, " - '1, 1, "numpy.float64", "float64", "values", "F", 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, ' + '1, 1, "numpy.float64", "values", 1, ' "bound_values_bind_fixed, &bound_values, bound_values_bind_extents)" ) in c_source assert "call c_f_pointer(bound_values, values, [values_extent_0, values_extent_1])" in bridge_source diff --git a/tests/fortran/arrays/codegen/test_specialized_array_roles.py b/tests/fortran/arrays/codegen/test_specialized_array_roles.py index 2fcb78f90..d1266b245 100644 --- a/tests/fortran/arrays/codegen/test_specialized_array_roles.py +++ b/tests/fortran/arrays/codegen/test_specialized_array_roles.py @@ -5,7 +5,12 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies -from prik.policy.models import OptionalMode +from prik.policy.models import ( + ArrayEntrypointABI, + EntrypointPassingConvention, + NativeArraySourceKind, + OptionalMode, +) from prik.codegen import CBindingGenerator from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -19,6 +24,7 @@ def _later_array_plan(): def optional(values: Float64[:] = ...) -> None: ... def any_rank(values: Float64[...]) -> Float64: ... def labels(values: String[8][:]) -> None: ... +def labels_any_width(values: String[...][:]) -> None: ... """, module_name="later_array_buffers", ) @@ -43,20 +49,43 @@ def hidden_labels() -> String[4][2]: ... def test_optional_assumed_rank_and_character_arrays_have_explicit_distinct_roles(): functions = {function.binding.python_name: function for function in _later_array_plan().namespaces[0].functions} optional = functions["optional"].arguments[0] - assumed = functions["any_rank"].arguments[0].array - character = functions["labels"].arguments[0].array + assumed_argument = functions["any_rank"].arguments[0] + assumed = assumed_argument.array + character_argument = functions["labels"].arguments[0] + character = character_argument.array + assumed_width_argument = functions["labels_any_width"].arguments[0] + handle_sources = ( + NativeArraySourceKind.NDARRAY, + NativeArraySourceKind.ALLOCATABLE_HANDLE, + NativeArraySourceKind.POINTER_HANDLE, + ) assert optional.binding.optional_mode is OptionalMode.NULLABLE_VALUE assert optional.entrypoint.optional_mode is OptionalMode.NULLABLE_VALUE + assert optional.native_array_actual is not None + assert optional.native_array_actual.accepted_sources == handle_sources assert assumed is not None assert assumed.rank is None - assert assumed.contiguous is True + assert assumed.contiguous is False + assert assumed.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + assert assumed.signed_strides is True + assert assumed_argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER + assert assumed_argument.entrypoint.pass_array_metadata is False assert assumed.runtime_rank_role == "later_array_buffers.any_rank.values:rank" assert len(assumed.extent_roles) == 15 + assert assumed_argument.native_array_actual is not None + assert assumed_argument.native_array_actual.rank is None + assert assumed_argument.native_array_actual.accepted_sources == handle_sources assert character is not None assert character.rank == 1 + assert character.entrypoint_abi is ArrayEntrypointABI.RAW_ADDRESS assert character.itemsize == 8 assert character.itemsize_role == "later_array_buffers.labels.values:itemsize" + assert character_argument.native_array_actual is not None + assert character_argument.native_array_actual.accepted_sources == handle_sources + assert assumed_width_argument.native_array_actual is not None + assert assumed_width_argument.native_array_actual.dtype == "S" + assert assumed_width_argument.native_array_actual.accepted_sources == handle_sources def test_optional_assumed_rank_and_character_lowering_follow_named_plan_fields(): @@ -66,20 +95,22 @@ def test_optional_assumed_rank_and_character_lowering_follow_named_plan_fields() assert "PyObject * bound_values_obj = Py_None;" in c_source assert "if (bound_values_obj != Py_None)" in c_source - assert "prik_array_validate(bound_values_obj, NPY_FLOAT64, 1, 15, PRIK_ARRAY_LAYOUT_F_CONTIGUOUS" in c_source - assert "NPY_FLOAT64, 1, 15, PRIK_ARRAY_LAYOUT_F_CONTIGUOUS" in c_source + assert ("prik_array_validate(bound_values_obj, NPY_FLOAT64, 1, 15, PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F") in c_source + assert ( + "prik_native_array_backend_for_actual(bound_values_capsule, 1, 15, " + 'CFI_type_double, sizeof(double), "float64", "values")' + ) in c_source + assert "NPY_FLOAT64, 1, 15, PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F" in c_source assert "bound_values_rank = (int64_t)PyArray_NDIM" in c_source assert "NPY_STRING, 1, 1, PRIK_ARRAY_LAYOUT_ANY_CONTIGUOUS" in c_source assert "bound_values_itemsize != 8" in c_source - assert "if (c_associated(bound_values)) then" in bridge_source - assert "select case (values_rank)" in bridge_source - assert "case (1)" in bridge_source - assert "case (15)" in bridge_source + assert "real(c_double), dimension(..) :: values" in bridge_source + assert "select case (values_rank)" not in bridge_source assert "character(kind=c_char, len=8), pointer, contiguous, dimension(:) :: values" in bridge_source assert max(map(len, bridge_source.splitlines())) <= 132 -def test_native_array_fallback_unpacks_planned_runtime_rank_and_itemsize_roles(): +def test_native_array_descriptor_result_unpacks_planned_runtime_rank_and_itemsize_roles(): functions = {function.binding.python_name: function for function in _later_array_plan().namespaces[0].functions} generator = CBindingGenerator() diff --git a/tests/fortran/arrays/codegen/test_strided_array_lowering.py b/tests/fortran/arrays/codegen/test_strided_array_lowering.py index ef8e846cc..2fa0c788c 100644 --- a/tests/fortran/arrays/codegen/test_strided_array_lowering.py +++ b/tests/fortran/arrays/codegen/test_strided_array_lowering.py @@ -1,4 +1,4 @@ -"""Positive-strided ordinary array view lowering.""" +"""Signed-stride ordinary array view lowering.""" from __future__ import annotations @@ -6,6 +6,7 @@ from tests.fortran._support.ownership_policy import parse_pyi_text from prik.policy.completion import complete_semantic_policies +from prik.policy.models import ArrayEntrypointABI, EntrypointPassingConvention from prik.pipeline.wrapper import WrapperGenerator from prik.planning import WrapperPlanner @@ -24,7 +25,7 @@ def strided(values: Float64[{dimensions}]) -> None: ... return WrapperPlanner().build(module) -def test_strided_array_plan_names_bounds_and_element_strides_explicitly(): +def test_strided_array_plan_selects_one_descriptor_without_parallel_stride_roles(): argument = _strided_plan().namespaces[0].functions[0].arguments[0] array = argument.array @@ -32,71 +33,69 @@ def test_strided_array_plan_names_bounds_and_element_strides_explicitly(): assert array.rank == 2 assert array.axes == ("strided", "strided") assert array.contiguous is False - assert array.upper_bound_roles == ( - f"{argument.owner_path}:upper-bound:0", - f"{argument.owner_path}:upper-bound:1", - ) - assert array.stride_roles == ( - f"{argument.owner_path}:stride:0", - f"{argument.owner_path}:stride:1", - ) - assert array.dense_actual_role == f"{argument.owner_path}:dense-actual" - - -def test_strided_array_lowering_validates_and_passes_one_explicit_bridge_slice(): + assert array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + assert array.signed_strides is True + assert argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER + assert argument.entrypoint.pass_array_metadata is False + assert array.upper_bound_roles == () + assert array.stride_roles == () + assert array.dense_actual_role is None + + +def test_strided_array_lowering_hands_over_one_descriptor_from_either_source(): + """A strided dummy is reached by a descriptor, whoever supplied the array. + + This is the direct-entrypoint answer for an assumed-shape dummy: a bind(C) + procedure with no bridge receives a ``CFI_cdesc_t *``, and the extents and + signed strides travel inside it. So the generated C describes a NumPy array + into one and enters a handle's own, and the bridge dummy is the array + itself -- there is nothing left to reconstruct on the Fortran side. + """ artifacts = WrapperGenerator().generate(_strided_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert 'prik_array_actual_unpack(bound_values_obj, "float64", 2, bound_values_shape, "F"' in c_source - assert "NPY_FLOAT64, 2, 2, PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F, 0, 1" in c_source - assert "bound_values_upper_bound_0 = bound_values_actual.upper_bounds[0]" in c_source - assert "bound_values_stride_1 = bound_values_actual.strides[1]" in c_source - assert "bound_values_upper_bound_0" in c_source - assert "bound_values_stride_1" in c_source - assert "int bound_values_dense_actual = 0;" in c_source - assert "bound_values_dense_actual = PyArray_IS_F_CONTIGUOUS" in c_source - assert "if (!bound_values_dense_actual) {" in c_source + # A handle is entered through its own descriptor entry point. assert ( - "bind_c_strided(bound_values, bound_values_dense_actual, bound_values_extent_0, bound_values_extent_1," - in c_source + "prik_native_array_backend_for_actual(bound_values_capsule, 2, 2, " + 'CFI_type_double, sizeof(double), "float64", "values")' + ) in c_source + # A NumPy array has none, so one is built over its storage as it stands. + assert "prik_describe_numpy_array((CFI_cdesc_t *)&bound_values_parent" in c_source + assert "CFI_section(section, parent, lower, upper, step)" in c_source + # Signed strides are what this layout accepts now. + assert "NPY_FLOAT64, 2, 2, PRIK_ARRAY_LAYOUT_SIGNED_STRIDED_F, 0, 1" in c_source + # One descriptor crosses, not an address with extents beside it. + assert "double bind_c_strided(CFI_cdesc_t * values)" in c_source or ( + "void bind_c_strided(CFI_cdesc_t * values)" in c_source ) - assert "integer(c_int), value :: values_dense_actual" in bridge_source - assert "real(c_double), pointer, dimension(:, :) :: values_base" in bridge_source - assert "real(c_double), pointer, dimension(:, :) :: values" in bridge_source - assert "if (values_dense_actual /= 0_c_int) then" in bridge_source - assert "values => values_base" in bridge_source - assert ( - "values => values_base(1:values_upper_bound_0 + 1:values_stride_0, 1:values_upper_bound_1 + 1:values_stride_1)" - ) in bridge_source - assert "call native_strided(values)" in bridge_source - assert max(map(len, bridge_source.splitlines())) <= 132 - + assert "bound_values_dense_actual" not in c_source + assert "bound_values_upper_bound_0" not in c_source -def test_rank3_strided_array_pointer_sections_respect_free_form_line_limit(): - artifacts = WrapperGenerator().generate(_strided_plan(rank=3)) - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - - assert "values => values_base(&" in bridge_source - assert "& 1:values_upper_bound_2 + 1:values_stride_2)" in bridge_source + assert "real(c_double), dimension(:, :) :: values" in bridge_source + assert "call native_strided(values)" in bridge_source + # Nothing is rebuilt from an address any more. + assert "call c_f_pointer(" not in bridge_source + assert "values_base" not in bridge_source + assert "values_dense_actual" not in bridge_source assert max(map(len, bridge_source.splitlines())) <= 132 -def test_strided_role_edit_fails_before_backend_lowering(): +def test_descriptor_array_stride_role_edit_fails_before_backend_lowering(): plan = _strided_plan() array = plan.namespaces[0].functions[0].arguments[0].array assert array is not None - array.stride_roles = array.stride_roles[:1] + array.stride_roles = (f"{array.data_role}:stride:0",) - with pytest.raises(ValueError, match="invalid-array-stride-roles"): + with pytest.raises(ValueError, match="unexpected-array-descriptor-roles"): WrapperGenerator().generate(plan) -def test_strided_dense_actual_role_edit_fails_before_backend_lowering(): +def test_descriptor_array_dense_actual_role_edit_fails_before_backend_lowering(): plan = _strided_plan() array = plan.namespaces[0].functions[0].arguments[0].array assert array is not None - array.dense_actual_role = None + array.dense_actual_role = f"{array.data_role}:dense-actual" - with pytest.raises(ValueError, match="invalid-array-dense-actual-role"): + with pytest.raises(ValueError, match="unexpected-array-descriptor-roles"): WrapperGenerator().generate(plan) diff --git a/tests/fortran/arrays/end_to_end/test_array_contract_validation.py b/tests/fortran/arrays/end_to_end/test_array_contract_validation.py index 01628aad0..6e3e0274a 100644 --- a/tests/fortran/arrays/end_to_end/test_array_contract_validation.py +++ b/tests/fortran/arrays/end_to_end/test_array_contract_validation.py @@ -9,7 +9,6 @@ from tests.fortran._support.wrapper_build import ( _build_source_or_generated_pyi_and_import, ) -from prik.runtime.handles import _NativeArrayHandoff, AllocatableArray, PointerArray FIXTURES = Path(__file__).parent / "fixtures" ARRAY_CONTRACTS_F90_SOURCE = FIXTURES / "native" / "farray_contracts_f90.f90" @@ -19,80 +18,6 @@ pytestmark = pytest.mark.fortran_end_to_end -def _handoff(address): - return _NativeArrayHandoff(address) - - -def _absent_allocatable_handle(): - return AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("absent allocatable must not provide an array actual"), - "descriptor": lambda _handle: _handoff(301), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: None, - "allocated": lambda _handle: False, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, - ) - - -def _array_allocatable_handle(value): - return AllocatableArray( - dtype=value.dtype, - rank=value.ndim, - ops={ - "array_actual": lambda _handle: _handoff(value.ctypes.data), - "descriptor": lambda _handle: _handoff(303), - "shape": lambda _handle: value.shape, - "layout": lambda _handle: "F" if value.flags.f_contiguous else "C", - "writeable": lambda _handle: value.flags.writeable, - "native_byte_order": lambda _handle: value.dtype.isnative, - "aligned": lambda _handle: value.flags.aligned, - "to_numpy": lambda _handle: value, - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, - ) - - -def _absent_pointer_handle(): - return PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("absent pointer must not provide an array actual"), - "descriptor": lambda _handle: _handoff(304), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, - ) - - -def _array_pointer_handle(value): - return PointerArray( - dtype=value.dtype, - rank=value.ndim, - ops={ - "array_actual": lambda _handle: _handoff(value.ctypes.data), - "descriptor": lambda _handle: _handoff(306), - "shape": lambda _handle: value.shape, - "layout": lambda _handle: "F" if value.flags.f_contiguous else "C", - "writeable": lambda _handle: value.flags.writeable, - "native_byte_order": lambda _handle: value.dtype.isnative, - "aligned": lambda _handle: value.flags.aligned, - "to_numpy": lambda _handle: value, - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, - ) - - def test_remaining_array_contracts_are_validated_before_fortran_calls( pyi_parity_build_mode: str, tmp_path: Path, @@ -109,39 +34,6 @@ def test_remaining_array_contracts_are_validated_before_fortran_calls( pyi_parity_build_mode, ) - absent_allocatable = _absent_allocatable_handle() - absent_pointer = _absent_pointer_handle() - assert absent_allocatable.to_numpy() is None - assert absent_pointer.to_numpy() is None - with pytest.raises(TypeError): - module.sum_in(absent_allocatable.to_numpy()) - with pytest.raises(TypeError): - module.sum_in(absent_pointer.to_numpy()) - with pytest.raises(ValueError, match="unallocated"): - module.sum_in(absent_allocatable) - with pytest.raises(ValueError, match="unassociated"): - module.sum_in(absent_pointer) - - actual_values = np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float64) - assert module.sum_in(_array_allocatable_handle(actual_values)) == np.float64(10.0) - assert module.sum_in(_array_pointer_handle(actual_values)) == np.float64(10.0) - with pytest.raises(TypeError, match="dtype"): - module.sum_in(_array_allocatable_handle(np.array([1.0, 2.0], dtype=np.float32))) - - readonly_allocatable_array = np.array([1.0, 2.0], dtype=np.float64) - readonly_allocatable_array.setflags(write=False) - readonly_allocatable = _array_allocatable_handle(readonly_allocatable_array) - assert readonly_allocatable.to_numpy() is readonly_allocatable_array - with pytest.raises(TypeError, match="writeable"): - module.bump_inout(readonly_allocatable.to_numpy()) - - readonly_pointer_array = np.array([1.0, 2.0], dtype=np.float64) - readonly_pointer_array.setflags(write=False) - readonly_pointer = _array_pointer_handle(readonly_pointer_array) - assert readonly_pointer.to_numpy() is readonly_pointer_array - with pytest.raises(TypeError, match="writeable"): - module.bump_inout(readonly_pointer.to_numpy()) - readonly = np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float64) readonly.setflags(write=False) if pyi_parity_build_mode == "source": diff --git a/tests/fortran/arrays/end_to_end/test_array_direct_entrypoint_routing.py b/tests/fortran/arrays/end_to_end/test_array_direct_entrypoint_routing.py index 094963b9b..afec1416a 100644 --- a/tests/fortran/arrays/end_to_end/test_array_direct_entrypoint_routing.py +++ b/tests/fortran/arrays/end_to_end/test_array_direct_entrypoint_routing.py @@ -37,7 +37,7 @@ def test_arrays_all_direct_route_preserves_dtype_values_and_mutation( flags = np.array([True, True, False], dtype=np.bool_, order="F") assert bool(module.all_flags(np.int32(flags.size), flags)) is False inverted = module.invert_flags(np.int32(flags.size), flags) - np.testing.assert_array_equal(flags, np.array([False, False, True], dtype=np.bool_)) + np.testing.assert_array_equal(flags, np.array([False, False, True])) if inverted is not None: np.testing.assert_array_equal(inverted, flags) diff --git a/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py b/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py index e3eb532a8..a9a90b9d3 100644 --- a/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py +++ b/tests/fortran/arrays/end_to_end/test_array_wrapper_parity.py @@ -15,7 +15,6 @@ _sole_native_module, ) from prik import build_pyi_extension -from prik.runtime.handles import _NativeArrayHandoff, AllocatableArray, PointerArray FIXTURES = Path(__file__).parent / "fixtures" CONTRACTS = FIXTURES / "contracts" @@ -24,24 +23,6 @@ pytestmark = pytest.mark.fortran_end_to_end -def _native_array_actual(value: np.ndarray, *, pointer: bool): - state_name = "associated" if pointer else "allocated" - operations = { - "array_actual": lambda _handle: _NativeArrayHandoff(value.ctypes.data), - "descriptor": lambda _handle: _NativeArrayHandoff(value.ctypes.data), - "shape": lambda _handle: value.shape, - "layout": lambda _handle: "F" if value.flags.f_contiguous else "C", - "writeable": lambda _handle: value.flags.writeable, - "native_byte_order": lambda _handle: value.dtype.isnative, - "aligned": lambda _handle: value.flags.aligned, - "to_numpy": lambda _handle: value, - state_name: lambda _handle: True, - "nullify" if pointer else "deallocate": lambda _handle: None, - } - handle_type = PointerArray if pointer else AllocatableArray - return handle_type(dtype=value.dtype, rank=value.ndim, ops=operations) - - def test_fortran_array_wrapper_pipeline_matches_fmath_results_with_contiguous_arrays( pyi_parity_build_mode: str, tmp_path: Path, @@ -106,14 +87,6 @@ def test_required_array_buffers_use_canonical_wrapper_plan(tmp_path: Path): assert module.square_r8_contiguous(np.int32(values.size), values, output) == np.int32(values.size) np.testing.assert_array_equal(output, values**2) - handle_output = np.zeros_like(values) - assert module.square_r8_contiguous( - np.int32(values.size), - _native_array_actual(values, pointer=False), - _native_array_actual(handle_output, pointer=True), - ) == np.int32(values.size) - np.testing.assert_array_equal(handle_output, values**2) - empty = np.empty(0, dtype=np.float64) assert module.square_r8_contiguous(np.int32(0), empty, empty.copy()) == np.int32(0) diff --git a/tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py b/tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py index cc97cc224..6360e2366 100644 --- a/tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py +++ b/tests/fortran/arrays/end_to_end/test_assumed_rank_arrays.py @@ -8,7 +8,6 @@ from tests.fortran._support.wrapper_build import ( _build_source_or_generated_pyi_and_import, ) -from prik.runtime.handles import _NativeArrayHandoff, AllocatableArray, PointerArray FIXTURES = Path(__file__).parent / "fixtures" ASSUMED_RANK_F90_SOURCE = FIXTURES / "native" / "fassumed_rank_f90.f90" @@ -33,41 +32,6 @@ def assumed_rank_module(request: pytest.FixtureRequest, tmp_path_factory: pytest ) -def _allocated_handle_for_rejected_assumed_rank(value): - return AllocatableArray( - dtype=value.dtype, - rank=value.ndim, - ops={ - "array_actual": lambda _handle: pytest.fail("assumed-rank path must reject handles before handoff"), - "descriptor": lambda _handle: _NativeArrayHandoff(401), - "shape": lambda _handle: value.shape, - "layout": lambda _handle: "F" if value.flags.f_contiguous else "C", - "writeable": lambda _handle: value.flags.writeable, - "native_byte_order": lambda _handle: value.dtype.isnative, - "aligned": lambda _handle: value.flags.aligned, - "to_numpy": lambda _handle: value, - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, - ) - - -def _unassociated_handle_for_rejected_assumed_rank(): - return PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("assumed-rank path must reject handles before handoff"), - "descriptor": lambda _handle: _NativeArrayHandoff(402), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, - ) - - def test_assumed_rank_arguments_dispatch_to_runtime_rank( assumed_rank_module, ): @@ -92,12 +56,6 @@ def test_assumed_rank_arguments_dispatch_to_runtime_rank( with pytest.raises(TypeError): module.rank_weighted_sum(rank16) - handle_values = np.asfortranarray(np.array([1.0, 2.0], dtype=np.float64)) - with pytest.raises(TypeError): - module.rank_weighted_sum(_allocated_handle_for_rejected_assumed_rank(handle_values)) - with pytest.raises(TypeError): - module.rank_weighted_sum(_unassociated_handle_for_rejected_assumed_rank()) - def test_assumed_rank_bridge_dispatches_each_runtime_rank_argument( assumed_rank_module, diff --git a/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py b/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py index 30e5af9e1..9c86a4f5c 100644 --- a/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py +++ b/tests/fortran/arrays/end_to_end/test_layout_and_strided_arrays.py @@ -165,28 +165,40 @@ def test_rank2_assumed_shape_accepts_fortran_ordered_strided_views(compiled_mult compiled_multid_array_module.scale2_strided(contiguous_source, c_order_out) -def test_rank2_assumed_shape_rejects_non_positive_strides(compiled_multid_array_module): - source = _matrix() - out = np.zeros_like(source, order="F") +def test_rank2_assumed_shape_accepts_reversed_axes_and_refuses_what_is_not_a_section( + compiled_multid_array_module, +): + """A reversed axis is described; a broadcast one has nothing to describe. + + The dummy is reached through a descriptor, which records a signed step per + axis, so an axis that runs backwards is passed on as it stands and the + callee reads the same elements the caller sees. A zero step is not a + direction, it is a repetition, and Fortran has no array section for it -- + so that one is still refused, and says so in its own terms. + """ + reversed_source = _reversed_fortran_matrix() + out = np.zeros_like(reversed_source, order="F") checksum = np.zeros(1, dtype=np.float64) - reversed_source = _reversed_fortran_matrix() - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.scale2_strided(reversed_source, out) - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.checksum2_strided(reversed_source, checksum) + compiled_multid_array_module.scale2_strided(reversed_source, out) + np.testing.assert_allclose(out, 3.0 * reversed_source) + + compiled_multid_array_module.checksum2_strided(reversed_source, checksum) + np.testing.assert_allclose(checksum[0], _checksum2(reversed_source)) + + # Writing through a reversed view reaches the caller's own elements. + reversed_out = _reversed_fortran_matrix() + before = np.array(reversed_out, copy=True) + compiled_multid_array_module.scale2_strided(reversed_out, reversed_out) + np.testing.assert_allclose(reversed_out, 3.0 * before) broadcast_source = _broadcast_fortran_like_matrix() assert broadcast_source.strides[0] == 0 - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): + with pytest.raises(TypeError, match=r"not a Fortran array section"): compiled_multid_array_module.scale2_strided(broadcast_source, out) - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): + with pytest.raises(TypeError, match=r"not a Fortran array section"): compiled_multid_array_module.checksum2_strided(broadcast_source, checksum) - reversed_out = _reversed_fortran_matrix() - with pytest.raises(TypeError, match=r"expected ordering \(F\)"): - compiled_multid_array_module.scale2_strided(source, reversed_out) - def test_rank2_explicit_shape_requires_fortran_contiguous(compiled_multid_array_module): source = _matrix() diff --git a/tests/fortran/arrays/end_to_end/test_logical_kind_array_conversions.py b/tests/fortran/arrays/end_to_end/test_logical_kind_array_conversions.py index c306cbeec..aaf985ef9 100644 --- a/tests/fortran/arrays/end_to_end/test_logical_kind_array_conversions.py +++ b/tests/fortran/arrays/end_to_end/test_logical_kind_array_conversions.py @@ -7,6 +7,7 @@ import numpy as np import pytest +from prik import contracts from tests.fortran._support.wrapper_build import _build_inline_pyi_contract_module, _build_text_and_import @@ -64,11 +65,88 @@ inout_values = input_values .neqv. inout_values end subroutine exercise_64 + subroutine replace_c_bool(values) + logical(kind=c_bool), allocatable, intent(inout) :: values(:) + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = .false. + values(1) = .true. + values(3) = .true. + end subroutine replace_c_bool + + subroutine replace_8(values) + logical(kind=1), allocatable, intent(inout) :: values(:) + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = .false. + values(1) = .true. + values(3) = .true. + end subroutine replace_8 + + subroutine replace_16(values) + logical(kind=2), allocatable, intent(inout) :: values(:) + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = .false. + values(1) = .true. + values(3) = .true. + end subroutine replace_16 + + subroutine replace_32(values) + logical(kind=4), allocatable, intent(inout) :: values(:) + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = .false. + values(1) = .true. + values(3) = .true. + end subroutine replace_32 + + subroutine replace_64(values) + logical(kind=8), allocatable, intent(inout) :: values(:) + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = .false. + values(1) = .true. + values(3) = .true. + end subroutine replace_64 + end module logical_kind_arrays """ -def test_boolean_arrays_copy_only_in_required_directions_for_every_supported_width(tmp_path: Path): +_LOGICAL_POINTER_SOURCE = """ +module logical_pointer_arrays + implicit none +contains + subroutine replace_pointer_32(values) + logical(kind=4), pointer, intent(inout) :: values(:) + if (associated(values)) deallocate(values) + allocate(values(3)) + values = .false. + values(1) = .true. + values(3) = .true. + end subroutine replace_pointer_32 +end module logical_pointer_arrays +""" + + +_LOGICAL_KIND_DTYPES = { + "c_bool": np.bool_, + "8": np.bool_, + "16": np.int16, + "32": np.int32, + "64": np.int64, +} + + +def test_boolean_arrays_are_aliased_at_their_own_width_without_any_copy(tmp_path: Path): + """Every logical kind is passed as the caller's own buffer, whatever its width. + + NumPy has no Boolean wider than one byte, so a logical array is described by + the integer of matching width. The widths then agree by construction and the + buffer is aliased directly: no native-kind temporary is declared, nothing is + widened on the way in, and nothing is narrowed on the way out. + """ module = _build_text_and_import( _LOGICAL_KIND_ARRAY_SOURCE, "logical_kind_arrays.f90", @@ -80,19 +158,12 @@ def test_boolean_arrays_copy_only_in_required_directions_for_every_supported_wid }, ) bridge_source = (tmp_path / "bind_c_logical_kind_arrays_wrapper.f90").read_text(encoding="utf-8") - assert bridge_source.count("input_values_native = input_values") == 4 - assert bridge_source.count("inout_values_native = inout_values") == 4 - assert "output_values_native = output_values" not in bridge_source - assert bridge_source.count("output_values = merge(.true._c_bool, .false._c_bool, output_values_native)") == 4 - assert bridge_source.count("inout_values = merge(.true._c_bool, .false._c_bool, inout_values_native)") == 4 assert "call native_exercise_c_bool(n, input_values, output_values, inout_values)" in bridge_source - input_values = np.array([True, False, True, False], dtype=np.bool_) - initial_inout = np.array([False, False, True, True], dtype=np.bool_) - expected_output = np.logical_not(input_values) - expected_inout = np.logical_xor(input_values, initial_inout) - for suffix in ("c_bool", "8", "16", "32", "64"): - output_values = np.empty(input_values.shape, dtype=np.bool_) + for suffix, dtype in _LOGICAL_KIND_DTYPES.items(): + input_values = np.array([1, 0, 1, 0], dtype=dtype) + initial_inout = np.array([0, 0, 1, 1], dtype=dtype) + output_values = np.empty(input_values.shape, dtype=dtype) inout_values = initial_inout.copy() result = getattr(module, f"exercise_{suffix}")( @@ -103,14 +174,22 @@ def test_boolean_arrays_copy_only_in_required_directions_for_every_supported_wid ) assert result is None - assert input_values.dtype == output_values.dtype == inout_values.dtype == np.dtype(np.bool_) - np.testing.assert_array_equal(output_values, expected_output) - np.testing.assert_array_equal(inout_values, expected_inout) + assert output_values.dtype == inout_values.dtype == np.dtype(dtype), suffix + # The values are Fortran logicals, so they are read back as truth + # rather than compared against any one integer the compiler chose. + np.testing.assert_array_equal( + output_values.astype(bool), np.logical_not(input_values.astype(bool)), err_msg=suffix + ) + np.testing.assert_array_equal( + inout_values.astype(bool), + np.logical_xor(input_values.astype(bool), initial_inout.astype(bool)), + err_msg=suffix, + ) def test_numbered_boolean_pyi_contracts_probe_and_call_every_supported_width(tmp_path: Path): contract = """ -from prik.contracts import Bool8, Bool16, Bool32, Bool64, Int32 +from prik.contracts import Allocatable, Bool8, Bool16, Bool32, Bool64, Int32 def exercise_c_bool( n: Int32, @@ -146,6 +225,12 @@ def exercise_64( output_values: Bool64[n], inout_values: Bool64[n], ) -> None: ... + +def replace_c_bool(values: Allocatable[Bool8[:]]) -> None: ... +def replace_8(values: Allocatable[Bool8[:]]) -> None: ... +def replace_16(values: Allocatable[Bool16[:]]) -> None: ... +def replace_32(values: Allocatable[Bool32[:]]) -> None: ... +def replace_64(values: Allocatable[Bool64[:]]) -> None: ... """ module, result = _build_inline_pyi_contract_module( tmp_path, @@ -155,14 +240,14 @@ def exercise_64( ) bridge_source = next(path for path in result.generated_sources if path.suffix == ".f90").read_text(encoding="utf-8") assert "call native_exercise_8(n, input_values, output_values, inout_values)" in bridge_source - assert "logical(kind=2), dimension(input_values_extent_0) :: input_values_native" in bridge_source - assert "logical(kind=4), dimension(input_values_extent_0) :: input_values_native" in bridge_source - assert "logical(kind=8), dimension(input_values_extent_0) :: input_values_native" in bridge_source - - input_values = np.array([True, False, True, False], dtype=np.bool_) - initial_inout = np.array([False, False, True, True], dtype=np.bool_) - for suffix in ("c_bool", "8", "16", "32", "64"): - output_values = np.empty(input_values.shape, dtype=np.bool_) + # Each numbered contract width aliases a buffer of its own size. + for kind in ("logical(c_bool)", "logical(2)", "logical(4)", "logical(8)"): + assert f"{kind}, pointer, contiguous, dimension(:) :: input_values" in bridge_source, kind + + for suffix, dtype in _LOGICAL_KIND_DTYPES.items(): + input_values = np.array([1, 0, 1, 0], dtype=dtype) + initial_inout = np.array([0, 0, 1, 1], dtype=dtype) + output_values = np.empty(input_values.shape, dtype=dtype) inout_values = initial_inout.copy() getattr(module, f"exercise_{suffix}")( @@ -172,5 +257,67 @@ def exercise_64( inout_values, ) - np.testing.assert_array_equal(output_values, np.logical_not(input_values)) - np.testing.assert_array_equal(inout_values, np.logical_xor(input_values, initial_inout)) + np.testing.assert_array_equal( + output_values.astype(bool), np.logical_not(input_values.astype(bool)), err_msg=suffix + ) + np.testing.assert_array_equal( + inout_values.astype(bool), + np.logical_xor(input_values.astype(bool), initial_inout.astype(bool)), + err_msg=suffix, + ) + + handle_contracts = { + "c_bool": contracts.Bool8, + "8": contracts.Bool8, + "16": contracts.Bool16, + "32": contracts.Bool32, + "64": contracts.Bool64, + } + for suffix, dtype in _LOGICAL_KIND_DTYPES.items(): + handle = contracts.Allocatable[handle_contracts[suffix][:]]() + try: + assert getattr(module, f"replace_{suffix}")(handle) is None + assert handle.dtype == np.dtype(dtype) + assert handle.shape == (3,) + np.testing.assert_array_equal(handle.to_numpy().astype(bool), [True, False, True]) + finally: + handle.close() + + +def test_caller_created_wide_logical_pointer_uses_its_native_width(tmp_path: Path): + contract = """ +from prik.contracts import Annotated, Bool32, Pointer, PointerPolicy + +def replace_pointer_32( + values: Annotated[ + Pointer[Bool32[:]], + PointerPolicy( + nullable=True, + transfer="call_local", + target_owner="caller", + lifetime="call", + deallocation="deallocate_resize", + shape_source="pointer_bounds", + contiguity="contiguous", + reassociation="allocate_resize", + aliasing="descriptor", + mutability="mutable", + ), + ], +) -> None: ... +""" + module, _result = _build_inline_pyi_contract_module( + tmp_path, + module_name="logical_pointer_arrays", + source_text=_LOGICAL_POINTER_SOURCE, + contract_text=contract, + ) + pointer = contracts.Pointer[contracts.Bool32[:]]() + try: + assert module.replace_pointer_32(pointer) is None + assert pointer.dtype == np.dtype(np.int32) + assert pointer.shape == (3,) + np.testing.assert_array_equal(pointer.to_numpy().astype(bool), [True, False, True]) + pointer.deallocate() + finally: + pointer.close() diff --git a/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py b/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py new file mode 100644 index 000000000..dfded60d7 --- /dev/null +++ b/tests/fortran/arrays/end_to_end/test_native_handle_array_forms.py @@ -0,0 +1,638 @@ +"""Generated handles used as every supported ordinary Fortran array form.""" + +from __future__ import annotations + +import gc +import sys +from pathlib import Path + +import numpy as np +import pytest + +from prik import contracts +from prik.runtime.handles import AllocatableArray, PointerArray +from tests.fortran._support.wrapper_build import _build_text_and_import + + +pytestmark = pytest.mark.fortran_end_to_end + + +NATIVE_HANDLE_ARRAY_FORMS_SOURCE = """\ +module fhandle_array_forms_f90 + implicit none + real(8), allocatable :: values(:) + real(8), allocatable :: matrix(:, :) + real(8), allocatable :: hyper(:, :, :, :, :, :, :, :, :, :, :, :, :, :, :) + real(8), target :: backing(8) + real(8), pointer :: strided_values(:) + character(len=:), allocatable :: words(:) +contains + subroutine setup() + integer :: i + + allocate(values(4)) + values = [1.0_8, 2.0_8, 3.0_8, 4.0_8] + allocate(matrix(2, 3)) + matrix = reshape([(1.0_8 * i, i = 1, 6)], [2, 3]) + allocate(hyper(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)) + hyper = 1.0_8 + backing = [(1.0_8 * i, i = 1, 8)] + strided_values => backing(1:8:2) + allocate(character(len=5) :: words(2)) + words = [character(len=5) :: "alpha", "bravo"] + end subroutine setup + + function explicit_total(actual, n) result(total) + integer(4), intent(in) :: n + real(8), intent(in) :: actual(n) + real(8) :: total + + total = sum(actual) + end function explicit_total + + function assumed_total(actual) result(total) + real(8), intent(in) :: actual(:) + real(8) :: total + + total = sum(actual) + end function assumed_total + + function flat_total(actual, n) result(total) + integer(4), intent(in) :: n + real(8), intent(in) :: actual(*) + real(8) :: total + + total = sum(actual(:n)) + end function flat_total + + function optional_total(actual) result(total) + real(8), intent(in), optional :: actual(:) + real(8) :: total + + if (present(actual)) then + total = sum(actual) + else + total = -1.0_8 + end if + end function optional_total + + function rank_score(actual) result(score) + real(8), intent(in) :: actual(..) + integer(4) :: score + + select rank (actual) + rank (1) + score = 100 + size(actual) + rank (2) + score = 200 + size(actual) + rank (15) + score = 1500 + size(actual) + rank default + score = -1 + end select + end function rank_score + + function element_width(actual) result(width) + character(len=*), intent(in) :: actual(:) + integer(4) :: width + + width = len(actual) + end function element_width +end module fhandle_array_forms_f90 +""" + + +def test_generated_handles_cover_supported_ordinary_array_forms(tmp_path: Path): + module = _build_text_and_import( + NATIVE_HANDLE_ARRAY_FORMS_SOURCE, + "fhandle_array_forms_f90.f90", + tmp_path, + { + "bind_c_fhandle_array_forms_f90_wrapper.f90", + "fhandle_array_forms_f90_wrapper.c", + "fhandle_array_forms_f90_wrapper.h", + }, + ) + + values = module.values + strided = module.strided_values + assert isinstance(values, AllocatableArray) + assert isinstance(strided, PointerArray) + with pytest.raises(ValueError, match="unallocated"): + module.assumed_total(values) + with pytest.raises(ValueError, match="unassociated"): + module.assumed_total(strided) + + module.setup() + + assert module.explicit_total(values, np.int32(4)) == np.float64(10.0) + assert module.assumed_total(values) == np.float64(10.0) + assert module.flat_total(values, np.int32(4)) == np.float64(10.0) + assert module.optional_total() == np.float64(-1.0) + assert module.optional_total(None) == np.float64(-1.0) + assert module.optional_total(values) == np.float64(10.0) + assert module.rank_score(values) == np.int32(104) + assert module.rank_score(module.matrix) == np.int32(206) + assert module.rank_score(module.hyper) == np.int32(1501) + assert module.assumed_total(strided) == np.float64(16.0) + assert module.element_width(module.words) == np.int32(5) + + with pytest.raises(TypeError, match="incompatible shape at axis 0"): + module.explicit_total(values, np.int32(3)) + with pytest.raises(TypeError, match="does not match expected dtype"): + module.assumed_total(module.words) + + +NATIVE_HANDLE_DESCRIPTOR_MATRIX_SOURCE = """\ +module fhandle_descriptor_matrix_f90 + use iso_c_binding, only: c_bool + implicit none + + type :: holder + real(8), allocatable :: field_allocatable_values_with_long_name(:) + logical(4), allocatable :: field_flags(:) + character(len=4), allocatable :: field_fixed_words(:) + character(len=4), allocatable :: field_missing_words(:) + character(len=4), pointer :: field_missing_pointer(:) => null() + real(8), pointer :: field_ptr(:) => null() + character(len=:), pointer :: field_words(:) => null() + end type holder + + integer(4), allocatable :: ints(:) + real(4), allocatable :: reals(:) + complex(8), allocatable :: complexes(:) + logical(c_bool), allocatable :: flags(:) + character(len=4), allocatable :: fixed_words(:) + character(len=:), allocatable :: deferred_words(:) + real(8), allocatable :: empty(:) + real(8), allocatable :: spare(:) + real(8), allocatable :: probe(:) + real(8), allocatable :: optional_probe(:) + real(8), allocatable :: pair_left(:) + real(8), allocatable :: pair_right(:) + real(8), allocatable :: pair_third(:) + real(8), allocatable :: cube(:, :, :) + real(8), target :: store(8) + real(8), pointer :: reversed(:) => null() + real(8), pointer :: strided(:) => null() + real(8), pointer :: unassociated(:) => null() + type(holder) :: parent + +contains + + subroutine setup() + integer :: i + + allocate(ints(3)); ints = [1_4, 2_4, 3_4] + allocate(reals(2)); reals = [1.5_4, 2.5_4] + allocate(complexes(2)); complexes = [(1.0_8, 2.0_8), (3.0_8, 4.0_8)] + allocate(flags(3)); flags = [.true._c_bool, .false._c_bool, .true._c_bool] + allocate(fixed_words(2)); fixed_words = ['abcd', 'efgh'] + allocate(character(len=6) :: deferred_words(2)) + deferred_words = ['alphas', 'bravos'] + allocate(empty(0)) + allocate(probe(3)); probe = 2.0_8 + allocate(optional_probe(3)); optional_probe = 2.0_8 + allocate(pair_left(3)); pair_left = 6.0_8 + allocate(pair_right(3)); pair_right = 7.0_8 + allocate(pair_third(3)); pair_third = 8.0_8 + allocate(cube(2, 3, 4)); cube = 1.0_8 + store = [(1.0_8 * i, i = 1, 8)] + reversed => store(8:1:-1) + strided => store(1:8:2) + allocate(parent%field_allocatable_values_with_long_name(3)) + parent%field_allocatable_values_with_long_name = 5.0_8 + allocate(parent%field_flags(3)); parent%field_flags = [.true., .false., .true.] + allocate(parent%field_fixed_words(2)); parent%field_fixed_words = ['abcd', 'efgh'] + parent%field_ptr => store(2:6:2) + allocate(character(len=5) :: parent%field_words(2)) + parent%field_words = ['alpha', 'beta '] + end subroutine setup + + subroutine reshape_alloc(values, n) + real(8), allocatable, intent(inout) :: values(:) + integer(4), intent(in) :: n + + if (allocated(values)) deallocate(values) + allocate(values(n)) + values = 4.0_8 + end subroutine reshape_alloc + + subroutine grow_pair(first, second, n) + real(8), allocatable, intent(inout) :: first(:), second(:) + integer(4), intent(in) :: n + + if (allocated(first)) deallocate(first) + if (allocated(second)) deallocate(second) + allocate(first(n)); first = 8.0_8 + allocate(second(n)); second = 9.0_8 + end subroutine grow_pair + + subroutine grow_three(first, second, third, n) + real(8), allocatable, intent(inout) :: first(:), second(:), third(:) + integer(4), intent(in) :: n + + if (allocated(first)) deallocate(first) + if (allocated(second)) deallocate(second) + if (allocated(third)) deallocate(third) + allocate(first(n)); first = 8.0_8 + allocate(second(n)); second = 9.0_8 + allocate(third(n)); third = 10.0_8 + end subroutine grow_three + + function optional_state(values) result(state) + real(8), allocatable, optional, intent(in) :: values(:) + integer(4) :: state + + if (.not. present(values)) then + state = 0 + else if (.not. allocated(values)) then + state = 1 + else + state = int(sum(values), kind=4) + end if + end function optional_state + + subroutine grow_and_count(values, n, produced) + real(8), allocatable, intent(inout) :: values(:) + integer(4), intent(in) :: n + integer(4), intent(out) :: produced + + if (allocated(values)) deallocate(values) + allocate(values(n)); values = 11.0_8 + produced = n + end subroutine grow_and_count + + function optional_alloc_state(values) result(state) + real(8), allocatable, intent(in), optional :: values(:) + integer(4) :: state + + if (.not. present(values)) then + state = 0 + else if (allocated(values)) then + state = 2 + else + state = 1 + end if + end function optional_alloc_state + + function assumed_total(actual) result(total) + real(8), intent(in) :: actual(:) + real(8) :: total + + total = sum(actual) + end function assumed_total + + function int_total(actual) result(total) + integer(4), intent(in) :: actual(:) + integer(4) :: total + + total = sum(actual) + end function int_total + + function real4_total(actual) result(total) + real(4), intent(in) :: actual(:) + real(4) :: total + + total = sum(actual) + end function real4_total + + function complex_total(actual) result(total) + complex(8), intent(in) :: actual(:) + complex(8) :: total + + total = sum(actual) + end function complex_total + + function true_count(actual) result(counted) + logical(c_bool), intent(in) :: actual(:) + integer(4) :: counted + + counted = count(actual) + end function true_count + + function word_width(actual) result(width) + character(len=*), intent(in) :: actual(:) + integer(4) :: width + + width = len(actual) + end function word_width + + function cube_score(actual) result(score) + real(8), intent(in) :: actual(:, :, :) + integer(4) :: score + + score = size(actual) + end function cube_score +end module fhandle_descriptor_matrix_f90 +""" + + +@pytest.fixture(scope="module") +def descriptor_matrix(tmp_path_factory): + module = _build_text_and_import( + NATIVE_HANDLE_DESCRIPTOR_MATRIX_SOURCE, + "fhandle_descriptor_matrix_f90.f90", + tmp_path_factory.mktemp("descriptor_matrix"), + { + "bind_c_fhandle_descriptor_matrix_f90_wrapper.f90", + "fhandle_descriptor_matrix_f90_wrapper.c", + "fhandle_descriptor_matrix_f90_wrapper.h", + }, + ) + module.setup() + return module + + +@pytest.mark.parametrize( + ("name", "dtype", "shape", "expected"), + [ + ("ints", np.int32, (3,), [1, 2, 3]), + ("reals", np.float32, (2,), [1.5, 2.5]), + ("complexes", np.complex128, (2,), [1 + 2j, 3 + 4j]), + ("flags", np.bool_, (3,), [True, False, True]), + ("fixed_words", "S4", (2,), [b"abcd", b"efgh"]), + ("deferred_words", "S6", (2,), [b"alphas", b"bravos"]), + ("empty", np.float64, (0,), []), + ("cube", np.float64, (2, 3, 4), None), + ], +) +def test_every_supported_element_type_reports_its_own_dtype_shape_and_view( + descriptor_matrix, + name: str, + dtype, + shape: tuple[int, ...], + expected, +): + """One descriptor read answers dtype, shape and the view, for every element type.""" + handle = getattr(descriptor_matrix, name) + + assert isinstance(handle, AllocatableArray) + assert handle.allocated is True + assert handle.dtype == np.dtype(dtype) + assert handle.shape == shape + + view = handle.to_numpy() + assert view.dtype == np.dtype(dtype) + assert view.shape == shape + if expected is not None: + np.testing.assert_array_equal(view, np.array(expected, dtype=dtype)) + + +def test_each_element_type_reaches_a_matching_ordinary_dummy(descriptor_matrix): + """The storage behind a handle satisfies an ordinary dummy of its own type.""" + assert descriptor_matrix.int_total(descriptor_matrix.ints) == np.int32(6) + assert descriptor_matrix.real4_total(descriptor_matrix.reals) == np.float32(4.0) + assert descriptor_matrix.complex_total(descriptor_matrix.complexes) == np.complex128(4 + 6j) + assert descriptor_matrix.true_count(descriptor_matrix.flags) == np.int32(2) + assert descriptor_matrix.word_width(descriptor_matrix.fixed_words) == np.int32(4) + assert descriptor_matrix.word_width(descriptor_matrix.deferred_words) == np.int32(6) + assert descriptor_matrix.cube_score(descriptor_matrix.cube) == np.int32(24) + # A zero-sized allocation is present storage that happens to hold nothing. + assert descriptor_matrix.assumed_total(descriptor_matrix.empty) == np.float64(0.0) + + +def test_pointer_targets_report_their_shape_and_reach_an_ordinary_dummy(descriptor_matrix): + """A pointer's storage satisfies an array dummy however its target is laid out. + + The dummy is reached through a descriptor, which carries a signed stride per + axis, so the direction an axis runs is something the callee is told rather + than something the caller has to undo. A reversed target and a strided one + both arrive, and both sum to what their own elements sum to. Extraction + itself stays gated behind PointerPolicy. + """ + reversed_handle = descriptor_matrix.reversed + strided_handle = descriptor_matrix.strided + + assert isinstance(reversed_handle, PointerArray) + assert reversed_handle.associated is True + assert reversed_handle.shape == (8,) + assert strided_handle.shape == (4,) + + assert descriptor_matrix.assumed_total(strided_handle) == np.float64(16.0) + # store is 1..8, so the reversed view holds the same elements either way. + assert descriptor_matrix.assumed_total(reversed_handle) == np.float64(36.0) + + for handle in (reversed_handle, strided_handle): + with pytest.raises(NotImplementedError, match="unsupported by completed policy"): + handle.to_numpy() + + +def test_an_unassociated_pointer_reports_absence_through_every_inquiry(descriptor_matrix): + handle = descriptor_matrix.unassociated + + assert handle.associated is False + assert handle.shape is None + assert handle.to_numpy() is None + with pytest.raises(ValueError, match="unassociated"): + descriptor_matrix.assumed_total(handle) + + +def test_a_derived_type_field_view_retains_its_parent(descriptor_matrix): + """A field's storage belongs to its parent, so a view has to keep it alive.""" + parent = descriptor_matrix.parent + field = parent.field_allocatable_values_with_long_name + pointer_field = parent.field_ptr + logical_field = parent.field_flags + fixed_words_field = parent.field_fixed_words + missing_words_field = parent.field_missing_words + missing_pointer_field = parent.field_missing_pointer + words_field = parent.field_words + + assert field.shape == (3,) + np.testing.assert_allclose(field.to_numpy(), np.array([5.0, 5.0, 5.0])) + assert pointer_field.associated is True + assert pointer_field.shape == (3,) + assert logical_field.dtype == np.dtype(np.int32) + np.testing.assert_array_equal(logical_field.to_numpy().astype(bool), [True, False, True]) + assert fixed_words_field.allocated is True + assert fixed_words_field.shape == (2,) + assert fixed_words_field.dtype == np.dtype("S4") + assert fixed_words_field.to_numpy().tolist() == [b"abcd", b"efgh"] + assert descriptor_matrix.word_width(fixed_words_field) == np.int32(4) + assert missing_words_field.allocated is False + assert missing_words_field.shape is None + assert missing_words_field.to_numpy() is None + assert missing_pointer_field.associated is False + assert missing_pointer_field.shape is None + missing_pointer_field.associate(missing_pointer_field) + assert missing_pointer_field.associated is False + assert words_field.associated is True + assert words_field.shape == (2,) + assert words_field.dtype == np.dtype("S5") + words_field.deallocate() + assert words_field.associated is False + assert words_field.shape is None + + view = field.to_numpy() + assert view.base is not None + del field + del parent + gc.collect() + np.testing.assert_allclose(view, np.array([5.0, 5.0, 5.0])) + + +def test_reallocating_through_a_dummy_updates_every_later_inquiry(descriptor_matrix): + """Allocation state written by the callee reaches the caller's handle. + + `spare` is this test's alone, so the shared fixture keeps the state every + other test in this module reads. + """ + spare = descriptor_matrix.spare + assert spare.allocated is False + assert spare.shape is None + + descriptor_matrix.reshape_alloc(spare, np.int32(3)) + assert spare.shape == (3,) + np.testing.assert_allclose(spare.to_numpy(), np.array([4.0, 4.0, 4.0])) + assert descriptor_matrix.assumed_total(spare) == np.float64(12.0) + + descriptor_matrix.reshape_alloc(spare, np.int32(5)) + assert spare.shape == (5,) + + spare.deallocate() + assert spare.allocated is False + assert spare.shape is None + assert spare.to_numpy() is None + with pytest.raises(ValueError, match="unallocated"): + descriptor_matrix.assumed_total(spare) + + +def test_a_bound_handle_reaches_a_native_call_without_running_python(descriptor_matrix): + """Argument handoff costs no Python frame once the arguments are parsed. + + A handle publishes its backend to C, so the binding reads the capsule, + validates it against the dummy and enters the descriptor itself. Nothing on + that path imports the runtime, looks an operation up on the handle, or packs + descriptor fields into Python values for C to read back -- which is the + whole point of the capsule, and is only observable as the absence of a + Python call. + """ + ordinary = descriptor_matrix.ints + descriptor = descriptor_matrix.probe + int_total = descriptor_matrix.int_total + reshape_alloc = descriptor_matrix.reshape_alloc + assumed_total = descriptor_matrix.assumed_total + optional_alloc_state = descriptor_matrix.optional_alloc_state + called: list[str] = [] + + def record(frame, event, _arg): + if event == "call": + called.append(f"{frame.f_code.co_filename}:{frame.f_code.co_name}") + + sys.setprofile(record) + try: + int_total(ordinary) + assumed_total(descriptor) + reshape_alloc(descriptor, np.int32(2)) + assert optional_alloc_state() == np.int32(0) + assert optional_alloc_state(None) == np.int32(0) + assert optional_alloc_state(descriptor) == np.int32(2) + finally: + sys.setprofile(None) + + assert called == [] + assert descriptor.shape == (2,) + + +def test_two_descriptor_dummies_reach_borrowed_and_owned_storage_alike(descriptor_matrix): + """Two descriptors are live at once, so both callees' writes reach their entities. + + Each argument is entered in turn and the call is made inside the last + consumer, where every descriptor the Fortran runtime built is still valid. + A module array and a caller-created handle are placed the same way, and + both see the reallocation the callee performed. + """ + left = descriptor_matrix.pair_left + right = descriptor_matrix.pair_right + + descriptor_matrix.grow_pair(left, right, np.int32(2)) + assert left.shape == (2,) + assert right.shape == (2,) + np.testing.assert_allclose(left.to_numpy(), np.array([8.0, 8.0])) + np.testing.assert_allclose(right.to_numpy(), np.array([9.0, 9.0])) + + owned_first = contracts.Allocatable[contracts.Float64[:]]() + try: + # One borrowed and one owned handle in the same call: the chain enters + # whatever each publishes without distinguishing them. + descriptor_matrix.grow_pair(left, owned_first, np.int32(3)) + assert left.shape == (3,) + assert owned_first.shape == (3,) + np.testing.assert_allclose(owned_first.to_numpy(), np.array([9.0, 9.0, 9.0])) + finally: + owned_first.close() + + +def test_three_descriptor_dummies_keep_every_borrowed_descriptor_live(descriptor_matrix): + first = descriptor_matrix.pair_left + second = descriptor_matrix.pair_right + third = descriptor_matrix.pair_third + + descriptor_matrix.grow_three(first, second, third, np.int32(4)) + + assert first.shape == second.shape == third.shape == (4,) + np.testing.assert_allclose(first.to_numpy(), np.full(4, 8.0)) + np.testing.assert_allclose(second.to_numpy(), np.full(4, 9.0)) + np.testing.assert_allclose(third.to_numpy(), np.full(4, 10.0)) + + +def test_a_handle_reaches_a_call_with_a_hidden_output_without_running_python(descriptor_matrix): + """An intent(out) argument is carried, not read after the consumer returns. + + The call runs inside the consumer holding the descriptor, so the hidden + output is written through the address this frame carried in; the frame + outlives every consumer it enters, so reading it afterwards is sound and + no Python runs on the way. + """ + spare = descriptor_matrix.pair_left + called: list[str] = [] + + def record(frame, event, _arg): + if event == "call": + called.append(frame.f_code.co_name) + + descriptor_matrix.grow_and_count(spare, np.int32(2)) + sys.setprofile(record) + try: + produced = descriptor_matrix.grow_and_count(spare, np.int32(4)) + finally: + sys.setprofile(None) + + assert called == [] + assert int(produced[1]) == 4 + assert spare.shape == (4,) + + +def test_an_optional_descriptor_argument_runs_no_python_however_it_is_supplied(descriptor_matrix): + """Absence is decided in C, for an omitted argument as much as a supplied one. + + An absent optional has no handle to publish a backend, so nothing is + entered for it and the binding establishes the unallocated placeholder the + bridge is handed. Deciding that needs the argument object and nothing else, + so none of the three ways of supplying it goes back into Python. + """ + # This array is this test's alone, so its sum stays what the fixture set. + present = descriptor_matrix.optional_probe + called: list[str] = [] + + def record(frame, event, _arg): + if event == "call": + called.append(frame.f_code.co_name) + + descriptor_matrix.optional_state() + descriptor_matrix.optional_state(None) + descriptor_matrix.optional_state(present) + + sys.setprofile(record) + try: + omitted = descriptor_matrix.optional_state() + explicit_none = descriptor_matrix.optional_state(None) + supplied = descriptor_matrix.optional_state(present) + finally: + sys.setprofile(None) + + assert called == [] + assert omitted == np.int32(0) + assert explicit_none == np.int32(0) + assert supplied == np.int32(6) diff --git a/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py b/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py new file mode 100644 index 000000000..2b54239be --- /dev/null +++ b/tests/fortran/arrays/end_to_end/test_signed_stride_handoff.py @@ -0,0 +1,402 @@ +"""Signed-stride handoff to ordinary array dummies, from every supported source.""" + +from __future__ import annotations + +import sys + +import numpy as np +import pytest + +from prik.runtime.handles import PointerArray +from tests.fortran._support.wrapper_build import _build_text_and_import + + +pytestmark = pytest.mark.fortran_end_to_end + + +SIGNED_STRIDE_SOURCE = """\ +module fsigned_strides_f90 + use iso_c_binding + implicit none + + type :: holder + real(8), pointer :: field_ptr(:) => null() + end type holder + + real(8), target :: store(24) + real(8), pointer :: reversed_ptr(:) => null() + real(8), pointer :: strided_ptr(:) => null() + real(8), pointer :: unassociated_ptr(:) => null() + type(holder) :: parent + character(len=4), allocatable :: words(:) + +contains + + subroutine setup() + integer :: i + + do i = 1, 24 + store(i) = real(i, kind=8) + end do + reversed_ptr => store(8:1:-1) + strided_ptr => store(1:8:2) + parent%field_ptr => store(6:1:-1) + if (allocated(words)) deallocate(words) + allocate(words(2)) + words = ['abcd', 'efgh'] + end subroutine setup + + ! Bridged assumed-shape: not bind(C), so a bridge exists. + function total1(a) result(t) + real(8), intent(in) :: a(:) + real(8) :: t + + t = sum(a) + end function total1 + + ! Direct assumed-shape: bind(C), so C calls it with no bridge at all. + function total1_bindc(a) result(t) bind(c, name="fsigned_total1_bindc") + real(c_double), intent(in) :: a(:) + real(c_double) :: t + + t = sum(a) + end function total1_bindc + + ! Weighted so that reading the axes in the wrong order changes the answer. + function checksum2(a) result(t) + real(8), intent(in) :: a(:, :) + real(8) :: t + integer :: i, j + + t = 0.0_8 + do j = 1, size(a, 2) + do i = 1, size(a, 1) + t = t + a(i, j) * (100.0_8 * i + 10.0_8 * j) + end do + end do + end function checksum2 + + function total3(a) result(t) + real(8), intent(in) :: a(:, :, :) + real(8) :: t + + t = sum(a) + end function total3 + + subroutine negate1(a) + real(8), intent(inout) :: a(:) + + a = -a + end subroutine negate1 + + ! Two descriptor dummies in one call. + function dot2(a, b) result(t) + real(8), intent(in) :: a(:), b(:) + real(8) :: t + + t = sum(a * b) + end function dot2 + + function optional_total(a) result(t) + real(8), intent(in), optional :: a(:) + real(8) :: t + + if (present(a)) then + t = sum(a) + else + t = -1.0_8 + end if + end function optional_total + + function rank_and_size(a) result(s) + real(8), intent(in) :: a(..) + integer(4) :: s + + s = 100_4 * int(rank(a), 4) + int(size(a), 4) + end function rank_and_size + + ! An assumed-shape dummy always sees lower bound 1, whatever the actual had. + function first_and_last(a) result(t) + real(8), intent(in) :: a(:) + real(8) :: t + + t = a(1) * 1000.0_8 + a(size(a)) + real(lbound(a, 1), kind=8) + end function first_and_last + + ! Raw-address dummies: the declaration says the layout, nothing is conveyed. + function explicit_total(a, n) result(t) + integer(4), intent(in) :: n + real(8), intent(in) :: a(n) + real(8) :: t + + t = sum(a) + end function explicit_total + + function flat_total(a, n) result(t) + integer(4), intent(in) :: n + real(8), intent(in) :: a(*) + real(8) :: t + + t = sum(a(:n)) + end function flat_total + + function contig_total(a) result(t) + real(8), intent(in), contiguous :: a(:) + real(8) :: t + + t = sum(a) + end function contig_total + + function word_width(a) result(w) + character(len=*), intent(in) :: a(:) + integer(4) :: w + + w = int(len(a), 4) + end function word_width +end module fsigned_strides_f90 +""" + + +@pytest.fixture(scope="module") +def signed(tmp_path_factory): + module = _build_text_and_import( + SIGNED_STRIDE_SOURCE, + "fsigned_strides_f90.f90", + tmp_path_factory.mktemp("signed-strides"), + { + "bind_c_fsigned_strides_f90_wrapper.f90", + "fsigned_strides_f90_wrapper.c", + "fsigned_strides_f90_wrapper.h", + }, + ) + module.setup() + return module + + +def _base(n=8): + return np.arange(1.0, n + 1.0) + + +def _matrix(rows=4, cols=3): + return np.asfortranarray(np.arange(1.0, rows * cols + 1.0).reshape((rows, cols), order="F")) + + +def _checksum2(array): + total = 0.0 + for i, j in np.ndindex(array.shape): + total += array[i, j] * (100.0 * (i + 1) + 10.0 * (j + 1)) + return total + + +@pytest.mark.parametrize( + "view", + [ + pytest.param(lambda: _base()[::-1], id="rank-one-reversal"), + pytest.param(lambda: _base()[::-2], id="step-minus-2"), + pytest.param(lambda: _base()[::2], id="step-2"), + pytest.param(lambda: _base(), id="contiguous"), + pytest.param(lambda: _base()[:0], id="zero-sized"), + pytest.param(lambda: _base()[:0][::-1], id="zero-sized-reversed"), + pytest.param(lambda: _base()[:1][::-1], id="single-element-reversed"), + ], +) +def test_rank_one_numpy_views_reach_an_assumed_shape_dummy(signed, view): + """Whatever direction an axis runs, the callee reads the caller's elements.""" + array = view() + + assert signed.total1(array) == pytest.approx(float(array.sum())) + assert signed.total1_bindc(array) == pytest.approx(float(array.sum())) + + +@pytest.mark.parametrize( + "view", + [ + pytest.param(lambda: _matrix()[::-1, :], id="axis-0-reversed"), + pytest.param(lambda: _matrix()[:, ::-1], id="axis-1-reversed"), + pytest.param(lambda: _matrix()[::-1, ::-1], id="both-axes-reversed"), + pytest.param(lambda: _matrix(8, 3)[::-2, :], id="mixed-sign-strided"), + pytest.param(lambda: _matrix(8, 3)[::2, :], id="positive-strided"), + pytest.param(lambda: _matrix(0, 3), id="zero-sized-axis"), + ], +) +def test_rank_two_numpy_views_keep_their_axis_order(signed, view): + """The weighting makes a transposed or misread axis a different answer.""" + array = view() + + assert signed.checksum2(array) == pytest.approx(_checksum2(array)) + + +def test_a_singleton_axis_preserves_padding_between_later_sections(signed): + """An unobservable singleton stride does not collapse a later padded axis.""" + storage = np.arange(5.0) + view = np.ndarray((2, 1, 2), dtype=np.float64, buffer=storage, strides=(8, 0, 24)) + + assert signed.total3(view) == pytest.approx(float(view.sum())) + + +def test_a_reversed_view_is_written_through_to_the_callers_storage(signed): + """intent(inout) reaches the caller's own elements, in their own order.""" + array = _base() + reversed_view = array[::-1] + before = np.array(reversed_view, copy=True) + + signed.negate1(reversed_view) + + np.testing.assert_allclose(reversed_view, -before) + # The original, unreversed array holds the same negated elements. + np.testing.assert_allclose(array, -_base()) + + +def test_an_assumed_shape_dummy_rebases_every_actual_to_one(signed): + """A descriptor's own lower bounds are not portable, and are not relied on. + + gfortran normalises a section to zero and ifx keeps the parent's subscript, + so nothing may read them. An assumed-shape dummy has lower bound 1 whatever + it was handed, which is what the callee sees. + """ + for view in (_base(), _base()[::-1], _base()[::2], _base()[::-2]): + expected = view[0] * 1000.0 + view[-1] + 1.0 + assert signed.first_and_last(view) == pytest.approx(expected) + + +def test_a_reversed_pointer_handle_reaches_the_same_dummy(signed): + """A handle's descriptor already records its direction; it is entered as it is.""" + reversed_handle = signed.reversed_ptr + strided_handle = signed.strided_ptr + + assert isinstance(reversed_handle, PointerArray) + assert reversed_handle.shape == (8,) + + # store is 1..24; the reversed view covers store(8:1:-1). + assert signed.total1(reversed_handle) == pytest.approx(36.0) + assert signed.total1_bindc(reversed_handle) == pytest.approx(36.0) + assert signed.total1(strided_handle) == pytest.approx(16.0) + + +def test_a_reversed_derived_field_handle_reaches_the_same_dummy(signed): + """A field handle is entered through its parent, and keeps its direction.""" + field = signed.parent.field_ptr + + assert isinstance(field, PointerArray) + assert field.shape == (6,) + # store(6:1:-1) holds 6, 5, 4, 3, 2, 1. + assert signed.total1(field) == pytest.approx(21.0) + + +def test_one_call_takes_several_descriptors_from_different_sources(signed): + """Every descriptor in a call is live at once, whoever supplied it.""" + left = _base(4)[::-1] + right = _base(4) + + assert signed.dot2(left, right) == pytest.approx(float((left * right).sum())) + + # A borrowed handle and a NumPy array in the same call. + handle = signed.strided_ptr + ones = np.ones(4) + assert signed.dot2(handle, ones) == pytest.approx(16.0) + assert signed.dot2(ones, handle) == pytest.approx(16.0) + + # Two borrowed handles, both reversed, in the same call. + assert signed.dot2(signed.reversed_ptr, signed.reversed_ptr) == pytest.approx(204.0) + + +def test_an_optional_descriptor_dummy_accepts_omitted_none_and_reversed(signed): + """Absence is decided before anything is described.""" + assert signed.optional_total() == pytest.approx(-1.0) + assert signed.optional_total(None) == pytest.approx(-1.0) + assert signed.optional_total(_base()) == pytest.approx(36.0) + assert signed.optional_total(_base()[::-1]) == pytest.approx(36.0) + assert signed.optional_total(signed.reversed_ptr) == pytest.approx(36.0) + + +def test_assumed_rank_dummies_read_rank_and_size_from_the_descriptor(signed): + """An assumed-rank dummy accepts every supported rank and section direction.""" + assert signed.rank_and_size(_base()) == np.int32(108) + assert signed.rank_and_size(_matrix()) == np.int32(212) + assert signed.rank_and_size(_base()[::-1]) == np.int32(108) + assert signed.rank_and_size(_matrix()[::-1, ::-1]) == np.int32(212) + assert signed.rank_and_size(signed.reversed_ptr) == np.int32(108) + + +def test_a_character_dummy_reports_its_own_width_from_either_source(signed): + """Character arrays keep their runtime element width on the portable path.""" + assert signed.word_width(signed.words) == np.int32(4) + assert signed.word_width(np.array([b"abcd", b"efgh"], dtype="S4")) == np.int32(4) + + with pytest.raises(TypeError, match=r"runs backwards|cannot record a direction|expected ordering"): + signed.word_width(np.array([b"abcd", b"efgh"], dtype="S4")[::-1]) + + +def test_a_bound_handle_reaches_a_signed_stride_call_without_running_python(signed): + """The reversed handoff costs no Python frame once the arguments are parsed.""" + handle = signed.reversed_ptr + total1 = signed.total1 + called: list[str] = [] + + def record(frame, event, _arg): + if event == "call": + called.append(frame.f_code.co_name) + + total1(handle) + sys.setprofile(record) + try: + value = total1(handle) + finally: + sys.setprofile(None) + + assert called == [] + assert value == pytest.approx(36.0) + + +def test_raw_address_dummies_refuse_what_an_address_cannot_convey(signed): + """Each refusal names the restriction it comes from. + + An explicit-shape or assumed-size dummy receives the address of the first + element and nothing else, so a direction has nowhere to be recorded. A + CONTIGUOUS dummy keeps its requirement even though its calling convention + carries a descriptor. Neither is the same as a layout Fortran has no form + for at all. + """ + reversed_view = _base()[::-1] + + with pytest.raises(TypeError, match=r"contiguous|expected ordering"): + signed.explicit_total(reversed_view, np.int32(8)) + with pytest.raises(TypeError, match=r"contiguous|expected ordering"): + signed.flat_total(reversed_view, np.int32(8)) + with pytest.raises(TypeError, match=r"expected ordering|contiguous"): + signed.contig_total(reversed_view) + + # Positive strides still reach the ones that accept them. + assert signed.explicit_total(_base(), np.int32(8)) == pytest.approx(36.0) + assert signed.contig_total(_base()) == pytest.approx(36.0) + + +def test_layouts_that_are_not_array_sections_stay_refused(signed): + """A broadcast or overlapping view has no contiguous parent to be a section of.""" + broadcast = np.broadcast_to(np.arange(1.0, 4.0), (4, 3)) + assert broadcast.strides[0] == 0 + with pytest.raises(TypeError, match=r"not a Fortran array section"): + signed.checksum2(broadcast) + + overlapping = np.lib.stride_tricks.as_strided(_base(), shape=(4, 3), strides=(8, 8)) + with pytest.raises(TypeError, match=r"expected ordering|not a Fortran array section"): + signed.checksum2(overlapping) + + indivisible = np.ndarray((2, 2), dtype=np.float64, buffer=np.arange(8.0), strides=(16, 40)) + with pytest.raises(TypeError, match=r"not a Fortran array section"): + signed.checksum2(indivisible) + + +def test_absent_and_mismatched_storage_stay_refused(signed): + """State and type checks are unchanged by how the storage is handed over.""" + with pytest.raises(ValueError, match=r"unassociated"): + signed.total1(signed.unassociated_ptr) + with pytest.raises(TypeError, match=r"dtype"): + signed.total1(np.arange(4, dtype=np.float32)) + with pytest.raises(TypeError, match=r"dtype|rank"): + signed.total1(_matrix()) + with pytest.raises(TypeError, match=r"byte order"): + signed.total1(_base().astype(">f8")) + # word_width declares character(len=*), which takes whatever width it is + # given, so a different width is not a mismatch for it. + assert signed.word_width(np.array([b"abcdef"], dtype="S6")) == np.int32(6) diff --git a/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py b/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py index a4affb595..13e86542d 100644 --- a/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py +++ b/tests/fortran/data_types/codegen/test_primitive_scalar_result_lowering.py @@ -60,7 +60,8 @@ def test_direct_bool_result_normalizes_the_fortran_truth_bit_before_c_conversion assert "integer(c_int8_t) :: result" in fortran_source assert "logical(c_bool) :: c_result" in fortran_source assert "c_result = native_not_flag(value)" in fortran_source - assert "result = iand(transfer(c_result, 0_c_int8_t), 1_c_int8_t)" in fortran_source + # Reduced the way C converts to `_Bool`: any non-zero value is true. + assert "result = merge(1_c_int8_t, 0_c_int8_t, transfer(c_result, 0_c_int8_t) /= 0_c_int8_t)" in fortran_source def test_generator_rejects_a_non_normalized_direct_bool_result_abi(): diff --git a/tests/fortran/data_types/runtime/test_contract_scalar_constructors.py b/tests/fortran/data_types/runtime/test_contract_scalar_constructors.py index f3bacccdf..d637e2cbf 100644 --- a/tests/fortran/data_types/runtime/test_contract_scalar_constructors.py +++ b/tests/fortran/data_types/runtime/test_contract_scalar_constructors.py @@ -44,3 +44,20 @@ def test_primitive_contract_constructors_reject_values_and_array_annotations(): contracts.Float64[:]() with pytest.raises(TypeError, match="default constructor takes no arguments"): contracts.Int32(3) + + +def test_logical_descriptor_handles_report_their_native_array_width(): + cases = ( + (contracts.Bool, np.bool_), + (contracts.Bool8, np.bool_), + (contracts.Bool16, np.int16), + (contracts.Bool32, np.int32), + (contracts.Bool64, np.int64), + ) + + for element_type, dtype in cases: + handle = contracts.Allocatable[element_type[:]]() + try: + assert handle.dtype == np.dtype(dtype) + finally: + handle.close() diff --git a/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py b/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py new file mode 100644 index 000000000..771edfbd1 --- /dev/null +++ b/tests/fortran/derived_types/codegen/test_derived_array_field_lowering.py @@ -0,0 +1,136 @@ +"""Bridge lowering for live views over fixed derived-type array fields.""" + +from __future__ import annotations + +from prik.codegen.fortran.bridge import FortranBridgeGenerator +from prik.parsers.fortran.parser import parse_fortran_project +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.planning import WrapperPlanner +from prik.policy.completion import complete_semantic_policies +from prik.printers.fortran import FortranSourcePrinter +from prik.semantics.fortran2ir import fortran_project_to_semantic_modules + + +ARRAY_FIELD_SOURCE = """ +module field_state + use iso_fortran_env, only: real64 + implicit none + + type :: box + real(real64) :: grid(2, 3) + end type box + + type(box) :: plain_box + type(box), target :: tgt_box +end module field_state +""" + + +def _bridge_source_for(source: str, module_name: str) -> str: + parsed = parse_fortran_project({f"{module_name}.f90": source}) + modules = fortran_project_to_semantic_modules(parsed) + _apply_source_python_exports(modules) + module = _merge_wrapper_modules(modules, name=module_name) + complete_semantic_policies(module) + plan = WrapperPlanner().build(module) + return FortranSourcePrinter().visit(FortranBridgeGenerator().visit(plan)) + + +def _bridge_source(): + return _bridge_source_for(ARRAY_FIELD_SOURCE, "field_state") + + +def test_owned_array_field_takes_its_address_through_the_owner_pointer(): + """An owner reached as a pointer makes its components addressable. + + The owner arrives as an address and is associated with a Fortran pointer, so + its components are subobjects of a pointer target. `c_loc` may name them + whatever the field's own declaration said, and no capture is needed. + """ + source = _bridge_source() + + assert "result = c_loc(owner%grid)" in source + assert "extent_0 = int(size(owner%grid, 1), c_int64_t)" in source + + +def test_plain_module_object_field_captures_its_address_in_c(): + """A plain module object is named directly, so nothing about it is a target. + + `c_loc` cannot name a member of a module object that was declared without + `target`, so the address is taken on the C side, exactly as a non-addressable + module array's is. + """ + source = _bridge_source() + + assert "result = prik_capture_address(native_plain_box%grid)" in source + assert "c_loc(native_plain_box%grid)" not in source + + +def _procedure(source: str, name: str) -> str: + """Return the text of one generated procedure, by name.""" + start = source.index(f"function {name}(") + return source[start : source.index(f"end function {name}", start)] + + +def test_array_field_getters_report_extents_instead_of_passing_a_descriptor(): + """A fixed field's rank and contiguity are known, so extents carry everything. + + The earlier lowering handed the component to a C consumer as a descriptor + purely to reach its base address. A fixed component is contiguous and its + rank is fixed, so the base pointer plus one extent per axis says the same + thing without a callback round-trip, and the getter returns an address + rather than driving a consumer. + """ + source = _bridge_source() + + for name in ("bind_c_prik_field_box_grid_get", "bind_c_prik_module_field_plain_box_grid_get"): + getter = _procedure(source, name) + assert "type(c_ptr) :: result" in getter + assert "c_funptr" not in getter + assert "c_f_procpointer" not in getter + + # No descriptor-consumer interface is declared for an ordinary array field. + assert "_grid_consumer" not in source + + +def test_deferred_character_pointer_field_uses_only_legal_inquiry_entrypoints(): + source = _bridge_source_for( + """ +module deferred_field_state + implicit none + type :: box + character(len=:), pointer :: words(:) => null() + end type box + type(box) :: plain_box +end module deferred_field_state +""", + "deferred_field_state", + ) + + shape = _procedure(source, "bind_c_prik_field_handle_box_words_shape") + assert "logical(c_bool) :: result" in shape + assert "result = associated(owner%words)" in shape + assert "_words_consumer" not in source + assert "character(kind=c_char, len=:), pointer, dimension(:), intent(inout)" not in source + + +def test_fixed_character_descriptor_fields_use_guarded_ordinary_projections(): + source = _bridge_source_for( + """ +module fixed_character_fields + implicit none + type :: box + character(len=5), allocatable :: words(:) + character(len=5), pointer :: aliases(:) => null() + end type box + type(box) :: plain_box +end module fixed_character_fields +""", + "fixed_character_fields", + ) + + assert source.count("character(kind=c_char, len=*), dimension(:), intent(inout) :: value") == 4 + assert "if (allocated(owner%words)) then" in source + assert "if (associated(owner%aliases)) then" in source + assert "if (allocated(native_plain_box%words)) then" in source + assert "if (associated(native_plain_box%aliases)) then" in source diff --git a/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py b/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py index 350d2e436..3b7c282a1 100644 --- a/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py +++ b/tests/fortran/derived_types/end_to_end/test_module_derived_aliases.py @@ -6,7 +6,10 @@ import numpy as np import pytest from prik.runtime.handles import AllocatableArray -from tests.fortran._support.wrapper_build import _build_source_or_generated_pyi_and_import +from tests.fortran._support.wrapper_build import ( + _build_source_or_generated_pyi_and_import, + _build_text_and_import, +) FIXTURES = Path(__file__).parent / "fixtures" DERIVED_ALIAS_F90_SOURCE = FIXTURES / "native" / "fmodule_derived_alias_f90.f90" @@ -76,3 +79,68 @@ def test_aliased_derived_module_object_borrows_native_state( assert isinstance(current_values, AllocatableArray) assert current_values.allocated is False assert current_values.to_numpy() is None + + +PLAIN_DERIVED_ARRAY_FIELD_SOURCE = """ +module fplain_derived_fields_f90 + use iso_fortran_env, only: int32, real64 + implicit none + + type :: box + real(real64) :: grid(2, 3) + integer(int32) :: n + end type box + + type(box) :: plain_box + type(box), target :: tgt_box + +contains + function make_box(seed) result(value) + real(real64), intent(in) :: seed + type(box) :: value + value%grid = seed + value%n = 3 + end function make_box + + subroutine touch_plain() + plain_box%grid(1, 1) = plain_box%grid(1, 1) + 1.0d0 + end subroutine touch_plain +end module fplain_derived_fields_f90 +""" + + +def test_derived_array_fields_are_live_views_without_a_target_declaration(tmp_path: Path): + """An ordinary array component is borrowed live wherever its owner comes from. + + A derived field's address never comes from `c_loc` on the field, so `target` + on the component, the type, or the containing module variable changes + nothing. The owner is reached as a Fortran pointer and its component is + handed to a C consumer during the call, which is what makes the borrow work + for a wrapper-owned instance and for a plain module object alike. + """ + module = _build_text_and_import( + PLAIN_DERIVED_ARRAY_FIELD_SOURCE, + "fplain_derived_fields_f90.f90", + tmp_path, + { + "bind_c_fplain_derived_fields_f90_wrapper.f90", + "fplain_derived_fields_f90_wrapper.c", + "fplain_derived_fields_f90_wrapper.h", + }, + ) + + # A wrapper-owned instance borrows its own component storage. + instance = module.make_box(np.float64(2.0)) + assert instance.grid.shape == (2, 3) + assert instance.grid.flags["F_CONTIGUOUS"] is True + instance.grid[0, 0] = np.float64(9.0) + assert instance.grid[0, 0] == np.float64(9.0) + + # A module object without `target` borrows the same way, in both directions. + module.plain_box.grid[0, 0] = np.float64(5.0) + module.touch_plain() + assert module.plain_box.grid[0, 0] == np.float64(6.0) + + # The addressable module object is no different. + module.tgt_box.grid[1, 2] = np.float64(4.5) + assert module.tgt_box.grid[1, 2] == np.float64(4.5) diff --git a/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py b/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py new file mode 100644 index 000000000..e7c14e310 --- /dev/null +++ b/tests/fortran/infrastructure/building/compiling/test_logical_interop_flags.py @@ -0,0 +1,45 @@ +from pathlib import Path + +import pytest + +from prik.compiler.compilers import Compiler +from prik.compiler.objects import ObjectFile + + +def _fortran_compile_command(vendor: str, *, standard_logicals: bool, tmp_path: Path) -> tuple[str, ...]: + compiler = Compiler(vendor, execute_commands=False, standard_logicals=standard_logicals) + compiler._executable = lambda _language, _tools: "fc" + compiler.compile_object( + ObjectFile( + source=tmp_path / "module.f90", + object_path=tmp_path / "module.o", + language="fortran", + ) + ) + return compiler.command_log[0] + + +@pytest.mark.parametrize( + ("vendor", "option"), + [("intel", "-standard-semantics"), ("PGI", "-Munixlogical"), ("nvidia", "-Munixlogical")], +) +def test_fortran_compilation_requests_the_interoperable_logical_by_default(vendor: str, option: str, tmp_path: Path): + """A logical must reach C as 0 or 1, so the vendor option is on without being asked for.""" + assert option in _fortran_compile_command(vendor, standard_logicals=True, tmp_path=tmp_path) + + +@pytest.mark.parametrize( + ("vendor", "option"), + [("intel", "-standard-semantics"), ("PGI", "-Munixlogical"), ("nvidia", "-Munixlogical")], +) +def test_standard_logicals_can_be_turned_off_for_prebuilt_objects(vendor: str, option: str, tmp_path: Path): + """Opting out is the only way to link objects built without the option, whose mangling differs.""" + assert option not in _fortran_compile_command(vendor, standard_logicals=False, tmp_path=tmp_path) + + +@pytest.mark.parametrize("vendor", ["GNU", "LLVM"]) +def test_compilers_that_already_interoperate_add_no_logical_option(vendor: str, tmp_path: Path): + """gfortran and Flang already store .true. as 1, so they must stay flag-free.""" + command = _fortran_compile_command(vendor, standard_logicals=True, tmp_path=tmp_path) + assert "-standard-semantics" not in command + assert "-Munixlogical" not in command diff --git a/tests/fortran/infrastructure/building/pipeline/test_logical_interop_option_routing.py b/tests/fortran/infrastructure/building/pipeline/test_logical_interop_option_routing.py new file mode 100644 index 000000000..e961cb779 --- /dev/null +++ b/tests/fortran/infrastructure/building/pipeline/test_logical_interop_option_routing.py @@ -0,0 +1,82 @@ +"""Every build entrypoint that compiles Fortran must carry the logical-interop choice.""" + +import pytest + +from prik.pipeline import build as pipeline_build + + +@pytest.fixture +def recorded_compiler_options(monkeypatch): + """Capture the ``standard_logicals`` each entrypoint hands its compiler.""" + recorded: list[bool | None] = [] + real_new_compiler = pipeline_build._new_compiler + + def _record(*args, **kwargs): + recorded.append(kwargs.get("standard_logicals")) + raise _StopBuild + + monkeypatch.setattr(pipeline_build, "_new_compiler", _record) + assert real_new_compiler is not _record + return recorded + + +class _StopBuild(Exception): + """Raised in place of constructing a compiler, so no build work follows.""" + + +@pytest.mark.parametrize("standard_logicals", [True, False]) +def test_fortran_source_build_hands_the_choice_to_its_compiler( + tmp_path, recorded_compiler_options, standard_logicals: bool +): + source = tmp_path / "flags.f90" + source.write_text("module flags\n logical :: on = .true.\nend module flags\n") + + with pytest.raises(_StopBuild): + pipeline_build.build_fortran_extension( + [source], output_dir=tmp_path / "out", standard_logicals=standard_logicals + ) + + assert recorded_compiler_options == [standard_logicals] + + +@pytest.mark.parametrize("standard_logicals", [True, False]) +def test_contract_build_hands_the_choice_to_its_compiler(tmp_path, recorded_compiler_options, standard_logicals: bool): + """A contract build compiles the native Fortran too, so it must not drop the choice.""" + package = tmp_path / "contract" + package.mkdir() + (package / "__init__.pyi").write_text("from . import flags\n") + (package / "flags.pyi").write_text("from prik.contracts import Bool8\n\non: Bool8\n") + native = tmp_path / "flags.f90" + native.write_text("module flags\n logical :: on = .true.\nend module flags\n") + + with pytest.raises(_StopBuild): + pipeline_build.build_pyi_extension( + package / "__init__.pyi", + output_dir=tmp_path / "out", + native_fortran_sources=[native], + standard_logicals=standard_logicals, + ) + + assert recorded_compiler_options == [standard_logicals] + + +@pytest.mark.parametrize("standard_logicals", [True, False]) +def test_c_build_with_fortran_dependencies_hands_the_choice_to_its_compiler( + tmp_path, recorded_compiler_options, standard_logicals: bool +): + """A C build that links Fortran compiles it with the same compiler, so the choice applies.""" + source = tmp_path / "api.c" + source.write_text("int answer(void) { return 1; }\n") + native = tmp_path / "flags.f90" + native.write_text("module flags\n logical :: on = .true.\nend module flags\n") + + with pytest.raises(_StopBuild): + pipeline_build.build_c_extension( + source, + output_dir=tmp_path / "out", + output_name="c_logical_option", + native_fortran_sources=[native], + standard_logicals=standard_logicals, + ) + + assert recorded_compiler_options == [standard_logicals] diff --git a/tests/fortran/infrastructure/cli/pipeline/_support.py b/tests/fortran/infrastructure/cli/pipeline/_support.py index 1f91fbd93..c4902d63e 100644 --- a/tests/fortran/infrastructure/cli/pipeline/_support.py +++ b/tests/fortran/infrastructure/cli/pipeline/_support.py @@ -1,3 +1,4 @@ +import sys import types import prik.cli as prik_cli @@ -81,6 +82,9 @@ def error(self, message): parser = FakeParser() monkeypatch.setattr(prik_cli, "_parser_for_argv", lambda argv: (parser, argv)) + # main() falls back to sys.argv when called without an argv, and an empty + # command line prints help instead of dispatching. + monkeypatch.setattr(sys, "argv", ["prik", "input.f90"]) return parser diff --git a/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py b/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py index 4c87bdee1..ff7a91d81 100644 --- a/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py +++ b/tests/fortran/infrastructure/cli/pipeline/test_argument_contract.py @@ -321,6 +321,29 @@ def test_source_build_routes_disabled_input_compilation_to_the_pipeline(monkeypa assert calls[0][1]["native_objects"] == ["libnative.so"] +@pytest.mark.parametrize( + ("cli_arguments", "forwarded"), + [({}, True), ({"standard_logicals": False}, False)], +) +def test_source_build_routes_the_logical_interop_choice_to_the_pipeline( + monkeypatch, cli_arguments: dict[str, bool], forwarded: bool +): + """--no-standard-logicals is the only opt-out, so it must reach the build unchanged.""" + from prik.pipeline import build as pipeline_build + + calls = [] + result = types.SimpleNamespace(compiled=False) + monkeypatch.setattr( + pipeline_build, + "build_fortran_extension", + lambda *args, **kwargs: calls.append((args, kwargs)) or result, + ) + args = _main_args(paths=[str(TEST_FILE)], **cli_arguments) + + assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler="gfortran")) is result + assert calls[0][1]["standard_logicals"] is forwarded + + def test_fortran_pyi_build_defers_c_driver_selection_to_the_compiler_pair(monkeypatch): from prik.pipeline import build as pipeline_build @@ -342,6 +365,29 @@ def test_fortran_pyi_build_defers_c_driver_selection_to_the_compiler_pair(monkey assert calls[0][1]["input_c_compiler"] is None +@pytest.mark.parametrize( + ("cli_arguments", "forwarded"), + [({}, True), ({"standard_logicals": False}, False)], +) +def test_pyi_contract_build_routes_the_logical_interop_choice_to_the_pipeline( + monkeypatch, cli_arguments: dict[str, bool], forwarded: bool +): + """A contract build compiles the same native Fortran, so it must carry the same choice.""" + from prik.pipeline import build as pipeline_build + + calls = [] + result = types.SimpleNamespace(compiled=False) + monkeypatch.setattr( + pipeline_build, + "build_pyi_extension", + lambda *args, **kwargs: calls.append((args, kwargs)) or result, + ) + args = _main_args(paths=["contract.pyi"], language="fortran", **cli_arguments) + + assert prik_cli._run_wrap_build(args, types.SimpleNamespace(compiler="gfortran")) is result + assert calls[0][1]["standard_logicals"] is forwarded + + @pytest.mark.parametrize( ("native_language", "expected_compilers"), [ diff --git a/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py b/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py index 00d75921d..1d958ee5b 100644 --- a/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py +++ b/tests/fortran/infrastructure/codegen/test_native_entrypoint_routing.py @@ -5,6 +5,7 @@ from prik.planning import NativeGeneratedCodeGroupKind, WrapperPlanner from prik.policy import complete_semantic_policies from prik.policy.models import ( + ArrayEntrypointABI, EntrypointPassingConvention, EntrypointProjectionAction, NativeEntrypointAction, @@ -48,6 +49,40 @@ def test_direct_plan_keeps_one_projected_sequence_and_no_adapter_facets(): assert slot.passing is EntrypointPassingConvention.C_VALUE +def test_bind_c_descriptor_arrays_call_the_user_symbol_without_an_adapter(): + plan = _plan( + """ +module direct_descriptor_arrays + use iso_c_binding +contains + integer(c_int) function fixed_rank(values) bind(C, name="direct_fixed_rank") result(output) + real(c_double), intent(in) :: values(:) + output = int(size(values), c_int) + end function fixed_rank + + integer(c_int) function any_rank(values) bind(C, name="direct_any_rank") result(output) + real(c_double), intent(in) :: values(..) + output = int(rank(values), c_int) + end function any_rank +end module direct_descriptor_arrays +""" + ) + + assert plan.bridge is None + for function in plan.namespaces[0].functions: + argument = function.arguments[0] + assert function.entrypoint.action is NativeEntrypointAction.DIRECT_C_ABI + assert function.bridge is None + assert argument.array.entrypoint_abi is ArrayEntrypointABI.C_DESCRIPTOR + assert argument.entrypoint.passing is EntrypointPassingConvention.C_DESCRIPTOR_POINTER + + generated = WrapperGenerator().generate(plan) + binding = next(source.text for source in generated.sources if source.path.suffix == ".c") + assert generated.required_headers == ("ISO_Fortran_binding.h",) + assert "int32_t direct_fixed_rank(CFI_cdesc_t * values);" in binding + assert "int32_t direct_any_rank(CFI_cdesc_t * values);" in binding + + def test_adapted_plan_attaches_one_narrow_adapter_to_the_shared_projection(): plan = _plan( """ diff --git a/tests/fortran/infrastructure/runtime/test_native_support.py b/tests/fortran/infrastructure/runtime/test_native_support.py index ce00489fc..f1c78d128 100644 --- a/tests/fortran/infrastructure/runtime/test_native_support.py +++ b/tests/fortran/infrastructure/runtime/test_native_support.py @@ -1,5 +1,7 @@ """Public native-binding support surface checks.""" +import re + from tests.fortran._support.paths import REPO_ROOT @@ -10,29 +12,15 @@ def test_native_binding_support_is_header_only_and_exposes_the_small_prik_api(): header = SUPPORT_HEADER.read_text(encoding="utf-8") assert not SUPPORT_SOURCE.exists() - assert '#define PRIK_NATIVE_ARRAY_HANDLE_CAPSULE_NAME "prik.native_array_handle.v1"' in header - assert "#define PRIK_NATIVE_ARRAY_HANDLE_ABI_VERSION 1u" in header - assert "typedef struct {" in header - assert "prik_native_array_release_fn release;" in header - - expected_api = ( - "prik_native_array_handle_release", - "prik_native_array_handle_capsule_destructor", - "prik_native_array_handle_capsule_new", - "prik_native_array_handle_from_capsule", - "prik_array_actual_unpack", - "prik_array_validate", - "prik_release_owned_memory", - ) - for name in expected_api: - assert name in header - assert "PRIK_NO_INLINE static int prik_array_actual_unpack(" in header + assert "static inline int prik_array_validate(" in header assert "static inline int prik_array_validate_ndarray(" in header assert "PyArrayObject *array," in header assert header.count("PyArray_Check(value)") == 1 assert "PRIK_ARRAY_LAYOUT_POSITIVE_STRIDED_F" in header assert "prik_array_actual" in header + assert "prik_release_owned_memory" in header + assert "prik_capture_address" in header scalar_suffixes = ( "bool", @@ -50,3 +38,121 @@ def test_native_binding_support_is_header_only_and_exposes_the_small_prik_api(): assert f"prik_{suffix}_unpack" in header assert f"prik_{suffix}_to_python" in header assert f"prik_{suffix}_to_numpy" in header + + +BACKEND_RECORD = ( + ("uint32_t", "descriptor_kind"), + ("uint32_t", "descriptor_attribute"), + ("uint32_t", "rank"), + ("uint32_t", "descriptor_size"), + ("int32_t", "cfi_type"), + ("size_t", "element_size"), + ("void *", "context"), + ("prik_native_array_with_descriptor_fn", "with_descriptor"), + ("prik_native_array_release_fn", "release"), +) + + +def _backend_record_fields(header: str) -> tuple[tuple[str, str], ...]: + """Return the declared record in order, as (type, name) pairs.""" + # A struct body has no braces of its own, so this cannot span the record before it. + body = re.search(r"typedef struct \{\n([^{}]*?)\n\} prik_native_array_backend;", header, re.S) + assert body is not None, "prik_native_array_backend is not declared as one struct" + fields = [] + for line in body.group(1).strip().splitlines(): + spelling, _, name = line.strip().rstrip(";").rpartition(" ") + if name.startswith("*"): + spelling, name = f"{spelling} *", name[1:] + fields.append((spelling.strip(), name)) + return tuple(fields) + + +def _layout_tag_members(header: str) -> tuple[str, ...]: + """Return the field names the layout tag folds, in the order it folds them.""" + body = re.search(r"\} layout\[\] = \{(.*?)\};", header, re.S) + assert body is not None, "the layout tag does not declare what it folds" + return tuple(re.findall(r"PRIK_NATIVE_ARRAY_BACKEND_FIELD\((\w+)\)", body.group(1))) + + +def test_the_capsule_name_covers_semantic_version_and_the_whole_record(): + """The name protects the callback contract and the compiled record layout.""" + header = SUPPORT_HEADER.read_text(encoding="utf-8") + + assert '#define PRIK_NATIVE_ARRAY_BACKEND_CAPSULE_PREFIX "prik.native_array_backend.v1"' in header + assert _backend_record_fields(header) == BACKEND_RECORD + assert _layout_tag_members(header) == tuple(name for _spelling, name in BACKEND_RECORD) + # Name, offset and width all come from the one token naming the field. + assert ( + "{#member, offsetof(prik_native_array_backend, member), " + "sizeof(((prik_native_array_backend *)0)->member)}" in header + ) + # The size goes in first, so a change that only moves the tail is caught too. + assert "tag = (tag ^ (uint64_t)sizeof(prik_native_array_backend))" in header + # And the name is folded, which is what makes a rename reach every reader. + assert "for (character = layout[index].name; *character != '\\0'; ++character)" in header + + +def test_native_array_backend_capsule_exposes_one_entry_point_and_its_readers(): + """One entry point reaches the descriptor; the readers validate a producer. + + The context is what that entry point needs to get there, and a release + marks that context as this extension's to free. + """ + header = SUPPORT_HEADER.read_text(encoding="utf-8") + + for name in ( + "prik_native_array_backend_capsule_new", + "prik_native_array_backend_capsule_destructor", + "prik_native_array_backend_from_capsule", + "prik_native_array_backend_for_descriptor", + "prik_native_array_backend_for_actual", + "prik_native_array_backend_owned_descriptor", + "prik_native_array_backend_layout_tag", + "prik_native_array_backend_capsule_name", + "prik_native_array_backend_release", + "prik_native_array_owned_with_descriptor", + ): + assert name in header + + assert "invalid prik native array descriptor attribute" in header + assert "backend->descriptor_attribute != expected_descriptor_attribute" in header + assert "does not expose the descriptor attribute required by the dummy argument" in header + + +def test_native_array_backend_release_is_idempotent_and_never_frees_borrowed_storage(): + """Owned storage is released once; borrowed storage is never released here. + + A backend owns its context exactly when it carries a release callback, so a + module variable's or a field's backend -- which carries none -- cannot reach + the free below, and clearing the context makes an explicit close() and + finalization both safe. + """ + header = SUPPORT_HEADER.read_text(encoding="utf-8") + start = header.index("static inline void prik_native_array_backend_release(") + body = header[start : header.index("\n}\n", start)] + + assert "backend->release == NULL || backend->context == NULL" in body + assert "backend->context = NULL;" in body + assert "backend->release(context);" in body + assert "free(context);" in body + # A released owned backend reports itself closed rather than handing over + # storage that is gone. + reader = header[header.index("static inline prik_native_array_backend *prik_native_array_backend_from_capsule(") :] + reader = reader[: reader.index("\n}\n")] + assert "backend->release != NULL && backend->context == NULL" in reader + assert "prik native array handle is closed" in reader + + +def test_address_capture_primitive_has_external_linkage_behind_one_opt_in(): + """The one support symbol the generated Fortran bridge links against. + + Every other helper here is `static`, which the bridge could not call. This + one stands in for `c_loc` where Fortran cannot form it, so it must be + externally visible -- and therefore defined in exactly one translation unit, + which the opt-in macro is what enforces. + """ + header = SUPPORT_HEADER.read_text(encoding="utf-8") + + assert "#ifdef PRIK_BINDING_CAPTURE_ADDRESS" in header + assert "void *prik_capture_address(void *base)" in header + assert "static inline void *prik_capture_address" not in header diff --git a/tests/fortran/memory_management/codegen/test_native_handle_planning.py b/tests/fortran/memory_management/codegen/test_native_handle_planning.py index 10d823028..9d97621c5 100644 --- a/tests/fortran/memory_management/codegen/test_native_handle_planning.py +++ b/tests/fortran/memory_management/codegen/test_native_handle_planning.py @@ -189,6 +189,11 @@ def test_native_handle_plans_keep_datatype_specific_state(): alloc = functions["alloc"].arguments[0] pointer = functions["pointer"].arguments[0] + # Neither kind is established from C. An allocatable cannot be: the + # standard requires a null base address for that attribute. A pointer + # could be, but a descriptor C built is not the caller's entity, so a + # callee that re-associates the dummy would change only that copy. Both + # therefore take the descriptor the Fortran runtime made. for argument, descriptor_kind in ( (alloc, NativeArrayDescriptorKind.ALLOCATABLE), (pointer, NativeArrayDescriptorKind.POINTER), @@ -197,12 +202,13 @@ def test_native_handle_plans_keep_datatype_specific_state(): assert handle is not None assert handle is argument.projected_call_slot.native_array_handle assert handle.descriptor_kind is descriptor_kind - assert handle.handoff.abi is NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL - assert handle.default_handle.construction is NativeArrayDefaultConstruction.FACT_PACKED_EMPTY + assert handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR + assert handle.default_handle.construction is NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR assert handle.default_handle.descriptor_ownership is NativeArrayDescriptorOwnership.OWNED - assert handle.default_handle.owner_storage_role is None + # Lazily attached storage is the wrapper's, so the plan names the slot + # the generated binder allocates and the handle's finalizer releases. + assert handle.default_handle.owner_storage_role == f"{argument.owner_path}:default-owner-storage" assert NativeArrayOperation.DESTROY in handle.default_handle.operations - assert len(handle.handoff.extent_roles) == handle.array.rank == 1 assert argument.binding.python_action is PythonBarrierAction.WRAPPER_INSTANCE assert argument.entrypoint.handoff_mode is ArgumentHandoffMode.NATIVE_DESCRIPTOR @@ -217,7 +223,6 @@ def test_native_handle_plans_keep_datatype_specific_state(): assert replacement.native_array_handle is not None assert replacement.native_array_handle.handoff.abi is NativeDescriptorHandoffABI.DIRECT_STANDARD_DESCRIPTOR assert replacement.native_array_handle.output_projection is NativeArrayOutputProjection.PROJECTED_HANDLE - assert replacement.native_array_handle.handoff.extent_roles == () assert ( replacement.native_array_handle.default_handle.construction is NativeArrayDefaultConstruction.LAZY_OWNED_DESCRIPTOR @@ -320,7 +325,11 @@ def test_module_variables_use_borrowed_handle_plans_and_operation_sets(): assert NativeArrayOperation.ELEMENT_LENGTH in names.operations assert NativeArrayOperation.RESIZE not in names.operations assert NativeArrayOperation.DESTROY not in pointer.operations - assert allocatable.required_headers == () + # A module allocatable reads its own descriptor whether or not it is a + # target, so `Aliased` selects the same interop and headers as a plain one. + assert allocatable.extraction_action.value == "descriptor_view" + assert allocatable.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR + assert allocatable.required_headers == ("ISO_Fortran_binding.h",) assert plain.extraction_action.value == "descriptor_view" assert plain.descriptor_interop is NativeArrayDescriptorInterop.MODULE_ALLOCATABLE_C_DESCRIPTOR assert plain.required_headers == ("ISO_Fortran_binding.h",) @@ -329,14 +338,12 @@ def test_module_variables_use_borrowed_handle_plans_and_operation_sets(): def test_deferred_character_module_handles_use_runtime_element_length(): + """The live descriptor supplies a deferred character element width.""" artifacts = WrapperGenerator().generate(_module_handle_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") - assert "bind_c_module_names_element_length()" in c_source - assert '"elem_len", (unsigned long long)(bind_c_module_names_element_length())' in c_source - assert "function bind_c_module_names_element_length() result(result)" in bridge_source - assert "result = len(native_module_names, kind=c_int64_t)" in bridge_source + assert "out->result = PyLong_FromLongLong((long long)source->elem_len)" in c_source + assert "prik_native_array_read_element_length" in c_source def test_generated_native_handle_artifacts_follow_one_typed_action_vocabulary(): @@ -345,19 +352,18 @@ def test_generated_native_handle_artifacts_follow_one_typed_action_vocabulary(): bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") assert artifacts.required_headers == ("ISO_Fortran_binding.h",) - assert "prik_bind_array(" in c_source - assert '"_native_array_descriptor_argument_for_binding_positional"' in c_source - assert '"_native_array_descriptor_handoff_for_binding_positional"' in c_source - assert '"_native_array_handle_from_generated_ops"' in c_source + assert "prik_describe_numpy_array(" in c_source + assert '"_native_array_backend_for_binding_positional"' in c_source + assert '"_native_array_handle_from_generated_dispatch"' in c_source assert '"_bind_contract_native_array_handle"' in c_source - assert "prik_native_array_handle_capsule_new(" in c_source - assert "prik_native_array_handle_from_capsule(" in c_source + assert "prik_native_array_backend_capsule_new(" in c_source + assert "prik_native_array_backend_for_descriptor(" in c_source assert "PRIK_NATIVE_ARRAY_KIND_ALLOCATABLE" in c_source assert "PRIK_NATIVE_ARRAY_KIND_POINTER" in c_source - assert "prik_native_array_handle_release(owner_handle)" in c_source - assert "bound_values_native_handle = prik_native_array_handle_from_capsule(bound_values_item" in c_source + assert "prik_native_array_backend_release(owner_backend)" in c_source + assert ("bound_values_native_backend = prik_native_array_backend_for_descriptor(bound_values_item") in c_source assert "prik_bind_default_memory_handles_replace_values" in c_source - assert "prik_owned_memory_handles_replace_values_destroy" in c_source + assert "prik_owned_memory_handles_replace_values_dispatch" in c_source assert "bound_values_default_binder" in c_source assert "CFI_CDESC_T(1)" in c_source assert "CFI_CDESC_T(2)" in c_source @@ -374,53 +380,53 @@ def test_generated_native_handle_artifacts_follow_one_typed_action_vocabulary(): optional_c_end = c_source.index("static PyObject * wrap_replace(", optional_c_start) optional_binding = c_source[optional_c_start:optional_c_end] assert "} else {" in optional_binding - assert "bound_values_elem_len = sizeof(double);" in optional_binding - assert "bound_values_descriptor_rank = 1;" in optional_binding + # The absent branch hands the bridge an unallocated placeholder to pair + # with its present flag. A null base address is the only form the standard + # lets C establish for this attribute, and absence is when there is nothing + # to point at. + assert "CFI_establish((CFI_cdesc_t *)&bound_values_storage, NULL, CFI_attribute_allocatable" in optional_binding assert "bound_values = (CFI_cdesc_t *)&bound_values_storage;" in optional_binding assert "result_value = native_make(n)" in bridge_source assert "result_value = native_make_matrix(n, m)" in bridge_source assert "call prik_collect_allocatable_array_result(native_maybe_make(n), result)" in bridge_source assert "if (allocated(value)) then" in bridge_source assert "call move_alloc(value, result)" in bridge_source - assert "allocated(CFI_cdesc_t * result);" in c_source - assert "_allocated(owner_descriptor));" in c_source assert "_deallocate(owner_descriptor);" in c_source assert "_destroy(owner_descriptor);" in c_source - assert "_shape(owner_descriptor, &extent_0);" in c_source + assert "owner_backend->with_descriptor(owner_backend->context, prik_native_array_read_shape" in c_source assert "character(kind=c_char, len=:), allocatable :: value_value" in bridge_source assert "result_itemsize" in c_source assert "CFI_type_char" in c_source assert "character(kind=c_char, len=:), allocatable, dimension(:) :: names" in bridge_source assert "result_owner_status = CFI_establish(result, NULL, CFI_attribute_pointer" in c_source assert ( - "PRIK_NATIVE_ARRAY_KIND_POINTER, 1, CFI_type_double, sizeof(double), sizeof(CFI_CDESC_T(1)), result" in c_source + "PRIK_NATIVE_ARRAY_KIND_POINTER, PRIK_NATIVE_ARRAY_ATTRIBUTE_POINTER, 1, " + "(uint32_t)sizeof(CFI_CDESC_T(1)), CFI_type_double, " + "sizeof(double), result" in c_source ) -def test_constant_owned_handle_operations_do_not_emit_unused_descriptor_locals(): +def test_owned_descriptor_handles_publish_one_dispatcher_and_capability_tuple(): artifacts = WrapperGenerator().generate(_native_handle_plan()) c_source = next(source.text for source in artifacts.sources if source.path.suffix == ".c") - for operation in ("aligned", "descriptor", "destroy", "layout", "native_byte_order", "writeable"): - function = _generated_c_function( - c_source, - f"prik_owned_memory_handles_make_return_{operation}", - ) - assert "owner_handle" in function - assert "owner_descriptor" not in function - - allocated = _generated_c_function( + dispatch = _generated_c_function( c_source, - "prik_owned_memory_handles_make_return_allocated", + "prik_owned_memory_handles_make_return_dispatch", ) - assert "owner_descriptor" in allocated + assert 'strcmp(operation, "allocated") == 0' in dispatch + assert 'strcmp(operation, "shape") == 0' in dispatch + assert 'strcmp(operation, "to_numpy") == 0' in dispatch + assert 'strcmp(operation, "destroy") == 0' in dispatch + assert "owner_backend" in dispatch + assert "owner_descriptor" in dispatch + assert 'Py_BuildValue("(ssssss)", "allocated", "deallocate", "destroy", "resize", "shape", "to_numpy")' in c_source @pytest.mark.parametrize( ("edit", "diagnostic"), [ ("required_presence", "inconsistent-native-descriptor-presence"), - ("projected_facts", "invalid-direct-native-descriptor-roles"), ("owned_storage", "invalid-owned-native-descriptor-roles"), ("default_storage", "inconsistent-default-handle-owner-storage-role"), ("disabled_default", "invalid-disabled-default-handle-policy"), @@ -438,8 +444,6 @@ def test_native_handle_plan_edits_fail_central_validation(edit: str, diagnostic: functions = _functions(plan) if edit == "required_presence": functions["alloc"].arguments[0].native_array_handle.handoff.presence_role = "edited:present" - elif edit == "projected_facts": - functions["replace"].arguments[0].native_array_handle.handoff.extent_roles = ("edited:extent",) elif edit == "owned_storage": functions["make"].results[0].native_array_handle.handoff.owner_storage_role = None elif edit == "default_storage": @@ -463,7 +467,7 @@ def test_native_handle_plan_edits_fail_central_validation(edit: str, diagnostic: elif edit == "default_abi": functions["replace"].arguments[ 0 - ].native_array_handle.handoff.abi = NativeDescriptorHandoffABI.FACT_PACKED_CALL_LOCAL + ].native_array_handle.handoff.abi = NativeDescriptorHandoffABI.OWNED_RESULT_STORAGE elif edit == "operation": functions["pointer"].arguments[0].native_array_handle.operations = () else: diff --git a/tests/fortran/memory_management/runtime/test_handle_lifecycle.py b/tests/fortran/memory_management/runtime/test_handle_lifecycle.py index e186c9a33..a444a17ef 100644 --- a/tests/fortran/memory_management/runtime/test_handle_lifecycle.py +++ b/tests/fortran/memory_management/runtime/test_handle_lifecycle.py @@ -1,23 +1,23 @@ """Runtime ownership, factory, close, and finalizer behavior for native handles.""" -import ctypes import gc import numpy as np import pytest from prik.runtime.handles import ( AllocatableArray, PointerArray, - _native_array_descriptor_handoff_for_binding, - _native_array_handle_from_generated_ops, + _native_array_backend_for_binding, + _native_array_handle_from_generated_dispatch, ) from tests.fortran._support.native_array_handles import ( _ArrayState, _common_ops, - _required_handoff_ops, + _generated_handle_dispatch, + _handle_dispatch, ) -def test_generated_handle_factory_adapts_private_operations_to_runtime_protocol(): +def test_generated_handle_factory_adapts_one_dispatcher_to_runtime_protocol(): owner = object() value = np.arange(3, dtype=np.float64) calls = [] @@ -26,14 +26,6 @@ def shape(): calls.append(("shape", ())) return (3,) - def array_actual(): - calls.append(("array_actual", ())) - return 1001 - - def descriptor(): - calls.append(("descriptor", ())) - return ctypes.c_void_p(1002) - def allocated(): calls.append(("allocated", ())) return True @@ -42,17 +34,17 @@ def to_numpy(): calls.append(("to_numpy", ())) return value - handle = _native_array_handle_from_generated_ops( + operations = { + "shape": shape, + "allocated": allocated, + "to_numpy": to_numpy, + } + handle = _native_array_handle_from_generated_dispatch( "allocatable", "float64", 1, - { - "shape": shape, - "array_actual": array_actual, - "descriptor": descriptor, - "allocated": allocated, - "to_numpy": to_numpy, - }, + _generated_handle_dispatch(operations), + operations, owner=owner, descriptor_ownership="borrowed", to_numpy_policy="borrowed_view", @@ -65,42 +57,25 @@ def to_numpy(): assert handle.owner is owner assert handle.generation == 9 assert handle.shape == (3,) + assert handle.allocated is True assert handle.to_numpy() is value - assert handle._array_actual_for_binding(expected_dtype="float64", expected_rank=1).address == 1001 - assert handle._descriptor_for_binding(expected_dtype="float64", expected_rank=1) == { - "base_addr": 1002, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 3, "sm": 8}], - } - assert calls == [ - ("allocated", ()), - ("shape", ()), - ("allocated", ()), - ("to_numpy", ()), - ("allocated", ()), - ("allocated", ()), - ("shape", ()), - ("array_actual", ()), - ("allocated", ()), - ("shape", ()), - ("descriptor", ()), - ] + assert {name for name, _args in calls} == {"allocated", "shape", "to_numpy"} + assert all(args == () for _name, args in calls) def test_generated_handle_factory_splats_shape_operations_to_scalar_extents(): calls = [] - handle = _native_array_handle_from_generated_ops( + operations = { + "shape": lambda: (2, 3), + "allocated": lambda: True, + "resize": lambda *extents: calls.append(("resize", extents)), + } + handle = _native_array_handle_from_generated_dispatch( "allocatable", "float64", 2, - { - "shape": lambda: (2, 3), - "array_actual": lambda: 1001, - "descriptor": lambda: 1002, - "allocated": lambda: True, - "resize": lambda *extents: calls.append(("resize", extents)), - }, + _generated_handle_dispatch(operations), + operations, to_numpy_policy="unsupported", ) @@ -121,27 +96,28 @@ def call(received_owner, *args): return call - handle = _native_array_handle_from_generated_ops( + operations = { + "shape": operation("shape", (3,)), + "allocated": operation("allocated", True), + "to_numpy": operation("to_numpy", value), + "resize": operation("resize"), + "destroy": operation("destroy"), + } + handle = _native_array_handle_from_generated_dispatch( "allocatable", "float64", 1, - { - "shape": operation("shape", (3,)), - "array_actual": operation("array_actual", 0x5678), - "descriptor": operation("descriptor", owner), - "allocated": operation("allocated", True), - "to_numpy": operation("to_numpy", value), - "resize": operation("resize"), - "destroy": operation("destroy"), - }, + _generated_handle_dispatch(operations), + operations, owner=owner, descriptor_ownership="owned", + native_backend=owner, ) assert handle.shape == (3,) + assert handle.allocated is True assert handle.to_numpy() is value - assert handle._array_actual_for_binding().address == 0x5678 - assert _native_array_descriptor_handoff_for_binding( + assert _native_array_backend_for_binding( handle, descriptor_kind="allocatable", expected_dtype=np.float64, @@ -150,65 +126,32 @@ def call(received_owner, *args): handle.resize((5,)) handle.close() - assert calls == [ - ("allocated", owner, ()), - ("shape", owner, ()), - ("allocated", owner, ()), - ("to_numpy", owner, ()), - ("allocated", owner, ()), - ("allocated", owner, ()), - ("shape", owner, ()), - ("array_actual", owner, ()), - ("allocated", owner, ()), - ("shape", owner, ()), - ("descriptor", owner, ()), - ("resize", owner, (np.int64(5),)), - ("destroy", owner, ()), - ] - - -def test_generated_owned_handle_normalizes_compiler_zero_extent_descriptor_records(): - owner = 1234 - descriptor = { - "base_addr": 5678, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 1, "extent": -1, "sm": 8}], + assert {name for name, _owner, _args in calls} == { + "allocated", + "shape", + "to_numpy", + "resize", + "destroy", } - handle = _native_array_handle_from_generated_ops( - "allocatable", - "float64", - 1, - { - "shape": lambda _native_owner: descriptor, - "array_actual": lambda _native_owner: 5678, - "descriptor": lambda _native_owner: 5678, - "to_numpy": lambda _native_owner: descriptor, - "allocated": lambda _native_owner: True, - "destroy": lambda _native_owner: None, - }, - owner=owner, - descriptor_ownership="owned", - ) - - assert handle.shape == (0,) - assert handle.to_numpy().shape == (0,) + assert all(received_owner is owner for _name, received_owner, _args in calls) + assert ("resize", owner, (np.int64(5),)) in calls + assert calls.count(("destroy", owner, ())) == 1 def test_generated_handle_resolves_deferred_character_dtype_from_runtime_element_length(): state = {"itemsize": 3} - handle = _native_array_handle_from_generated_ops( + operations = { + "shape": lambda: (2,), + "element_length": lambda: state["itemsize"], + "allocated": lambda: True, + "to_numpy": lambda: np.array([b"red", b"sky"], dtype=f"S{state['itemsize']}"), + } + handle = _native_array_handle_from_generated_dispatch( "allocatable", None, 1, - { - "shape": lambda: (2,), - "element_length": lambda: state["itemsize"], - "array_actual": lambda: 0x1234, - "descriptor": lambda: 0x1234, - "allocated": lambda: True, - "to_numpy": lambda: np.array([b"red", b"sky"], dtype=f"S{state['itemsize']}"), - }, + _generated_handle_dispatch(operations), + operations, ) assert handle.dtype == np.dtype("S3") @@ -223,17 +166,17 @@ def test_generated_owned_handle_factory_releases_owner_once_when_construction_fa def destroy(received_owner): calls.append(("destroy", received_owner)) - with pytest.raises(ValueError, match="requires generated operation 'descriptor'"): - _native_array_handle_from_generated_ops( + with pytest.raises(ValueError, match="requires generated operation 'allocated'"): + operations = { + "shape": lambda _owner: (1,), + "destroy": destroy, + } + _native_array_handle_from_generated_dispatch( "allocatable", "float64", 1, - { - "shape": lambda _owner: (1,), - "array_actual": lambda _owner: 0x5678, - "allocated": lambda _owner: True, - "destroy": destroy, - }, + _generated_handle_dispatch(operations), + operations, owner=owner, descriptor_ownership="owned", to_numpy_policy="unsupported", @@ -243,21 +186,21 @@ def destroy(received_owner): assert calls == [("destroy", owner)] -def test_generated_handle_factory_rejects_invalid_descriptor_kind_and_handoff_result(): +def test_generated_handle_factory_rejects_an_invalid_descriptor_kind(): ops = { "shape": lambda: (1,), - "array_actual": lambda: object(), - "descriptor": lambda: 1, "allocated": lambda: True, "to_numpy": lambda: np.zeros(1, dtype=np.float64), } with pytest.raises(ValueError, match="generated native array handle kind"): - _native_array_handle_from_generated_ops("target", "float64", 1, ops) - - handle = _native_array_handle_from_generated_ops("allocatable", "float64", 1, ops) - with pytest.raises(TypeError, match="handoff address must be an integer"): - handle._array_actual_for_binding(expected_dtype="float64", expected_rank=1) + _native_array_handle_from_generated_dispatch( + "target", + "float64", + 1, + _generated_handle_dispatch(ops), + ops, + ) def test_owned_handle_close_calls_destroy_once_and_blocks_later_use(): @@ -266,11 +209,13 @@ def test_owned_handle_close_calls_destroy_once_and_blocks_later_use(): handle = AllocatableArray( dtype="float64", rank=1, - ops={ - **_common_ops(state), - "allocated": lambda _handle: True, - "destroy": lambda _handle: calls.append(("destroy", _handle.shape, _handle.to_numpy())), - }, + **_handle_dispatch( + { + **_common_ops(state), + "allocated": lambda _handle: True, + "destroy": lambda _handle: calls.append(("destroy", state.shape, state.value)), + } + ), descriptor_ownership="owned", ) @@ -295,12 +240,13 @@ def destroy(_handle): handle = AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "destroy": destroy, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "allocated": lambda _handle: True, + "destroy": destroy, + } + ), descriptor_ownership="owned", to_numpy_policy="unsupported", ) @@ -322,12 +268,13 @@ def test_owned_handle_finalizer_calls_destroy_once(): handle = AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "destroy": lambda _handle: calls.append("destroy"), - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "allocated": lambda _handle: True, + "destroy": lambda _handle: calls.append("destroy"), + } + ), descriptor_ownership="owned", to_numpy_policy="unsupported", ) @@ -343,11 +290,12 @@ def test_owned_handle_construction_requires_generated_destroy_operation(): AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "allocated": lambda _handle: True, + } + ), descriptor_ownership="owned", to_numpy_policy="unsupported", ) @@ -359,13 +307,14 @@ def test_borrowed_handle_close_and_finalizer_do_not_destroy_native_storage(): handle = PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "destroy": lambda _handle: calls.append("destroy"), - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + "destroy": lambda _handle: calls.append("destroy"), + } + ), to_numpy_policy="unsupported", ) diff --git a/tests/fortran/memory_management/runtime/test_native_array_actual_handoff.py b/tests/fortran/memory_management/runtime/test_native_array_actual_handoff.py deleted file mode 100644 index 8bf550ed7..000000000 --- a/tests/fortran/memory_management/runtime/test_native_array_actual_handoff.py +++ /dev/null @@ -1,598 +0,0 @@ -"""Runtime handoff validation for arrays and native storage handles.""" - -import numpy as np -import pytest -from prik.runtime.handles import ( - AllocatableArray, - PointerArray, - _NativeArrayHandoff, - _native_array_actual_argument_for_binding_positional, - _native_array_actual_for_binding, -) -from tests.fortran._support.native_array_handles import _handoff - - -def test_array_actual_hook_rejects_generated_none_handoff(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(203), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "array_actual": lambda _handle: None, - }, - to_numpy_policy="unsupported", - ) - - with pytest.raises(TypeError, match="array_actual operation must return a native handoff object"): - _native_array_actual_for_binding(handle) - - -def test_native_array_handoff_requires_non_null_pointer_address(): - assert _NativeArrayHandoff(1).address == 1 - with pytest.raises(TypeError, match="address must be an integer"): - _NativeArrayHandoff(True) - with pytest.raises(TypeError, match="address must be an integer"): - _NativeArrayHandoff("1") - with pytest.raises(ValueError, match="non-null positive pointer"): - _NativeArrayHandoff(0) - with pytest.raises(ValueError, match="non-null positive pointer"): - _NativeArrayHandoff(-1) - - -def test_array_actual_hook_rejects_generated_untyped_handoff_object(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(245), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "array_actual": lambda _handle: object(), - }, - to_numpy_policy="unsupported", - ) - - with pytest.raises(TypeError, match="received object"): - _native_array_actual_for_binding(handle) - - -def test_array_actual_hook_validates_expected_dtype_and_rank(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=2, - ops={ - "descriptor": lambda _handle: _handoff(204), - "shape": lambda _handle: (2, 3), - "allocated": lambda _handle: True, - "array_actual": lambda _handle: _handoff(205), - }, - to_numpy_policy="unsupported", - ) - - with pytest.raises(ValueError, match="expected rank 1"): - handle._array_actual_for_binding(expected_rank=1) - with pytest.raises(TypeError, match="expected dtype"): - handle._array_actual_for_binding(expected_dtype=np.int32) - - -def test_array_actual_hook_validates_expected_shape_layout_and_writeability(): - actual = _handoff(206) - calls = [] - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=2, - ops={ - "descriptor": lambda _handle: _handoff(207), - "shape": lambda _handle: (0, 3), - "allocated": lambda _handle: True, - "layout": lambda _handle: "F", - "writeable": lambda _handle: True, - "array_actual": lambda _handle: calls.append("array_actual") or actual, - }, - to_numpy_policy="unsupported", - ) - - assert ( - handle._array_actual_for_binding( - expected_dtype=np.float64, - expected_rank=2, - expected_shape=(0, 3), - expected_layout="F", - require_writeable=True, - ) - is actual - ) - assert calls == ["array_actual"] - - with pytest.raises(ValueError, match=r"expected shape .* axis 0"): - handle._array_actual_for_binding(expected_shape=(1, 3)) - with pytest.raises(ValueError, match="expected shape rank 1"): - handle._array_actual_for_binding(expected_shape=(0,)) - assert handle._array_actual_for_binding(expected_layout="f") is actual - with pytest.raises(ValueError, match="expected layout 'C'"): - handle._array_actual_for_binding(expected_layout="C") - - read_only = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(208), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "writeable": lambda _handle: False, - "array_actual": lambda _handle: _handoff(209), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(TypeError, match="must be writeable"): - read_only._array_actual_for_binding(require_writeable=True) - - missing_writeable = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(210), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "array_actual": lambda _handle: _handoff(211), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(NotImplementedError, match="operation 'writeable' is not available"): - missing_writeable._array_actual_for_binding(require_writeable=True) - - -def test_array_actual_hook_rejects_unsupported_layout_names_before_handoff(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(212), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "layout": lambda _handle: pytest.fail("unsupported expected layout must not call generated layout op"), - "array_actual": lambda _handle: pytest.fail("unsupported expected layout must block native handoff"), - }, - to_numpy_policy="unsupported", - ) - - with pytest.raises(ValueError, match="unsupported expected NumPy array layout 'A'"): - handle._array_actual_for_binding(expected_layout="A") - - invalid_actual = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(213), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "layout": lambda _handle: "K", - "array_actual": lambda _handle: pytest.fail("unsupported actual layout must block native handoff"), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(ValueError, match="layout operation returned unsupported layout 'K'"): - invalid_actual._array_actual_for_binding(expected_layout="F") - - -def test_array_actual_hook_validates_native_byte_order_and_alignment_ops(): - actual = _handoff(214) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(215), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "native_byte_order": lambda _handle: True, - "aligned": lambda _handle: True, - "array_actual": lambda _handle: actual, - }, - to_numpy_policy="unsupported", - ) - - assert ( - handle._array_actual_for_binding( - require_native_byte_order=True, - require_aligned=True, - ) - is actual - ) - - byte_swapped = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(216), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "native_byte_order": lambda _handle: False, - "array_actual": lambda _handle: _handoff(217), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(TypeError, match="native byte order"): - byte_swapped._array_actual_for_binding(require_native_byte_order=True) - - unaligned = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(218), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "aligned": lambda _handle: False, - "array_actual": lambda _handle: _handoff(219), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(TypeError, match="aligned"): - unaligned._array_actual_for_binding(require_aligned=True) - - -def test_array_actual_binding_helper_accepts_ndarray_path_with_shared_validation(): - values = np.asfortranarray(np.zeros((2, 3), dtype=np.float64)) - - assert ( - _native_array_actual_for_binding( - values, - expected_dtype=np.float64, - expected_rank=2, - expected_shape=(2, 3), - expected_layout="F", - require_writeable=True, - ) - is values - ) - - with pytest.raises(TypeError, match="expected rank 1"): - _native_array_actual_for_binding(values, expected_rank=1) - with pytest.raises(TypeError, match="expected dtype"): - _native_array_actual_for_binding(values, expected_dtype=np.float32) - with pytest.raises(TypeError, match=r"incompatible shape at axis 0"): - _native_array_actual_for_binding(values, expected_shape=(1, 3)) - with pytest.raises(TypeError, match=r"expected ordering \(C\)"): - _native_array_actual_for_binding(values, expected_layout="C") - - read_only = values.copy() - read_only.setflags(write=False) - with pytest.raises(TypeError, match="writeable"): - _native_array_actual_for_binding(read_only, require_writeable=True) - - -def test_array_actual_binding_helper_rejects_byte_swapped_and_unaligned_ndarrays(): - swapped_dtype = np.dtype(np.float64).newbyteorder("S") - swapped = np.array([1.0, 2.0], dtype=swapped_dtype) - with pytest.raises(TypeError, match="native byte order"): - _native_array_actual_for_binding(swapped, require_native_byte_order=True) - - storage = np.zeros(8 * 2 + 1, dtype=np.uint8) - unaligned = storage[1:].view(np.float64) - assert not unaligned.flags.aligned - with pytest.raises(TypeError, match="aligned"): - _native_array_actual_for_binding(unaligned, require_aligned=True) - - -def test_array_actual_binding_helper_routes_handles_without_numpy_conversion(): - alloc_actual = _handoff(220) - pointer_actual = _handoff(221) - calls = [] - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(222), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "layout": lambda _handle: "F", - "writeable": lambda _handle: True, - "native_byte_order": lambda _handle: True, - "aligned": lambda _handle: True, - "to_numpy": lambda _handle: pytest.fail("array-actual handoff must not call to_numpy"), - "array_actual": lambda _handle: calls.append("alloc") or alloc_actual, - }, - ) - pointer_handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(223), - "shape": lambda _handle: (2,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "layout": lambda _handle: "F", - "writeable": lambda _handle: True, - "native_byte_order": lambda _handle: True, - "aligned": lambda _handle: True, - "to_numpy": lambda _handle: pytest.fail("array-actual handoff must not call to_numpy"), - "array_actual": lambda _handle: calls.append("pointer") or pointer_actual, - }, - ) - - assert ( - _native_array_actual_for_binding( - alloc_handle, - expected_dtype=np.float64, - expected_rank=1, - expected_shape=(2,), - expected_layout="F", - require_writeable=True, - require_native_byte_order=True, - require_aligned=True, - ) - is alloc_actual - ) - assert ( - _native_array_actual_for_binding( - pointer_handle, - expected_dtype=np.float64, - expected_rank=1, - expected_shape=(2,), - expected_layout="F", - require_writeable=True, - require_native_byte_order=True, - require_aligned=True, - ) - is pointer_actual - ) - assert calls == ["alloc", "pointer"] - - -def test_array_actual_binding_helper_preserves_zero_length_array_actuals(): - values = np.zeros((0,), dtype=np.float64) - assert _native_array_actual_for_binding(values, expected_dtype=np.float64, expected_shape=(0,)) is values - - alloc_actual = _handoff(224) - pointer_actual = _handoff(225) - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(226), - "shape": lambda _handle: (0,), - "allocated": lambda _handle: True, - "array_actual": lambda _handle: alloc_actual, - }, - to_numpy_policy="unsupported", - ) - pointer_handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(227), - "shape": lambda _handle: (0,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "array_actual": lambda _handle: pointer_actual, - }, - to_numpy_policy="unsupported", - ) - - assert _native_array_actual_for_binding(alloc_handle, expected_shape=(0,)) is alloc_actual - assert _native_array_actual_for_binding(pointer_handle, expected_shape=(0,)) is pointer_actual - - -def test_array_actual_argument_abi_packer_uses_ndarray_data_pointer_and_shape_fields(): - values = np.asfortranarray(np.zeros((2, 3), dtype=np.float64)) - - assert _native_array_actual_argument_for_binding_positional( - values, - np.float64, - 2, - (2, 3), - "F", - True, - True, - True, - True, - True, - True, - ) == (values.ctypes.data, 2, values.dtype.itemsize, 2, 3, 1, 2, 1, 1) - - -def test_array_actual_argument_abi_packer_flattens_contiguous_storage_shape(): - values = np.asfortranarray(np.arange(6, dtype=np.float64).reshape((2, 3), order="F")) - - assert _native_array_actual_argument_for_binding_positional( - values, - expected_dtype=np.float64, - expected_rank=-1, - expected_shape=None, - require_native_byte_order=True, - require_aligned=True, - require_contiguous=True, - flatten_storage=True, - ) == (values.ctypes.data, values.size) - - -def test_array_actual_argument_abi_packer_flattens_final_edge_after_checked_prefix(): - values = np.asfortranarray(np.arange(24, dtype=np.float64).reshape((2, 3, 4), order="F")) - - assert _native_array_actual_argument_for_binding_positional( - values, - expected_dtype=np.float64, - expected_rank=2, - expected_shape=(2, None), - expected_layout="F", - require_native_byte_order=True, - require_aligned=True, - require_contiguous=True, - flatten_storage=True, - flat_axis=1, - ) == (values.ctypes.data, 2, 12) - - -def test_array_actual_argument_abi_packer_flattens_leading_edge_before_checked_suffix(): - values = np.arange(24, dtype=np.float64).reshape((2, 3, 4), order="C") - - assert _native_array_actual_argument_for_binding_positional( - values, - expected_dtype=np.float64, - expected_rank=2, - expected_shape=(None, 4), - expected_layout="C", - require_native_byte_order=True, - require_aligned=True, - require_contiguous=True, - flatten_storage=True, - flat_axis=0, - ) == (values.ctypes.data, 6, 4) - - -@pytest.mark.parametrize( - ("values", "expected_rank", "expected_shape", "flat_axis", "message"), - [ - ( - np.array(1.0, dtype=np.float64), - 1, - (None,), - 0, - "expects NumPy array rank 1 through 15", - ), - ( - np.ones((2,), dtype=np.float64), - 2, - (2, None), - 1, - "expects NumPy array rank at least 2", - ), - ( - np.ones((2, 3, 4), dtype=np.float64), - 3, - (2, None, 4), - 1, - "axis must be the first or final contract dimension", - ), - ( - np.ones((2, 3, 4), dtype=np.float64), - 2, - (3, None), - 1, - "incompatible shape at axis 0", - ), - ( - np.ones((2, 3, 4), dtype=np.float64), - 2, - (None, 3), - 0, - "incompatible shape at axis 2", - ), - ], -) -def test_array_actual_argument_abi_packer_rejects_invalid_flat_shapes( - values, - expected_rank, - expected_shape, - flat_axis, - message, -): - with pytest.raises((TypeError, ValueError), match=message): - _native_array_actual_argument_for_binding_positional( - values, - expected_dtype=np.float64, - expected_rank=expected_rank, - expected_shape=expected_shape, - require_native_byte_order=True, - require_aligned=True, - require_contiguous=True, - flatten_storage=True, - flat_axis=flat_axis, - ) - - -def test_array_actual_argument_abi_packer_flattens_native_handle_shape(): - actual = _handoff(252) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=2, - ops={ - "descriptor": lambda _handle: _handoff(253), - "shape": lambda _handle: (2, 3), - "allocated": lambda _handle: True, - "layout": lambda _handle: "F", - "writeable": lambda _handle: True, - "native_byte_order": lambda _handle: True, - "aligned": lambda _handle: True, - "to_numpy": lambda _handle: pytest.fail("flat array-actual ABI packing must not call to_numpy"), - "array_actual": lambda _handle: actual, - }, - ) - - assert _native_array_actual_argument_for_binding_positional( - handle, - expected_dtype=np.float64, - expected_rank=-1, - expected_shape=None, - require_native_byte_order=True, - require_aligned=True, - require_contiguous=True, - flatten_storage=True, - ) == (actual.address, 6) - - -def test_array_actual_argument_abi_packer_rejects_absent_handles_before_generated_handoff(): - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(250), - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - "to_numpy": lambda _handle: pytest.fail("array-actual ABI packing must not call to_numpy"), - "array_actual": lambda _handle: pytest.fail("unallocated handle must block native array handoff"), - }, - ) - pointer_handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(251), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - "to_numpy": lambda _handle: pytest.fail("array-actual ABI packing must not call to_numpy"), - "array_actual": lambda _handle: pytest.fail("unassociated handle must block native array handoff"), - }, - ) - - with pytest.raises(ValueError, match="unallocated"): - _native_array_actual_argument_for_binding_positional(alloc_handle) - with pytest.raises(ValueError, match="unassociated"): - _native_array_actual_argument_for_binding_positional(pointer_handle) - - -def test_array_actual_binding_helper_rejects_absent_handles_none_and_non_arrays(): - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(230), - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - "to_numpy": lambda _handle: pytest.fail("array-actual handoff must not call to_numpy"), - "array_actual": lambda _handle: _handoff(231), - }, - ) - pointer_handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(232), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - "to_numpy": lambda _handle: pytest.fail("array-actual handoff must not call to_numpy"), - "array_actual": lambda _handle: _handoff(233), - }, - ) - - with pytest.raises(ValueError, match="unallocated"): - _native_array_actual_for_binding(alloc_handle) - with pytest.raises(ValueError, match="unassociated"): - _native_array_actual_for_binding(pointer_handle) - with pytest.raises(TypeError, match="received None"): - _native_array_actual_for_binding(None) - with pytest.raises(TypeError, match="received list"): - _native_array_actual_for_binding([1.0, 2.0]) diff --git a/tests/fortran/modules/codegen/test_module_array_view_lowering.py b/tests/fortran/modules/codegen/test_module_array_view_lowering.py new file mode 100644 index 000000000..841c41f7c --- /dev/null +++ b/tests/fortran/modules/codegen/test_module_array_view_lowering.py @@ -0,0 +1,195 @@ +"""Bridge lowering for the two fixed module-array address mechanisms.""" + +from __future__ import annotations + +from dataclasses import replace + +import pytest + +from prik.codegen.c.binding import CBindingGenerator +from prik.codegen.fortran.bridge import FortranBridgeGenerator +from prik.parsers.fortran.parser import parse_fortran_project +from prik.pipeline.build import _apply_source_python_exports, _merge_wrapper_modules +from prik.pipeline.wrapper import WrapperGenerator +from prik.planning import WrapperPlanner +from prik.policy.completion import complete_semantic_policies +from prik.printers.fortran import FortranSourcePrinter +from prik.semantics.fortran2ir import fortran_project_to_semantic_modules +from tests.fortran._support.ownership_policy import parse_pyi_text + + +MODULE_ARRAY_SOURCE = """ +module array_state + use iso_fortran_env, only: int32, real64 + implicit none + real(real64) :: plain(2, 3) + integer(int32) :: counts(3) + character(len=5) :: labels(2) + real(real64), target :: addressable(4) +end module array_state +""" + +DESCRIPTOR_CHARACTER_SOURCE = """ +module char_state + implicit none + character(len=5), allocatable :: fixed_alloc(:) + character(len=5), pointer :: fixed_ptr(:) => null() + character(len=:), allocatable :: deferred_alloc(:) + real(8), allocatable :: numbers(:) +end module char_state +""" + + +def _plan(): + parsed = parse_fortran_project({"array_state.f90": MODULE_ARRAY_SOURCE}) + modules = fortran_project_to_semantic_modules(parsed) + _apply_source_python_exports(modules) + module = _merge_wrapper_modules(modules, name="array_state") + complete_semantic_policies(module) + return WrapperPlanner().build(module) + + +def _bridge_module(): + return FortranBridgeGenerator().visit(_plan()) + + +def _lowered_getters(): + plan = _plan() + bridge = FortranBridgeGenerator() + bridge.visit(plan) + printer = FortranSourcePrinter() + return { + variable.binding.python_names[0]: printer.visit(bridge.visit(variable)[0]) + for namespace in plan.namespaces + for variable in namespace.variables + } + + +def test_addressable_module_array_takes_its_address_directly(): + """A `target` declaration lets `c_loc` name the array, so nothing else is emitted.""" + getter = _lowered_getters()["addressable"] + + assert "c_loc(native_addressable)" in getter + assert "capture_array_address" not in getter + + +@pytest.mark.parametrize("python_name", ["plain", "counts", "labels"]) +def test_ordinary_module_array_captures_its_address_in_c(python_name): + """Without `target`, the address is taken on the C side, never by `c_loc`. + + `c_loc` requires the variable it names to be a target, so an ordinary + declaration has no Fortran route to its own address. The getter hands the + whole array to a `bind(C)` procedure instead: an assumed-type assumed-size + dummy is passed as the bare base address, so C receives where the module + variable lives. The bridge forms no pointer and claims no target, and one + interface covers every element type including character. + """ + getter = _lowered_getters()[python_name] + + assert f"prik_capture_address(native_{python_name})" in getter + assert "c_loc" not in getter + assert "target" not in getter + + +def test_captured_address_declares_one_typeless_c_interface(): + """One assumed-type interface serves every captured element type.""" + module = _bridge_module() + + interfaces = [procedure for interface in module.interfaces for procedure in interface.procedures] + captures = [procedure for procedure in interfaces if procedure.name == "prik_capture_address"] + assert len(captures) == 1 + assert captures[0].bind_name == "prik_capture_address" + assert captures[0].parameters[0].type_name == "type(*)" + assert captures[0].parameters[0].attributes == ("dimension(*)",) + + +def test_capture_helper_is_declared_only_where_an_array_needs_it(): + """A module whose arrays are all addressable declares no capture interface.""" + module = parse_pyi_text( + "addressable: Annotated[Float64[4], Aliased]\n", + module_name="array_state", + ) + complete_semantic_policies(module) + bridge = FortranBridgeGenerator() + emitted = bridge.visit(WrapperPlanner().build(module)) + + names = [procedure.name for interface in emitted.interfaces for procedure in interface.procedures] + assert "prik_capture_address" not in names + + +def test_binding_opts_into_the_bundled_capture_primitive(): + """The C side selects the runtime definition of the symbol the bridge calls. + + The helper lives in the bundled support header rather than in emitted code, + because it is a fixed ABI primitive rather than something a plan describes. + It needs external linkage for the bridge to call it, so the binding opts in + once per extension and the header defines it in that translation unit alone. + """ + binding = CBindingGenerator().binding_module(_plan()) + + assert any(define.name == "PRIK_BINDING_CAPTURE_ADDRESS" for define in binding.defines) + assert not any(function.name == "prik_capture_address" for function in binding.functions) + + +def test_binding_omits_the_capture_primitive_when_no_array_needs_it(): + """An extension whose arrays are all addressable pulls in no capture symbol.""" + module = parse_pyi_text( + "addressable: Annotated[Float64[4], Aliased]\n", + module_name="array_state", + ) + complete_semantic_policies(module) + binding = CBindingGenerator().binding_module(WrapperPlanner().build(module)) + + assert not any(define.name == "PRIK_BINDING_CAPTURE_ADDRESS" for define in binding.defines) + + +def _undecided_plan(): + """Return a module-array plan whose address mechanism policy never selected.""" + module = parse_pyi_text("plain: Float64[3]\n", module_name="array_state") + complete_semantic_policies(module) + plan = WrapperPlanner().build(module) + variable = plan.namespaces[0].variables[0] + return plan, variable, replace(variable, array_address=None) + + +def test_module_array_view_plan_rejects_a_missing_address_mechanism(): + """The plan boundary reports the gap rather than letting lowering guess.""" + plan, _variable, undecided = _undecided_plan() + namespace = plan.namespaces[0] + namespace.variables = (undecided,) + + diagnostics = WrapperGenerator()._plan_diagnostics(plan) + + assert any(diagnostic.code == "missing-module-array-address-mechanism" for diagnostic in diagnostics) + + +def test_module_array_view_lowering_requires_a_completed_address_mechanism(): + """Lowering refuses to invent an address route policy did not select.""" + plan, _variable, undecided = _undecided_plan() + bridge = FortranBridgeGenerator() + bridge.visit(plan) + + with pytest.raises(ValueError, match="no completed address mechanism"): + bridge.visit(undecided) + + +def _character_bridge_source(): + parsed = parse_fortran_project({"char_state.f90": DESCRIPTOR_CHARACTER_SOURCE}) + modules = fortran_project_to_semantic_modules(parsed) + _apply_source_python_exports(modules) + module = _merge_wrapper_modules(modules, name="char_state") + complete_semantic_policies(module) + plan = WrapperPlanner().build(module) + return FortranSourcePrinter().visit(FortranBridgeGenerator().visit(plan)) + + +def test_fixed_character_callbacks_use_an_ordinary_descriptor_projection(): + """The callback declaration matches the descriptor attribute policy selected upstream.""" + source = _character_bridge_source() + + assert source.count("character(kind=c_char, len=*), dimension(:), intent(inout) :: value") == 2 + assert "character(kind=c_char, len=:), allocatable, dimension(:), intent(inout) :: value" in source + assert "real(c_double), allocatable, dimension(:), intent(inout) :: value" in source + assert "if (allocated(native_fixed_alloc)) then" in source + assert "if (associated(native_fixed_ptr)) then" in source + assert "call callback(native_deferred_alloc, context)" in source diff --git a/tests/fortran/modules/end_to_end/fixtures/native/fmodule_array_forms_f90.f90 b/tests/fortran/modules/end_to_end/fixtures/native/fmodule_array_forms_f90.f90 new file mode 100644 index 000000000..445fb52a2 --- /dev/null +++ b/tests/fortran/modules/end_to_end/fixtures/native/fmodule_array_forms_f90.f90 @@ -0,0 +1,97 @@ +module fmodule_array_forms_f90 + use iso_c_binding, only: c_bool + use iso_fortran_env, only: int32, real64 + implicit none + + type :: sample + real(real64) :: grid(2, 3) + integer(int32) :: n + end type sample + + ! Fixed shape: the declaration holds the shape; only the address is unknown. + real(real64) :: fixed_plain(4) = [1.0d0, 2.0d0, 3.0d0, 4.0d0] + real(real64), target :: fixed_target(4) = [5.0d0, 6.0d0, 7.0d0, 8.0d0] + real(real64) :: fixed_matrix(2, 3) + real(real64) :: fixed_shifted(5:8) = [9.0d0, 10.0d0, 11.0d0, 12.0d0] + integer(int32) :: fixed_counts(3) = [7, 8, 9] + logical(c_bool) :: fixed_flags(3) = [.true., .false., .true.] + + ! Allocatable: bounds, strides and element length are runtime facts. + real(real64), allocatable :: alloc_plain(:) + real(real64), allocatable, target :: alloc_target(:) + real(real64), allocatable :: alloc_matrix(:, :) + real(real64), allocatable :: alloc_shifted(:) + + ! Pointer: association is runtime state. + real(real64), pointer :: ptr_link(:) => null() + + ! Character, in every storage form its element length can take. + character(len=5) :: char_fixed(2) = ['alpha', 'bravo'] + character(len=5), target :: char_target(2) = ['gamma', 'delta'] + character(len=5), allocatable :: char_alloc(:) + character(len=:), allocatable :: char_deferred(:) + + ! Derived objects whose array field is reached through the owner. + type(sample) :: obj_plain + type(sample), target :: obj_target + +contains + + subroutine setup() + fixed_matrix = reshape([1.0d0, 2.0d0, 3.0d0, 4.0d0, 5.0d0, 6.0d0], [2, 3]) + allocate(alloc_plain(4)); alloc_plain = [1.0d0, 2.0d0, 3.0d0, 4.0d0] + allocate(alloc_target(3)); alloc_target = [9.0d0, 8.0d0, 7.0d0] + allocate(alloc_matrix(2, 3)); alloc_matrix = 2.0d0 + allocate(alloc_shifted(5:8)); alloc_shifted = [1.0d0, 2.0d0, 3.0d0, 4.0d0] + allocate(character(len=5) :: char_alloc(2)); char_alloc = ['epsil', 'zetaa'] + allocate(character(len=6) :: char_deferred(2)); char_deferred = ['etaaaa', 'thetaa'] + obj_plain%grid = 1.0d0; obj_plain%n = 3 + obj_target%grid = 2.0d0; obj_target%n = 4 + ptr_link => alloc_target + end subroutine setup + + ! One ordinary array dummy every numeric form above can be passed to. + function total(values) result(sum_of) + real(real64), intent(in) :: values(:) + real(real64) :: sum_of + sum_of = sum(values) + end function total + + function total_2d(values) result(sum_of) + real(real64), intent(in) :: values(:, :) + real(real64) :: sum_of + sum_of = sum(values) + end function total_2d + + function count_set(values) result(how_many) + logical(c_bool), intent(in) :: values(:) + integer(int32) :: how_many + how_many = count(values) + end function count_set + + function total_counts(values) result(sum_of) + integer(int32), intent(in) :: values(:) + integer(int32) :: sum_of + sum_of = sum(values) + end function total_counts + + function first_word(values) result(word) + character(len=5), intent(in) :: values(:) + character(len=5) :: word + word = values(1) + end function first_word + + ! An allocatable dummy adopts the descriptor's bounds, unlike the ones above. + function lower_bound_of(values) result(bound) + real(real64), allocatable, intent(in) :: values(:) + integer(int32) :: bound + bound = lbound(values, 1) + end function lower_bound_of + + subroutine bump_all() + fixed_plain(1) = fixed_plain(1) + 100.0d0 + alloc_plain(1) = alloc_plain(1) + 100.0d0 + obj_plain%grid(1, 1) = obj_plain%grid(1, 1) + 100.0d0 + end subroutine bump_all + +end module fmodule_array_forms_f90 diff --git a/tests/fortran/modules/end_to_end/test_logical_array_views.py b/tests/fortran/modules/end_to_end/test_logical_array_views.py new file mode 100644 index 000000000..ac4eda978 --- /dev/null +++ b/tests/fortran/modules/end_to_end/test_logical_array_views.py @@ -0,0 +1,141 @@ +"""How a Fortran logical array reaches Python, and at what width. + +A logical has no fixed representation in Fortran, and some compilers default to +one their own C compiler cannot read. prik requests the option that selects the +interoperable form, so `.true.` is the same byte everywhere and a one-byte +logical is exactly what `numpy.bool_` describes. Kinds wider than a byte have no +NumPy Boolean to be, so they report the integer of matching width instead. +""" + +import numpy as np +import pytest + +from tests.fortran._support.wrapper_build import _build_text_and_import + +pytestmark = pytest.mark.fortran_end_to_end + +LOGICAL_VIEW_SOURCE = """ +module flogical_view_f90 + use iso_c_binding, only: c_bool + implicit none + + logical(c_bool) :: narrow(4) + logical :: wide(4) + logical(c_bool), allocatable :: narrow_alloc(:) + logical, allocatable :: wide_alloc(:) + logical, target :: wide_store(4) + logical, pointer :: wide_pointer(:) => null() + +contains + + subroutine setup() + narrow = [.true., .false., .true., .false.] + wide = [.true., .false., .true., .false.] + if (.not. allocated(narrow_alloc)) allocate(narrow_alloc(4)) + narrow_alloc = [.true., .true., .false., .false.] + if (.not. allocated(wide_alloc)) allocate(wide_alloc(4)) + wide_alloc = [.true., .false., .true., .false.] + wide_store = [.true., .false., .true., .false.] + wide_pointer => wide_store + end subroutine setup + + subroutine negate() + narrow = .not. narrow + wide = .not. wide + end subroutine negate + + function count_narrow() result(total) + integer :: total + total = count(narrow) + end function count_narrow + + function count_wide() result(total) + integer :: total + total = count(wide) + end function count_wide + + function count_narrow_actual(values) result(total) + logical(c_bool), intent(in) :: values(:) + integer :: total + total = count(values) + end function count_narrow_actual + + function count_wide_actual(values) result(total) + logical, intent(in) :: values(:) + integer :: total + total = count(values) + end function count_wide_actual +end module flogical_view_f90 +""" + + +@pytest.fixture(scope="module") +def logical_view(tmp_path_factory): + module = _build_text_and_import( + LOGICAL_VIEW_SOURCE, + "flogical_view_f90.f90", + tmp_path_factory.mktemp("logical_view"), + { + "bind_c_flogical_view_f90_wrapper.f90", + "flogical_view_f90_wrapper.c", + "flogical_view_f90_wrapper.h", + }, + ) + module.setup() + return module + + +def test_a_one_byte_logical_is_a_numpy_boolean(logical_view): + """`logical(c_bool)` holds zero or one in one byte, which is `numpy.bool_`.""" + assert logical_view.narrow.dtype == np.dtype(np.bool_) + assert logical_view.narrow_alloc.to_numpy().dtype == np.dtype(np.bool_) + assert logical_view.narrow.tolist() == [True, False, True, False] + + +def test_a_wider_logical_reports_the_width_its_elements_occupy(logical_view): + """NumPy has no Boolean larger than a byte, so the width is stated instead.""" + wide = logical_view.wide + + assert wide.dtype == np.dtype(np.int32) + assert wide.astype(bool).tolist() == [True, False, True, False] + assert logical_view.wide_alloc.to_numpy().dtype == np.dtype(np.int32) + + +def test_logical_allocatable_handles_reach_matching_ordinary_dummies(logical_view): + assert logical_view.count_narrow_actual(logical_view.narrow_alloc) == np.int32(2) + assert logical_view.count_wide_actual(logical_view.wide_alloc) == np.int32(2) + + +def test_wide_logical_pointer_handles_reach_matching_ordinary_dummies(logical_view): + assert logical_view.wide_pointer.dtype == np.dtype(np.int32) + assert logical_view.wide_pointer.shape == (4,) + assert logical_view.count_wide_actual(logical_view.wide_pointer) == np.int32(2) + + +def test_a_held_view_keeps_agreeing_with_fortran_across_native_writes(logical_view): + """The view aliases the storage, and both sides read the same bytes. + + The interoperable representation is what makes this hold on every compiler: + without it one of them writes all bits set for `.true.`, which C and NumPy + would read as true where Fortran's own complement of it means false. + """ + narrow, wide = logical_view.narrow, logical_view.wide + try: + logical_view.negate() + + assert narrow.tolist() == [False, True, False, True] + assert int(narrow.sum()) == logical_view.count_narrow() + assert wide.astype(bool).tolist() == [False, True, False, True] + assert int(wide.astype(bool).sum()) == logical_view.count_wide() + finally: + logical_view.negate() + + +def test_reading_a_logical_view_does_not_disturb_native_storage(logical_view): + """Reading is a borrow: nothing is rewritten on the way out.""" + before = logical_view.count_narrow() + + for _ in range(3): + logical_view.narrow # noqa: B018 - the read itself is what is under test + + assert logical_view.count_narrow() == before diff --git a/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py b/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py new file mode 100644 index 000000000..8f5b150d0 --- /dev/null +++ b/tests/fortran/modules/end_to_end/test_module_array_storage_forms.py @@ -0,0 +1,173 @@ +"""Every storage form a module array can take, exercised through one module. + +The forms differ in what the declaration already fixes and what only exists at +runtime, and prik reaches them by three different mechanisms because of that. +This module holds them side by side so the differences are visible in one place: +what each is exposed as, whether it stays live, and what happens when it is +handed back to an ordinary Fortran array dummy. +""" + +from pathlib import Path + +import numpy as np +import pytest + +from tests.fortran._support.wrapper_build import _build_and_import + +FIXTURES = Path(__file__).parent / "fixtures" +ARRAY_FORMS_F90_SOURCE = FIXTURES / "native" / "fmodule_array_forms_f90.f90" +pytestmark = pytest.mark.fortran_end_to_end + + +@pytest.fixture(scope="module") +def array_forms(tmp_path_factory): + module = _build_and_import( + ARRAY_FORMS_F90_SOURCE, + tmp_path_factory.mktemp("array_forms"), + { + "bind_c_fmodule_array_forms_f90_wrapper.f90", + "fmodule_array_forms_f90_wrapper.c", + "fmodule_array_forms_f90_wrapper.h", + }, + ) + module.setup() + return module + + +# A fixed shape is entirely in the declaration, so these are plain views; an +# allocatable or pointer carries runtime state, so those are handles. +@pytest.mark.parametrize( + ("name", "exposed_as", "dtype"), + [ + ("fixed_plain", np.ndarray, "float64"), + ("fixed_target", np.ndarray, "float64"), + ("fixed_matrix", np.ndarray, "float64"), + ("fixed_shifted", np.ndarray, "float64"), + ("fixed_counts", np.ndarray, "int32"), + ("fixed_flags", np.ndarray, "bool"), + ("char_fixed", np.ndarray, "S5"), + ("char_target", np.ndarray, "S5"), + ], +) +def test_fixed_shape_module_arrays_are_plain_views(array_forms, name, exposed_as, dtype): + """A declared shape needs no handle: the value is the storage itself.""" + value = getattr(array_forms, name) + + assert isinstance(value, exposed_as) + assert value.dtype == np.dtype(dtype) + + +@pytest.mark.parametrize( + ("name", "dtype"), + [ + ("alloc_plain", "float64"), + ("alloc_target", "float64"), + ("alloc_matrix", "float64"), + ("alloc_shifted", "float64"), + ("char_alloc", "S5"), + ("char_deferred", "S6"), + ], +) +def test_allocatable_module_arrays_are_handles(array_forms, name, dtype): + """Allocation state is not in the declaration, so these carry it explicitly.""" + handle = getattr(array_forms, name) + + assert handle.allocated is True + assert handle.to_numpy().dtype == np.dtype(dtype) + + +def test_a_bare_pointer_is_a_handle_that_declines_to_hand_out_a_view(array_forms): + """A pointer says what it is associated with, not that a view is safe. + + Contiguity and target lifetime are not in a plain `pointer` declaration, so + no view can be justified from it alone; that needs an explicit + `PointerPolicy`. The association state is knowable and is reported. + """ + handle = array_forms.ptr_link + + assert handle.associated is True + assert handle.shape == (3,) + with pytest.raises(NotImplementedError, match="unsupported by completed policy"): + handle.to_numpy() + + +def test_derived_array_fields_are_views_through_either_owner(array_forms): + """An array component is reached through its owner, addressable or not.""" + assert isinstance(array_forms.obj_plain.grid, np.ndarray) + assert isinstance(array_forms.obj_target.grid, np.ndarray) + assert array_forms.obj_plain.grid.shape == (2, 3) + + +@pytest.mark.parametrize( + "name", + ["fixed_plain", "fixed_target", "fixed_shifted", "obj_plain", "obj_target"], +) +def test_views_stay_live_across_native_writes(array_forms, name): + """Every borrowed view names the storage native code writes, not a copy.""" + + def current(): + owner = getattr(array_forms, name) + return owner.grid if name.startswith("obj") else owner + + view = current() + before = float(view.flat[0]) + try: + view.flat[0] = before + 1.0 + assert float(current().flat[0]) == before + 1.0 + finally: + # The storage is shared with every other test in this module, so the + # write is undone rather than left for whatever runs next. + view.flat[0] = before + + +def test_every_numeric_form_reaches_one_ordinary_array_dummy(array_forms): + """`total(values(:))` accepts each form, however its storage is reached. + + A fixed array arrives as a view, an allocatable and a pointer as handles, + and a derived component through its owner. The dummy is an ordinary array + either way, so the conversion has to erase the difference. + """ + assert array_forms.total(array_forms.fixed_plain) == np.float64(10.0) + assert array_forms.total(array_forms.fixed_target) == np.float64(26.0) + assert array_forms.total(array_forms.fixed_shifted) == np.float64(42.0) + assert array_forms.total(array_forms.alloc_plain) == np.float64(10.0) + assert array_forms.total(array_forms.alloc_target) == np.float64(24.0) + assert array_forms.total_2d(array_forms.fixed_matrix) == np.float64(21.0) + assert array_forms.total_2d(array_forms.alloc_matrix) == np.float64(12.0) + assert array_forms.total_2d(array_forms.obj_plain.grid) == np.float64(6.0) + assert array_forms.total_counts(array_forms.fixed_counts) == np.int32(24) + assert array_forms.count_set(array_forms.fixed_flags) == np.int32(2) + assert array_forms.fixed_flags.tolist() == [True, False, True] + assert array_forms.first_word(array_forms.char_fixed) == "alpha" + + +def test_only_an_allocatable_dummy_carries_the_declared_lower_bound(array_forms): + """The two dummy forms disagree about bounds, and both are right. + + An ordinary array dummy declares its own bounds, so a declared lower bound + is discarded by the language and cannot be observed. An allocatable dummy is + the actual's descriptor, so it adopts them -- which is why a module + allocatable has to report the bounds it really has. + """ + assert array_forms.lower_bound_of(array_forms.alloc_shifted) == np.int32(5) + assert array_forms.lower_bound_of(array_forms.alloc_plain) == np.int32(1) + + # `fixed_shifted` is declared (5:8) and sums the same as any other four + # elements: nothing downstream can tell where it started. + assert array_forms.total(array_forms.fixed_shifted) == np.float64(42.0) + + +def test_a_character_handle_reaches_a_character_dummy_like_any_other(array_forms): + """A handle stands in for an array actual whatever its element type. + + A character actual is matched on its declared width as well as its kind, so + a handle whose elements are a different length is refused: it describes + storage the dummy cannot accept. + """ + assert array_forms.first_word(array_forms.char_alloc) == "epsil" + assert array_forms.first_word(array_forms.char_alloc.to_numpy()) == "epsil" + assert array_forms.first_word(array_forms.char_fixed) == "alpha" + + # `char_deferred` holds six-character elements; the dummy declares five. + with pytest.raises(TypeError, match="does not match expected dtype"): + array_forms.first_word(array_forms.char_deferred) diff --git a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py index 8ac369201..04268fe8c 100644 --- a/tests/fortran/modules/end_to_end/test_module_variables_and_state.py +++ b/tests/fortran/modules/end_to_end/test_module_variables_and_state.py @@ -126,6 +126,92 @@ def test_scalar_module_variables_use_attributes_and_parameters_have_no_native_se assert second_module.black.r == np.int32(0) +PLAIN_MODULE_ARRAY_SOURCE = """ +module fplain_module_arrays_f90 + use iso_fortran_env, only: int32, real64 + implicit none + real(real64) :: grid(2, 3) + integer(int32) :: counts(3) = [7, 8, 9] + character(len=5) :: labels(2) = ['alpha', 'bravo'] + real(real64), target :: addressable(2) = [1.0d0, 2.0d0] +contains + subroutine bump() + grid(1, 1) = grid(1, 1) + 1.0d0 + end subroutine bump + + function read_grid(row, column) result(value) + integer(int32), intent(in) :: row, column + real(real64) :: value + value = grid(row, column) + end function read_grid + + function read_count(index) result(value) + integer(int32), intent(in) :: index + integer(int32) :: value + value = counts(index) + end function read_count + + function read_label(index) result(value) + integer(int32), intent(in) :: index + character(len=5) :: value + value = labels(index) + end function read_label +end module fplain_module_arrays_f90 +""" + + +def test_fixed_module_arrays_without_target_expose_the_same_live_view(tmp_path: Path): + """A fixed module array is borrowed live whether or not it declares `target`. + + `target` is what lets `c_loc` name the array, not what gives the array a + stable address, so withholding it changes the route to the base address and + nothing the caller can observe. The claim here is that both declarations + produce one live view over the real module storage: Fortran writes appear + without re-reading the attribute, Python writes are visible to Fortran, and + neither form accepts whole-array replacement. + """ + module = _build_text_and_import( + PLAIN_MODULE_ARRAY_SOURCE, + "fplain_module_arrays_f90.f90", + tmp_path, + { + "bind_c_fplain_module_arrays_f90_wrapper.f90", + "fplain_module_arrays_f90_wrapper.c", + "fplain_module_arrays_f90_wrapper.h", + }, + ) + + assert module.grid.shape == (2, 3) + assert module.grid.dtype == np.dtype(np.float64) + assert module.grid.flags["F_CONTIGUOUS"] is True + np.testing.assert_array_equal(module.counts, np.array([7, 8, 9], dtype=np.int32)) + np.testing.assert_array_equal(module.labels, np.array([b"alpha", b"bravo"], dtype="S5")) + + # A view handed out earlier still names the storage Fortran writes. + view = module.grid + view[0, 0] = np.float64(10.0) + module.bump() + assert view[0, 0] == np.float64(11.0) + assert module.grid[0, 0] == np.float64(11.0) + + # A Python write reaches the storage Fortran reads, for every element type. + module.grid[1, 2] = np.float64(4.5) + assert module.read_grid(np.int32(2), np.int32(3)) == np.float64(4.5) + module.counts[1] = np.int32(99) + assert module.read_count(np.int32(2)) == np.int32(99) + module.labels[0] = b"omega" + assert module.read_label(np.int32(1)) == "omega" + + # The addressable declaration borrows identically. + module.addressable[0] = np.float64(6.0) + assert module.addressable[0] == np.float64(6.0) + + # Neither form hands the whole variable back to be reassigned. + for name in ("grid", "counts", "addressable"): + with pytest.raises(AttributeError, match="read-only"): + setattr(module, name, np.zeros(2)) + + CHARACTER_MODULE_ARRAY_SOURCE = """ module fchar_module_arrays_f90 implicit none @@ -388,6 +474,12 @@ def test_assumed_length_character_parameter_array_reports_its_inferred_width(tmp fixed_ptr => store deferred_ptr => store end subroutine setup + + subroutine allocate_deferred() + if (associated(deferred_ptr)) nullify(deferred_ptr) + allocate(character(len=6) :: deferred_ptr(3)) + deferred_ptr = ['a ', 'bb ', 'ccc '] + end subroutine allocate_deferred end module fchar_declared_arrays_f90 """ @@ -429,3 +521,16 @@ def test_declared_length_character_module_arrays_compile_and_expose_their_width( assert module.fixed_ptr.shape == (2,) assert module.deferred_ptr.associated is True assert module.deferred_ptr.shape == (2,) + assert module.deferred_ptr.dtype == np.dtype("S4") + + module.deferred_ptr.nullify() + assert module.deferred_ptr.associated is False + assert module.deferred_ptr.shape is None + + module.allocate_deferred() + assert module.deferred_ptr.associated is True + assert module.deferred_ptr.shape == (3,) + assert module.deferred_ptr.dtype == np.dtype("S6") + module.deferred_ptr.deallocate() + assert module.deferred_ptr.associated is False + assert module.deferred_ptr.shape is None diff --git a/tests/fortran/modules/policy/test_module_variable_policy.py b/tests/fortran/modules/policy/test_module_variable_policy.py index a69b76a0b..9ae5616f3 100644 --- a/tests/fortran/modules/policy/test_module_variable_policy.py +++ b/tests/fortran/modules/policy/test_module_variable_policy.py @@ -9,7 +9,12 @@ from prik.semantics.fortran2ir import fortran_project_to_semantic_modules from prik.semantics.models import RESOLVED_MODULE_VARIABLE_POLICY_METADATA from prik.policy.ownership import AssignmentMode -from prik.policy.models import ModuleGetterAction, ModuleVariablePolicy +from prik.policy.models import ( + ModuleArrayAddressMechanism, + ModuleGetterAction, + ModuleVariablePolicy, + NativeArrayDescriptorAttribute, +) def test_scalar_module_variable_policy_completes_access_and_storage_before_planning(): @@ -52,6 +57,35 @@ def test_scalar_module_variable_policy_completes_access_and_storage_before_plann assert policies["selected_scale"].native_assignment is AssignmentMode.NONE +def test_fixed_character_handles_publish_only_the_descriptor_attribute_their_callback_can_supply(): + parsed = parse_fortran_project( + { + "character_arrays.f90": """ +module character_arrays + character(len=5), allocatable :: fixed_alloc(:) + character(len=5), pointer :: fixed_pointer(:) => null() + character(len=:), allocatable :: deferred_alloc(:) + real(8), allocatable :: numbers(:) +end module character_arrays +""" + } + ) + modules = fortran_project_to_semantic_modules(parsed) + _apply_source_python_exports(modules) + module = _merge_wrapper_modules(modules, name="character_arrays") + complete_semantic_policies(module) + + handles = { + variable.name: variable.metadata[RESOLVED_MODULE_VARIABLE_POLICY_METADATA].native_array_handle + for variable in module.variables + } + + assert handles["fixed_alloc"].descriptor_attribute is NativeArrayDescriptorAttribute.OTHER + assert handles["fixed_pointer"].descriptor_attribute is NativeArrayDescriptorAttribute.OTHER + assert handles["deferred_alloc"].descriptor_attribute is NativeArrayDescriptorAttribute.ALLOCATABLE + assert handles["numbers"].descriptor_attribute is NativeArrayDescriptorAttribute.ALLOCATABLE + + def test_symbolic_source_parameters_use_native_getters_while_literals_stay_in_binding(): parsed = parse_fortran_project( { @@ -120,17 +154,53 @@ def test_parameter_arrays_complete_as_immutable_native_snapshots(): assert "dpmpar: Final[Float64[3]]" in PyiPrinter().emit(module) -def test_fixed_module_array_requires_explicit_addressable_alias_storage(): +def test_fixed_module_array_address_mechanism_follows_declared_addressability(): + """A fixed module array is borrowed either way; only its address route differs.""" module = parse_pyi_text( """ -from prik.contracts import Float64 +from prik.contracts import Aliased, Annotated, Float64 values: Float64[4] +addressable: Annotated[Float64[4], Aliased] """, module_name="plain_array_state", ) complete_semantic_policies(module) - policy = module.variables[0].metadata[RESOLVED_MODULE_VARIABLE_POLICY_METADATA] - assert policy.supported is False - assert "ordinary module array requires addressable Aliased target storage" in policy.blockers + policies = { + variable.name: variable.metadata[RESOLVED_MODULE_VARIABLE_POLICY_METADATA] for variable in module.variables + } + assert [policy.supported for policy in policies.values()] == [True, True] + assert policies["values"].getter_action is ModuleGetterAction.BORROWED_ARRAY_VIEW + assert policies["values"].array_address is ModuleArrayAddressMechanism.CAPTURED_ADDRESS + assert policies["addressable"].array_address is ModuleArrayAddressMechanism.TARGET_ADDRESS + # Neither route hands Python the whole variable back to reassign. + assert policies["values"].setter_action is SetterAction.REJECT_REPLACEMENT + assert policies["addressable"].setter_action is SetterAction.REJECT_REPLACEMENT + + +def test_logical_module_arrays_are_borrowed_at_every_width(): + """A live view aliases element for element, so the dtype reports the width. + + NumPy has no Boolean wider than one byte, so a logical array is described by + the integer of matching width rather than narrowed to `bool`. The widths + then agree for every Fortran kind and each is borrowed as a live view. + """ + module = parse_pyi_text( + """ +from prik.contracts import Bool, Bool32 + +narrow: Bool[3] +wide: Bool32[3] +""", + module_name="logical_state", + ) + complete_semantic_policies(module) + + policies = { + variable.name: variable.metadata[RESOLVED_MODULE_VARIABLE_POLICY_METADATA] for variable in module.variables + } + for name in ("narrow", "wide"): + assert policies[name].supported is True, name + assert policies[name].getter_action is ModuleGetterAction.BORROWED_ARRAY_VIEW, name + assert policies[name].blockers == (), name diff --git a/tests/fortran/optional_arguments/codegen/test_optional_lowering.py b/tests/fortran/optional_arguments/codegen/test_optional_lowering.py index fcdac1381..4f294cfab 100644 --- a/tests/fortran/optional_arguments/codegen/test_optional_lowering.py +++ b/tests/fortran/optional_arguments/codegen/test_optional_lowering.py @@ -18,6 +18,9 @@ OPTIONAL_FIXED_CONTRACT = ( Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" / "foptional_fixed" / "__init__.pyi" ) +OPTIONAL_MIXED_CONTRACT = ( + Path(__file__).parents[1] / "end_to_end" / "fixtures" / "contracts" / "foptional_f90" / "foptional_f90.pyi" +) def _artifacts(module): @@ -105,6 +108,22 @@ def optional_literal(value: Annotated[Float64, Immutable] | None = ...) -> Float assert "native_optional_literal(literal_0, value=value)" in fortran_source +def test_optional_descriptor_is_passed_into_contained_derived_dispatch(): + """A contained procedure receives, rather than host-associates, the descriptor.""" + module = pyi_file_to_semantic_module(OPTIONAL_MIXED_CONTRACT, module_name="foptional_f90") + fortran_source = _source(_artifacts(module), ".f90") + summarize = fortran_source.split("function bind_c_summarize", maxsplit=1)[1].split( + "end function bind_c_summarize", maxsplit=1 + )[0] + contained = summarize.split(" contains", maxsplit=1)[1] + + assert "if (present(values)) then" in fortran_source + assert "call prik_derived_optional_step_0(prik_optional_values=values)" in fortran_source + assert "real(c_double), dimension(:), optional :: prik_optional_values" in contained + assert "if (present(prik_optional_values)) then" in contained + assert "present(values)" not in contained + + def test_required_descriptor_keeps_python_presence_separate_from_native_state_and_copyout(): module = parse_pyi_text( """ diff --git a/tests/fortran/optional_arguments/end_to_end/fixtures/edited_contracts/optional_array_descriptors/optional_array_descriptors.pyi b/tests/fortran/optional_arguments/end_to_end/fixtures/edited_contracts/optional_array_descriptors/optional_array_descriptors.pyi index b9083e1d5..eb1c0f406 100644 --- a/tests/fortran/optional_arguments/end_to_end/fixtures/edited_contracts/optional_array_descriptors/optional_array_descriptors.pyi +++ b/tests/fortran/optional_arguments/end_to_end/fixtures/edited_contracts/optional_array_descriptors/optional_array_descriptors.pyi @@ -1,4 +1,31 @@ -from prik.contracts import Allocatable, Float64, Int32, Pointer +from prik.contracts import ( + Allocatable, + Annotated, + Float64, + Int32, + Pointer, + PointerAssociation, + PointerPolicy, +) def alloc_state(values: Allocatable[Float64[:]] | None = ...) -> Int32: ... def pointer_state(values: Pointer[Float64[:]] | None = ...) -> Int32: ... +def alloc_fill(values: Allocatable[Float64[:]]) -> None: ... +def pointer_bind( + values: Annotated[ + Pointer[Float64[:]], + PointerAssociation("runtime"), + PointerPolicy( + nullable=True, + transfer="call_local", + target_owner="module", + lifetime="module", + deallocation="never", + shape_source="pointer_bounds", + contiguity="contiguous", + reassociation="native", + aliasing="borrowed", + mutability="view", + ), + ], +) -> None: ... diff --git a/tests/fortran/optional_arguments/end_to_end/fixtures/native/optional_array_descriptors.f90 b/tests/fortran/optional_arguments/end_to_end/fixtures/native/optional_array_descriptors.f90 index 2bd1f154f..6e5a8cfd0 100644 --- a/tests/fortran/optional_arguments/end_to_end/fixtures/native/optional_array_descriptors.f90 +++ b/tests/fortran/optional_arguments/end_to_end/fixtures/native/optional_array_descriptors.f90 @@ -1,5 +1,8 @@ module optional_array_descriptors implicit none + + real(8), target :: pointer_target(3) = [1.0d0, 2.0d0, 3.0d0] + contains integer(4) function alloc_state(values) result(state) real(8), allocatable, optional, intent(in) :: values(:) @@ -24,4 +27,17 @@ integer(4) function pointer_state(values) result(state) state = int(sum(values), kind=4) end if end function pointer_state + subroutine alloc_fill(values) + real(8), allocatable, intent(inout) :: values(:) + + if (allocated(values)) deallocate(values) + allocate(values(3)) + values = [1.0d0, 2.0d0, 3.0d0] + end subroutine alloc_fill + + subroutine pointer_bind(values) + real(8), pointer, intent(inout) :: values(:) + + values => pointer_target + end subroutine pointer_bind end module optional_array_descriptors diff --git a/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py b/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py index 8aba150d3..1f3d76f76 100644 --- a/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py +++ b/tests/fortran/optional_arguments/end_to_end/test_optional_runtime.py @@ -7,12 +7,13 @@ from prik import build_pyi_extension from tests.fortran._support.wrapper_build import ( - _compile_native_object, _build_source_or_generated_pyi_and_import, + _compile_native_object, + _compiler, _import_from_build_dir, _sole_native_module, ) -from prik.runtime.handles import _NativeArrayHandoff, AllocatableArray, PointerArray +from prik.contracts import Allocatable, Float64, Pointer FIXTURES = Path(__file__).parent / "fixtures" OPTIONAL_F90_SOURCE = FIXTURES / "native" / "foptional_f90.f90" @@ -20,62 +21,6 @@ pytestmark = pytest.mark.fortran_end_to_end -def _unallocated_handle_for_rejected_optional_array(): - return AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("optional array path must reject handles before handoff"), - "descriptor": lambda _handle: _NativeArrayHandoff(501), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: None, - "allocated": lambda _handle: False, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, - ) - - -def _unassociated_handle_for_rejected_optional_array(): - return PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("optional array path must reject handles before handoff"), - "descriptor": lambda _handle: _NativeArrayHandoff(502), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, - ) - - -def _optional_descriptor_handle(value: np.ndarray | None, *, pointer: bool): - descriptor = { - "base_addr": 0 if value is None else value.ctypes.data, - "elem_len": np.dtype(np.float64).itemsize, - "rank": 1, - "dim": [ - { - "lower_bound": 0, - "extent": 0 if value is None else value.size, - "sm": np.dtype(np.float64).itemsize, - } - ], - } - operations = { - "array_actual": lambda _handle: _NativeArrayHandoff(value.ctypes.data), - "descriptor": lambda _handle: descriptor, - "shape": lambda _handle: None if value is None else value.shape, - "to_numpy": lambda _handle: value, - "associated" if pointer else "allocated": lambda _handle: value is not None, - "nullify" if pointer else "deallocate": lambda _handle: None, - } - handle_type = PointerArray if pointer else AllocatableArray - return handle_type(dtype=np.dtype(np.float64), rank=1, ops=operations) - - def test_optional_scalar_descriptors_distinguish_omitted_none_and_value(tmp_path: Path): source = FIXTURES / "native" / "optional_scalar_descriptors.f90" native_object = _compile_native_object(source, tmp_path / "native") @@ -83,6 +28,7 @@ def test_optional_scalar_descriptors_distinguish_omitted_none_and_value(tmp_path result = build_pyi_extension( entry, + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "build", @@ -116,22 +62,30 @@ def test_optional_array_descriptors_preserve_presence_and_storage_state(tmp_path result = build_pyi_extension( contract, + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "array_descriptors", ) module = _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) - values = np.array([1.0, 2.0, 3.0], dtype=np.float64) for function_name in ("alloc_state", "pointer_state"): function = getattr(module, function_name) assert function() == np.int32(0) assert function(None) == np.int32(0) - for function_name, pointer in (("alloc_state", False), ("pointer_state", True)): + # A handle that is present but empty must stay distinguishable from an + # absent argument, and the descriptor the callee fills has to be the + # handle's own for the third state to be reachable at all. + for function_name, contract, fill_name in ( + ("alloc_state", Allocatable[Float64[:]], "alloc_fill"), + ("pointer_state", Pointer[Float64[:]], "pointer_bind"), + ): function = getattr(module, function_name) - assert function(_optional_descriptor_handle(None, pointer=pointer)) == np.int32(1) - assert function(_optional_descriptor_handle(values, pointer=pointer)) == np.int32(6) + handle = contract() + assert function(handle) == np.int32(1) + getattr(module, fill_name)(handle) + assert function(handle) == np.int32(6) def test_optional_arguments_drive_fortran_present_behavior( @@ -194,10 +148,6 @@ def test_optional_arguments_drive_fortran_present_behavior( module.summarize(np.int32(5), scale="bad") with pytest.raises(TypeError): module.fill_optional(np.int32(3), np.empty(3, dtype=np.float32)) - with pytest.raises(TypeError): - module.fill_optional(np.int32(3), _unallocated_handle_for_rejected_optional_array()) - with pytest.raises(TypeError): - module.fill_optional(np.int32(3), _unassociated_handle_for_rejected_optional_array()) def test_optional_array_buffers_preserve_omission_and_identity(tmp_path: Path): @@ -206,6 +156,7 @@ def test_optional_array_buffers_preserve_omission_and_identity(tmp_path: Path): contract_package = FIXTURES / "edited_contracts" / "optional_arrays" result = build_pyi_extension( contract_package / "__init__.pyi", + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "build", diff --git a/tests/fortran/pointers/codegen/test_pointer_lowering.py b/tests/fortran/pointers/codegen/test_pointer_lowering.py index 5a4a2c7a8..46dd5862b 100644 --- a/tests/fortran/pointers/codegen/test_pointer_lowering.py +++ b/tests/fortran/pointers/codegen/test_pointer_lowering.py @@ -134,3 +134,23 @@ def test_pointer_lowering_assigns_descriptors_and_emits_manual_target_release(): # Release is manual and caller-driven, matching the ``deallocate`` a Fortran # caller would write for the same pointer; prik never runs it on its own. assert "deallocate(result)" in pointer_operations + + +def test_nullable_scalar_pointer_result_uses_attribute_independent_storage_sizing(): + module = parse_pyi_text( + """ +from prik.contracts import Addr, Aliased, Annotated, Arg, Destruction, Float64, Ownership, Pointer, Return, Transfer, native_call + +@native_call([Addr(Arg(0))], result=Pointer(Return(0))) +def select_scalar( + value: Annotated[Float64, Aliased], +) -> Annotated[Float64, Ownership("python"), Transfer("snapshot_copy"), Destruction("python_refcount")] | None: ... +""", + module_name="pointer_scalar_lowering", + ) + complete_semantic_policies(module) + artifacts = WrapperGenerator().generate(WrapperPlanner().build(module)) + bridge_source = next(source.text for source in artifacts.sources if source.path.suffix == ".f90") + + assert "storage_size(result_value, kind=c_size_t) / 8_c_size_t" in bridge_source + assert "c_sizeof(result_value)" not in bridge_source diff --git a/tests/fortran/pointers/end_to_end/fixtures/contracts/fpointer_handles_policy/fpointer_handles_f90.pyi b/tests/fortran/pointers/end_to_end/fixtures/contracts/fpointer_handles_policy/fpointer_handles_f90.pyi index a71159558..5480d2ea1 100644 --- a/tests/fortran/pointers/end_to_end/fixtures/contracts/fpointer_handles_policy/fpointer_handles_f90.pyi +++ b/tests/fortran/pointers/end_to_end/fixtures/contracts/fpointer_handles_policy/fpointer_handles_f90.pyi @@ -48,6 +48,7 @@ module_allocatable: Annotated[Allocatable[Float64[:]], Aliased] def associate_module_slice() -> None: ... def associate_module_contiguous() -> None: ... +def associate_module_reversed() -> None: ... def allocate_module_values() -> None: ... def box_associate_values(self: pointer_box) -> None: ... diff --git a/tests/fortran/pointers/end_to_end/test_pointer_handles.py b/tests/fortran/pointers/end_to_end/test_pointer_handles.py index f92f7ec25..b33c95872 100644 --- a/tests/fortran/pointers/end_to_end/test_pointer_handles.py +++ b/tests/fortran/pointers/end_to_end/test_pointer_handles.py @@ -14,6 +14,7 @@ _build_text_and_import, _build_source_or_generated_pyi_and_import, _compile_native_object, + _compiler, _import_from_build_dir, _sole_native_module, ) @@ -92,6 +93,10 @@ module_values => module_storage(2:4) end subroutine associate_module_contiguous + subroutine associate_module_reversed() + module_values => module_storage(5:2:-1) + end subroutine associate_module_reversed + subroutine select_module_values(values) real(8), pointer, intent(out) :: values(:) values => module_storage(2:4) @@ -193,6 +198,7 @@ def {total_name}(values: Pointer[Float64[:]]) -> Float64: ... ) result = build_pyi_extension( contract_dir / "__init__.pyi", + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=workdir / "build", @@ -231,6 +237,7 @@ def _pointer_handle_module(build_mode: str, tmp_path: Path): native_object = _compile_native_object(source, tmp_path / "native") result = build_pyi_extension( contract_dir / "__init__.pyi", + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "pyi_build", @@ -247,6 +254,7 @@ def _pointer_descriptor_view_module(tmp_path: Path): contract = CONTRACT_FIXTURES / "fpointer_handles_policy" / "__init__.pyi" result = build_pyi_extension( contract, + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "pyi_build", @@ -271,8 +279,7 @@ def test_module_and_derived_pointer_handles_track_native_association( assert module.module_values is module_handle assert module_handle.associated is True assert module_handle.shape == (2,) - with pytest.raises(ValueError, match="target is noncontiguous"): - module.sum_values(module_handle) + assert module.sum_values(module_handle) == np.float64(6.0) with pytest.raises(NotImplementedError, match="to_numpy extraction is unsupported"): module_handle.to_numpy() @@ -390,6 +397,33 @@ def test_caller_created_pointer_crosses_separately_built_extensions(tmp_path: Pa assert values.closed is True +def test_a_reversed_pointer_target_keeps_its_data_pointer_strides_and_span(tmp_path: Path): + """A negative stride reaches the view exactly as the descriptor records it. + + The descriptor's base address is the first element in Fortran order and its + stride multiplier is signed, which is also what NumPy indexes with, so the + view is built from them directly rather than from a window computed around + them. + """ + module = _pointer_descriptor_view_module(tmp_path) + handle = module.module_values + module.associate_module_reversed() + + view = handle.to_numpy() + + assert view.shape == (4,) + assert view.strides == (-8,) + np.testing.assert_allclose(view, np.array([5.0, 4.0, 3.0, 2.0], dtype=np.float64)) + + # The view spans the same storage as the forward slice of the same target. + view[0] = np.float64(50.0) + module.associate_module_contiguous() + np.testing.assert_allclose(module_handle_view := handle.to_numpy(), np.array([2.0, 3.0, 4.0])) + assert module_handle_view.strides == (8,) + module.associate_module_reversed() + np.testing.assert_allclose(handle.to_numpy(), np.array([50.0, 4.0, 3.0, 2.0])) + + def test_pointer_descriptor_views_preserve_slice_shape_strides_and_parent_lifetime(tmp_path: Path): module = _pointer_descriptor_view_module(tmp_path) @@ -446,7 +480,7 @@ def test_module_native_array_handles_use_canonical_plan(tmp_path: Path): contract = tmp_path / "pointer_handles" / "fpointer_handles_f90.pyi" contract.parent.mkdir() contract.write_text( - """from prik.contracts import Aliased, Allocatable, Annotated, Float64, Pointer, PointerAssociation, PointerPolicy + """from prik.contracts import Aliased, Allocatable, Annotated, Float64, Pointer, PointerAssociation, PointerPolicy, bind module_values: Annotated[ Pointer[Float64[:]], @@ -470,6 +504,8 @@ def associate_module_slice() -> None: ... def associate_module_contiguous() -> None: ... def allocate_module_values() -> None: ... def sum_values(values: Float64[:]) -> Float64: ... +@bind("sum_values") +def sum_four(values: Float64[4]) -> Float64: ... def sum_pointer_descriptor(values: Pointer[Float64[:]]) -> Float64: ... def sum_allocatable_descriptor(values: Allocatable[Float64[:]]) -> Float64: ... """, @@ -477,6 +513,7 @@ def sum_allocatable_descriptor(values: Allocatable[Float64[:]]) -> Float64: ... ) result = build_pyi_extension( contract, + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "build", @@ -508,6 +545,8 @@ def sum_allocatable_descriptor(values: Allocatable[Float64[:]]) -> Float64: ... assert allocatable_handle.allocated is True np.testing.assert_allclose(allocatable_handle.to_numpy(), np.array([10.0, 20.0, 30.0])) assert module.sum_allocatable_descriptor(allocatable_handle) == np.float64(60.0) + with pytest.raises(TypeError, match="incompatible shape at axis 0"): + module.sum_four(allocatable_handle) allocatable_handle.deallocate() assert allocatable_handle.allocated is False @@ -548,6 +587,7 @@ def sum_pointer_descriptor(values: Pointer[Float64[:]]) -> Float64: ... ) result = build_pyi_extension( contract, + input_compiler=_compiler(), native_objects=[native_object], native_include_dirs=[native_object.parent], output_dir=tmp_path / "build", @@ -680,3 +720,87 @@ def peak_kib() -> int: # A borrowed target is module storage the library keeps; releasing is the # caller's decision there too, so only the untouched path is asserted. assert module.borrow(np.int32(4)).associated is True + + +POINTER_REASSOCIATION_SOURCE = """\ +module fpointer_reassociate_f90 + implicit none + + real(8), target :: small_target(3) = [1.0_8, 2.0_8, 3.0_8] + real(8), target :: large_target(5) = [10.0_8, 20.0_8, 30.0_8, 40.0_8, 50.0_8] + +contains + + subroutine repoint(values) + real(8), pointer, intent(inout) :: values(:) + values => large_target + end subroutine repoint + + function total(values) result(sum_values) + real(8), pointer, intent(in) :: values(:) + real(8) :: sum_values + sum_values = 0.0_8 + if (associated(values)) sum_values = sum(values) + end function total + +end module fpointer_reassociate_f90 +""" + + +def test_callee_reassociation_of_an_inout_pointer_dummy_reaches_the_caller_handle(tmp_path: Path): + """A callee's ``values => target`` must reach the handle that was passed in. + + ``intent(inout)`` carries no output projection, so nothing re-reads the + descriptor after the call. The descriptor the callee re-points therefore + has to be the caller's own, not one the wrapper rebuilt for the call. + """ + source = tmp_path / "native" / "fpointer_reassociate_f90.f90" + source.parent.mkdir() + source.write_text(POINTER_REASSOCIATION_SOURCE, encoding="utf-8") + native_object = _compile_native_object(source, tmp_path / "native_build") + contract = tmp_path / "contracts" / "fpointer_reassociate_f90.pyi" + contract.parent.mkdir() + contract.write_text( + """from prik.contracts import Annotated, Float64, Pointer, PointerAssociation, PointerPolicy + +def repoint( + values: Annotated[ + Pointer[Float64[:]], + PointerAssociation("runtime"), + PointerPolicy( + nullable=True, + transfer="call_local", + target_owner="module", + lifetime="module", + deallocation="never", + shape_source="pointer_bounds", + contiguity="contiguous", + reassociation="native", + aliasing="borrowed", + mutability="view", + ), + ], +) -> None: ... + +def total(values: Pointer[Float64[:]]) -> Float64: ... +""", + encoding="utf-8", + ) + result = build_pyi_extension( + contract, + input_compiler=_compiler(), + native_objects=[native_object], + native_include_dirs=[native_object.parent], + output_dir=tmp_path / "build", + ) + module = _sole_native_module(_import_from_build_dir(result.module_name, result.output_dir)) + + handle = Pointer[Float64[:]]() + assert handle.associated is False + assert module.total(handle) == np.float64(0.0) + + module.repoint(handle) + + assert handle.associated is True + assert handle.shape == (5,) + assert module.total(handle) == np.float64(150.0) diff --git a/tests/fortran/pointers/policy/test_pointer_ownership_policy.py b/tests/fortran/pointers/policy/test_pointer_ownership_policy.py index c17eea2f6..32351a53d 100644 --- a/tests/fortran/pointers/policy/test_pointer_ownership_policy.py +++ b/tests/fortran/pointers/policy/test_pointer_ownership_policy.py @@ -389,6 +389,31 @@ class box: assert set(field_policy.operations) == {"associate", "associated", "deallocate", "nullify", "to_numpy"} +def test_deferred_character_pointer_arrays_require_a_legal_descriptor_interface(): + module = parse_pyi_text( + """ +deferred_ptr: Pointer[String[:][:]] + +def inspect(values: Pointer[String[:][:]]) -> None: ... +""", + module_name="deferred_character_pointer_arrays", + ) + + complete_semantic_policies(module) + + module_policy = module.variables[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] + argument_policy = module.functions[0].arguments[0].metadata[RESOLVED_NATIVE_ARRAY_HANDLE_POLICY_METADATA] + + assert module_policy.is_blocked is False + assert module_policy.descriptor_inquiries is False + assert module_policy.descriptor_interop == "none" + assert set(module_policy.operations) == {"associated", "deallocate", "nullify"} + assert argument_policy.is_blocked is True + assert argument_policy.descriptor_inquiries is False + assert argument_policy.descriptor_interop == "none" + assert "cannot cross a bind(C) descriptor interface" in argument_policy.blocker + + def test_complete_pointer_policy_metadata_round_trips_without_overriding_container_ownership(): module = parse_pyi_text( """ @@ -600,8 +625,11 @@ def make_target() -> Pointer[Float64[:]]: ... assert target_values.handle_kind == "borrowed_module_descriptor" assert target_values.owner_retention == "native_module" assert target_values.target_lifetime == "module" - assert target_values.to_numpy == "borrowed_view" - assert target_values.descriptor_interop == "none" + # A module allocatable reports its own descriptor whether or not it is a + # target, so `Aliased` selects neither a different NumPy exposure nor a + # different interop mechanism. + assert target_values.to_numpy == "descriptor_view" + assert target_values.descriptor_interop == "module_allocatable_c_descriptor" assert target_values.requires_pointer_c_descriptor_interop is False assert field_values.handle_kind == "borrowed_field_descriptor" @@ -632,7 +660,9 @@ def make_target() -> Pointer[Float64[:]]: ... assert argument_values.descriptor_interop == "none" assert argument_values.requires_pointer_c_descriptor_interop is False assert set(argument_values.operations) == {"allocated", "to_numpy"} - assert argument_values.default_construction == "fact_packed_empty" + # A non-optional descriptor argument is handed a descriptor the Fortran + # runtime built, so a caller-created handle needs storage of its own. + assert argument_values.default_construction == "lazy_owned_descriptor" assert argument_values.default_descriptor_ownership == "owned" assert argument_values.default_release == "wrapper_dealloc" assert argument_values.default_destroy_behavior == "handle_finalizer" @@ -649,7 +679,9 @@ def make_target() -> Pointer[Float64[:]]: ... assert optional_target.descriptor_interop == "pointer_c_descriptor" assert optional_target.requires_pointer_c_descriptor_interop is True assert set(optional_target.operations) == {"associate", "associated", "nullify", "to_numpy"} - assert optional_target.default_construction == "fact_packed_empty" + # An optional argument is a descriptor argument like any other when it is + # present, so a caller-created handle needs storage of its own to hand over. + assert optional_target.default_construction == "lazy_owned_descriptor" assert "destroy" in optional_target.default_operations assert "allocate" not in optional_target.operations assert "deallocate" not in optional_target.operations diff --git a/tests/fortran/pointers/runtime/test_pointer_array_actual_abi.py b/tests/fortran/pointers/runtime/test_pointer_array_actual_abi.py deleted file mode 100644 index fb9d8d78a..000000000 --- a/tests/fortran/pointers/runtime/test_pointer_array_actual_abi.py +++ /dev/null @@ -1,68 +0,0 @@ -"""Pointer array-actual handoff behavior.""" - -import numpy as np -import pytest -from prik.runtime.handles import ( - PointerArray, - _native_array_actual_argument_for_binding_positional, -) -from tests.fortran._support.native_array_handles import ( - _ArrayState, - _handoff, -) - - -def test_array_actual_argument_abi_packer_uses_pointer_native_array_actual_dtype_metadata(): - actual = _handoff(248) - handle = PointerArray( - dtype=np.dtype(np.float32), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(249), - "shape": lambda _handle: (3,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "array_actual": lambda _handle: actual, - }, - to_numpy_policy="unsupported", - ) - - assert _native_array_actual_argument_for_binding_positional( - handle, - None, - 1, - (3,), - None, - False, - False, - False, - False, - True, - False, - ) == (actual.address, np.dtype(np.float32).itemsize, 3) - - -def test_pointer_array_actual_hook_requires_associated_state_without_to_numpy(): - actual = _handoff(242) - calls = [] - state = _ArrayState() - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "descriptor": lambda _handle: _handoff(243), - "shape": lambda _handle: state.shape, - "associated": lambda _handle: state.shape is not None, - "nullify": lambda _handle: setattr(state, "shape", None), - "to_numpy": lambda _handle: pytest.fail("array-actual handoff must not call to_numpy"), - "array_actual": lambda _handle: calls.append("array_actual") or actual, - }, - ) - - with pytest.raises(ValueError, match="unassociated"): - handle._array_actual_for_binding(expected_dtype=np.float64, expected_rank=1) - - state.shape = (3,) - - assert handle._array_actual_for_binding(expected_dtype="float64", expected_rank=1) is actual - assert calls == ["array_actual"] diff --git a/tests/fortran/pointers/runtime/test_pointer_contract_handles.py b/tests/fortran/pointers/runtime/test_pointer_contract_handles.py index 2cd056471..d3f2431fe 100644 --- a/tests/fortran/pointers/runtime/test_pointer_contract_handles.py +++ b/tests/fortran/pointers/runtime/test_pointer_contract_handles.py @@ -8,24 +8,14 @@ AllocatableArray, PointerArray, _bind_contract_native_array_handle, - _native_array_actual_for_binding, + _numpy_view_from_descriptor_facts, +) +from tests.fortran._support.native_array_handles import ( + _absent_descriptor_facts, + _descriptor_facts_for_array, + _generated_handle_dispatch, + _handle_dispatch, ) - - -def _pointer_descriptor(value): - return { - "base_addr": int(value.ctypes.data), - "elem_len": int(value.dtype.itemsize), - "rank": value.ndim, - "dim": [ - { - "lower_bound": 1, - "extent": int(extent), - "sm": int(stride), - } - for extent, stride in zip(value.shape, value.strides, strict=True) - ], - } def test_contract_default_handle_constructors_preserve_dtype_rank_and_empty_state(): @@ -51,28 +41,24 @@ def test_contract_default_handle_constructors_preserve_dtype_rank_and_empty_stat def test_fresh_pointer_associate_copies_association_without_following_source_descriptor(): value = np.arange(6, dtype=np.float64)[::2] - source_state = {"descriptor": _pointer_descriptor(value)} + source_state = {"facts": _descriptor_facts_for_array(value)} def source_nullify(_handle): - source_state["descriptor"] = { - "base_addr": 0, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 0, "sm": 8}], - } + source_state["facts"] = _absent_descriptor_facts("float64", 1) source = PointerArray( dtype="float64", rank=1, - ops={ - "shape": lambda _handle: value.shape, - "array_actual": lambda _handle: int(value.ctypes.data), - "descriptor": lambda _handle: source_state["descriptor"], - "to_numpy": lambda _handle: source_state["descriptor"], - "associated": lambda _handle: source_state["descriptor"]["base_addr"] != 0, - "associate": lambda _handle, descriptor: source_state.update(descriptor=descriptor), - "nullify": source_nullify, - }, + **_handle_dispatch( + { + "shape": lambda _handle: value.shape if source_state["facts"][0] else None, + "descriptor": lambda _handle: source_state["facts"], + "to_numpy": lambda _handle: _numpy_view_from_descriptor_facts(source_state["facts"], "float64"), + "associated": lambda _handle: source_state["facts"][0] != 0, + "associate": lambda _handle, facts: source_state.update(facts=facts), + "nullify": source_nullify, + } + ), to_numpy_policy="descriptor_view", ) target = contracts.Pointer[contracts.Float64[:]]() @@ -81,7 +67,6 @@ def source_nullify(_handle): assert target.associated is True assert target.shape == (3,) np.testing.assert_array_equal(target.to_numpy(), value) - assert _native_array_actual_for_binding(target).address == value.ctypes.data source.nullify() assert source.associated is False @@ -95,19 +80,20 @@ def source_nullify(_handle): def test_fresh_pointer_pending_association_is_applied_when_native_storage_attaches(): value = np.arange(4, dtype=np.float64) - descriptor = _pointer_descriptor(value) + facts = _descriptor_facts_for_array(value) source = PointerArray( dtype="float64", rank=1, - ops={ - "shape": lambda _handle: value.shape, - "array_actual": lambda _handle: int(value.ctypes.data), - "descriptor": lambda _handle: descriptor, - "to_numpy": lambda _handle: descriptor, - "associated": lambda _handle: True, - "associate": lambda _handle, _descriptor: None, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: value.shape, + "descriptor": lambda _handle: facts, + "to_numpy": lambda _handle: _numpy_view_from_descriptor_facts(facts, "float64"), + "associated": lambda _handle: True, + "associate": lambda _handle, _facts: None, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="descriptor_view", ) target = contracts.Pointer[contracts.Float64[:]]() @@ -120,20 +106,21 @@ def associate(received_owner, facts): received.append((received_owner, facts)) state["associated"] = True + operations = { + "shape": lambda _owner: value.shape if state["associated"] else None, + "descriptor": lambda _owner: facts, + "associated": lambda _owner: state["associated"], + "associate": associate, + "nullify": lambda _owner: state.update(associated=False), + "destroy": lambda _owner: None, + } _bind_contract_native_array_handle( target, "pointer", "float64", 1, - { - "shape": lambda _owner: value.shape if state["associated"] else None, - "array_actual": lambda _owner: int(value.ctypes.data), - "descriptor": lambda received_owner: received_owner, - "associated": lambda _owner: state["associated"], - "associate": associate, - "nullify": lambda _owner: state.update(associated=False), - "destroy": lambda _owner: None, - }, + _generated_handle_dispatch(operations), + operations, owner, "owned", "unsupported", @@ -179,12 +166,14 @@ def test_generated_storage_rejects_incompatible_contract_handles( handle = prepare() with pytest.raises(error, match=message): + operations = {} _bind_contract_native_array_handle( handle, descriptor_kind, dtype, rank, - {}, + _generated_handle_dispatch(operations), + operations, object(), "owned", "unsupported", diff --git a/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py b/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py index 182b34371..4050e3a1a 100644 --- a/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py +++ b/tests/fortran/pointers/runtime/test_pointer_descriptor_abi.py @@ -1,526 +1,180 @@ -"""Pointer descriptor handoff and strided-view ABI behavior.""" +"""Descriptor handoff and view construction through the runtime ABI.""" import numpy as np import pytest + +import prik.contracts as contracts from prik.runtime.handles import ( AllocatableArray, PointerArray, - _NativeArrayDescriptorHandoff, - _native_array_descriptor_argument_for_binding, - _native_array_descriptor_argument_for_binding_positional, - _native_array_descriptor_for_binding, - _native_array_descriptor_handoff_for_binding, - _native_array_descriptor_handoff_for_binding_positional, - _numpy_view_from_pointer_c_descriptor, + _bind_contract_native_array_handle, + _native_array_backend_for_binding, + _native_array_backend_for_binding_positional, + _numpy_view_from_descriptor_facts, ) from tests.fortran._support.native_array_handles import ( - _ArrayState, - _common_ops, - _handoff, - _pointer_descriptor_for_array, - _pointer_descriptor_record_for_array, - _required_handoff_ops, + _descriptor_facts_for_array, + _generated_handle_dispatch, + _handle_dispatch, ) -def test_descriptor_hook_rejects_generated_none_handoff(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: None, - "allocated": lambda _handle: False, - "descriptor": lambda _handle: None, - }, - to_numpy_policy="unsupported", - ) - - with pytest.raises( - TypeError, match="descriptor operation must return descriptor fields or an integer data address" - ): - _native_array_descriptor_for_binding(handle, descriptor_kind="allocatable") - - -def test_descriptor_hook_validates_expected_dtype_rank_and_current_shape(): - descriptor = _handoff(235) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=2, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (2, 3), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: descriptor, - }, - to_numpy_policy="unsupported", - ) - - assert handle._descriptor_for_binding( - expected_dtype=np.float64, - expected_rank=2, - expected_shape=(2, 3), - ) == { - "base_addr": descriptor.address, - "elem_len": 8, - "rank": 2, - "dim": [ - {"lower_bound": 0, "extent": 2, "sm": 8}, - {"lower_bound": 0, "extent": 3, "sm": 16}, - ], - } - with pytest.raises(ValueError, match="expected rank 1"): - handle._descriptor_for_binding(expected_rank=1) - with pytest.raises(TypeError, match="expected dtype"): - handle._descriptor_for_binding(expected_dtype=np.int32) - with pytest.raises(ValueError, match=r"expected shape .* axis 1"): - handle._descriptor_for_binding(expected_shape=(2, 4)) - - -def test_descriptor_binding_helper_accepts_matching_handles_and_optional_none(): - alloc_descriptor = _handoff(236) - pointer_descriptor = _handoff(237) - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (0,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: alloc_descriptor, - }, - to_numpy_policy="unsupported", - ) - pointer_handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - "descriptor": lambda _handle: pointer_descriptor, - }, +def _bound_pointer(backend, *, dtype=np.float64, rank=1): + """Return a pointer handle standing for storage a wrapper attached.""" + handle = PointerArray( + dtype=np.dtype(dtype), + rank=rank, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "associated": lambda _handle: False, + "nullify": lambda _handle: None, + "descriptor": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ) + handle._native_backend = backend + return handle - assert _native_array_descriptor_for_binding( - alloc_handle, - descriptor_kind="allocatable", - expected_dtype=np.float64, - expected_rank=1, - expected_shape=(0,), - ) == { - "base_addr": alloc_descriptor.address, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 0, "sm": 8}], - } - assert _native_array_descriptor_for_binding( - pointer_handle, - descriptor_kind="pointer", - expected_dtype=np.float64, - expected_rank=1, - ) == { - "base_addr": pointer_descriptor.address, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 0, "sm": 8}], - } - assert _native_array_descriptor_for_binding(None, descriptor_kind="pointer", optional=True) is None - - -def test_descriptor_binding_helper_rejects_plain_arrays_none_and_wrong_kind(): - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: _handoff(238), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(TypeError, match="received ndarray"): - _native_array_descriptor_for_binding(np.ones(1), descriptor_kind="allocatable") - with pytest.raises(TypeError, match="received None"): - _native_array_descriptor_for_binding(None, descriptor_kind="allocatable") - with pytest.raises(TypeError, match="expected pointer native array handle"): - _native_array_descriptor_for_binding(alloc_handle, descriptor_kind="pointer") - with pytest.raises(ValueError, match="unsupported native array descriptor kind"): - _native_array_descriptor_for_binding(alloc_handle, descriptor_kind="coarray") - with pytest.raises(ValueError, match="unsupported native array descriptor kind"): - _native_array_descriptor_for_binding(None, descriptor_kind="coarray", optional=True) - - -def test_descriptor_argument_abi_packer_returns_required_descriptor_fields(): - descriptor = _handoff(239) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: descriptor, - }, - to_numpy_policy="unsupported", - ) +def test_descriptor_argument_hands_over_the_backend_the_handle_publishes(): + """The backend is what crosses; nothing rebuilds a descriptor in Python.""" + backend = object() + handle = _bound_pointer(backend) - assert _native_array_descriptor_argument_for_binding( + assert _native_array_backend_for_binding( handle, - descriptor_kind="allocatable", + descriptor_kind="pointer", expected_dtype=np.float64, expected_rank=1, - expected_shape=(2,), - ) == (descriptor.address, 8, 1, 0, 2, 8) - - -def test_descriptor_argument_abi_packer_positional_helper_matches_generated_call_shape(): - descriptor = _handoff(242) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: descriptor, - }, - to_numpy_policy="unsupported", - ) - - assert _native_array_descriptor_argument_for_binding_positional( - handle, - "allocatable", - "float64", - 1, - None, - False, - ) == (descriptor.address, 8, 1, 0, 2, 8) - assert _native_array_descriptor_argument_for_binding_positional( - None, - "allocatable", - "float64", - 1, + ) == (backend,) + assert _native_array_backend_for_binding( None, - True, - ) == (None, None, None, None, None, None, None) + descriptor_kind="pointer", + optional_absent=True, + ) == (None, None) -def test_descriptor_argument_abi_packer_maps_optional_presence_and_absence(): - descriptor = _handoff(240) - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - "descriptor": lambda _handle: descriptor, - }, - to_numpy_policy="unsupported", - ) - - *descriptor_fields, presence_token = _native_array_descriptor_argument_for_binding( - handle, +def test_an_optional_descriptor_argument_reports_presence_alongside_its_backend(): + backend = object() + supplied, presence = _native_array_backend_for_binding( + _bound_pointer(backend), descriptor_kind="pointer", expected_dtype=np.float64, expected_rank=1, optional_absent=True, ) - assert descriptor_fields == [descriptor.address, 8, 1, 0, 0, 8] - assert presence_token is not None - assert presence_token != descriptor.address - assert _native_array_descriptor_argument_for_binding( - None, - descriptor_kind="pointer", - expected_rank=1, - optional_absent=True, - ) == (None, None, None, None, None, None, None) - - -def test_descriptor_argument_abi_packer_rejects_wrong_kind_and_unsupported_descriptor_kind(): - alloc_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: _handoff(241), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(TypeError, match="expected pointer native array handle"): - _native_array_descriptor_argument_for_binding(alloc_handle, descriptor_kind="pointer") - with pytest.raises(ValueError, match="unsupported native array descriptor kind"): - _native_array_descriptor_argument_for_binding(None, descriptor_kind="coarray", optional_absent=True) - with pytest.raises(TypeError, match="received ndarray"): - _native_array_descriptor_argument_for_binding(np.ones(1), descriptor_kind="allocatable") - - -def test_projected_descriptor_handoff_requires_persistent_standard_descriptor_storage(): - owner = object() - direct = _NativeArrayDescriptorHandoff(owner) - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: direct, - }, - to_numpy_policy="unsupported", - ) - - assert _native_array_descriptor_handoff_for_binding( - handle, - descriptor_kind="allocatable", - expected_dtype=np.float64, - expected_rank=1, - expected_shape=(2,), - ) == (owner,) - assert _native_array_descriptor_handoff_for_binding_positional( + assert supplied is backend + assert isinstance(presence, int) + assert presence > 0 + + +@pytest.mark.parametrize( + ("value", "error", "message"), + [ + (np.zeros(2, dtype=np.float64), TypeError, "expected pointer native array handle"), + (None, TypeError, "handle argument is required"), + ( + AllocatableArray( + dtype=np.dtype(np.float64), + rank=1, + **_handle_dispatch({"shape": lambda _handle: None, "allocated": lambda _handle: False}), + to_numpy_policy="unsupported", + ), + TypeError, + "expected pointer native array handle", + ), + ], +) +def test_descriptor_argument_rejects_values_that_are_not_the_declared_handle(value, error, message: str): + with pytest.raises(error, match=message): + _native_array_backend_for_binding(value, descriptor_kind="pointer") + + +def test_descriptor_argument_rejects_a_mismatched_dtype_or_rank(): + handle = _bound_pointer(object()) + + with pytest.raises(ValueError, match="does not match expected rank 2"): + _native_array_backend_for_binding(handle, descriptor_kind="pointer", expected_rank=2) + with pytest.raises(TypeError, match="does not match expected dtype"): + _native_array_backend_for_binding(handle, descriptor_kind="pointer", expected_dtype=np.int32) + + +def test_descriptor_argument_refuses_a_handle_that_has_no_storage_yet(): + """A fresh contract handle publishes no backend until a binder attaches one.""" + handle = contracts.Pointer[contracts.Float64[:]]() + + with pytest.raises(TypeError, match="requires generated persistent descriptor storage"): + _native_array_backend_for_binding(handle, descriptor_kind="pointer") + + +def test_descriptor_argument_binds_a_fresh_contract_handle_then_reads_its_backend(): + handle = contracts.Pointer[contracts.Float64[:]]() + backend = object() + + def bind_default(value): + operations = { + "shape": lambda _owner: None, + "associated": lambda _owner: False, + "nullify": lambda _owner: None, + "descriptor": lambda _owner: None, + "associate": lambda _owner, _facts: None, + "destroy": lambda _owner: None, + } + _bind_contract_native_array_handle( + value, + "pointer", + "float64", + 1, + _generated_handle_dispatch(operations), + operations, + backend, + "owned", + "unsupported", + native_backend=backend, + ) + + assert _native_array_backend_for_binding_positional( handle, - "allocatable", - "float64", + "pointer", + np.float64, 1, - (2,), False, - ) == (owner,) - - -def test_owned_standard_descriptor_can_supply_fact_packed_read_only_handoff(): - owner = object() - direct = _NativeArrayDescriptorHandoff(owner) - record = { - "base_addr": 0x5678, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 2, "sm": 8}], - } - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (2,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - "descriptor": lambda _handle: direct, - "to_numpy": lambda _handle: record, - "destroy": lambda _handle: None, - }, - descriptor_ownership="owned", - to_numpy_policy="unsupported", - ) + bind_default, + ) == (backend,) - assert _native_array_descriptor_argument_for_binding( - handle, - descriptor_kind="pointer", - expected_dtype=np.float64, - expected_rank=1, - expected_shape=(2,), - ) == (0x5678, 8, 1, 0, 2, 8) - assert _native_array_descriptor_handoff_for_binding( - handle, - descriptor_kind="pointer", - expected_dtype=np.float64, - expected_rank=1, - expected_shape=(2,), - ) == (owner,) - assert _native_array_descriptor_handoff_for_binding_positional( - None, - "allocatable", - "float64", - 1, - None, - True, - ) == (None, None) - fact_packed = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: pytest.fail("descriptor handoff must not request array actual"), - "shape": lambda _handle: (2,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: _handoff(0x5678), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(TypeError, match="requires a generated direct descriptor handoff"): - _native_array_descriptor_handoff_for_binding(fact_packed, descriptor_kind="allocatable") - - -def test_pointer_c_descriptor_helper_builds_strided_numpy_view_from_decoded_fields(): - source = np.arange(24, dtype=np.float64).reshape((4, 6), order="F") - expected = source[1:4:2, 2:6:3] - - view = _numpy_view_from_pointer_c_descriptor(_pointer_descriptor_for_array(expected), dtype=np.float64) - - assert view.shape == expected.shape - assert view.strides == expected.strides - np.testing.assert_allclose(view, expected) - assert np.shares_memory(view, source) - - view[0, 1] = 77.0 - assert expected[0, 1] == 77.0 - - -def test_pointer_c_descriptor_helper_builds_negative_stride_numpy_view_from_decoded_fields(): - source = np.arange(10, dtype=np.float64) - expected = source[8:1:-2] - - view = _numpy_view_from_pointer_c_descriptor(_pointer_descriptor_for_array(expected), dtype=np.float64) - - assert view.shape == expected.shape - assert view.strides == expected.strides - np.testing.assert_allclose(view, expected) - assert np.shares_memory(view, source) - - view[0] = 88.0 - assert source[8] == 88.0 - - -def test_pointer_c_descriptor_helper_accepts_field_record_objects(): - source = np.arange(12, dtype=np.float64).reshape((3, 4)) - expected = source[:, 1:4:2] - - view = _numpy_view_from_pointer_c_descriptor(_pointer_descriptor_record_for_array(expected), dtype=np.float64) - - assert view.shape == expected.shape - assert view.strides == expected.strides - np.testing.assert_allclose(view, expected) - assert np.shares_memory(view, source) - - view[1, 0] = 55.0 - assert source[1, 1] == 55.0 - - -def test_pointer_c_descriptor_helper_validates_decoded_descriptor_fields(): - source = np.arange(4, dtype=np.float64) - descriptor = _pointer_descriptor_for_array(source) - - assert _numpy_view_from_pointer_c_descriptor({**descriptor, "base_addr": 0}, dtype=np.float64) is None - for field in ("base_addr", "elem_len", "rank", "dim"): - incomplete = dict(descriptor) - incomplete.pop(field) - with pytest.raises(TypeError, match=f"field {field!r} is required"): - _numpy_view_from_pointer_c_descriptor(incomplete, dtype=np.float64) - with pytest.raises(ValueError, match="base_addr must be non-negative"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "base_addr": -1}, dtype=np.float64) - with pytest.raises(TypeError, match="field 'base_addr' must be an integer"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "base_addr": True}, dtype=np.float64) - with pytest.raises(ValueError, match=r"elem_len .* itemsize"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "elem_len": 4}, dtype=np.float64) - with pytest.raises(ValueError, match="rank must be non-negative"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "rank": -1}, dtype=np.float64) - with pytest.raises(ValueError, match="rank 2 does not match 1 dimension records"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "rank": 2}, dtype=np.float64) - for field in ("lower_bound", "extent", "sm"): - bad_dim_record = dict(descriptor["dim"][0]) - bad_dim_record.pop(field) - with pytest.raises(TypeError, match=f"dim\\[0\\] field {field!r} is required"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "dim": [bad_dim_record]}, dtype=np.float64) - with pytest.raises(TypeError, match="field 'lower_bound' must be an integer"): - bad_dim = [{**descriptor["dim"][0], "lower_bound": True}] - _numpy_view_from_pointer_c_descriptor({**descriptor, "dim": bad_dim}, dtype=np.float64) - with pytest.raises(ValueError, match="dim\\[0\\]\\.extent must be non-negative"): - bad_dim = [{**descriptor["dim"][0], "extent": -1}] - _numpy_view_from_pointer_c_descriptor({**descriptor, "dim": bad_dim}, dtype=np.float64) - with pytest.raises(TypeError, match="field 'sm' must be an integer"): - bad_dim = [{**descriptor["dim"][0], "sm": True}] - _numpy_view_from_pointer_c_descriptor({**descriptor, "dim": bad_dim}, dtype=np.float64) - with pytest.raises(TypeError, match="field 'dim' must be a sequence"): - _numpy_view_from_pointer_c_descriptor({**descriptor, "dim": None}, dtype=np.float64) - - -def test_pointer_descriptor_view_policy_uses_decoded_descriptor_fields_for_strided_view(): - source = np.arange(10, dtype=np.float64) - strided = source[1::2] - state = _ArrayState(shape=strided.shape, value=strided) - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - **_common_ops(state), - "to_numpy": lambda _handle: _pointer_descriptor_for_array(strided), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, - to_numpy_policy="descriptor_view", - ) +def test_view_from_facts_preserves_a_strided_target(): + source = np.arange(8, dtype=np.float64) + strided = source[::2] - view = handle.to_numpy() + view = _numpy_view_from_descriptor_facts(_descriptor_facts_for_array(strided), np.float64) - assert view is not strided - assert np.shares_memory(view, source) - assert view.strides == strided.strides + assert view.shape == (4,) + assert view.strides == (16,) np.testing.assert_allclose(view, strided) - view[0] = 42.0 - assert source[1] == 42.0 + view[1] = np.float64(99.0) + assert source[2] == np.float64(99.0) -def test_pointer_descriptor_view_policy_rejects_decoded_descriptor_rank_mismatch(): - source = np.arange(4, dtype=np.float64) - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=2, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2, 2), - "to_numpy": lambda _handle: _pointer_descriptor_for_array(source), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, - to_numpy_policy="descriptor_view", - ) +def test_view_from_facts_preserves_a_negative_stride_target(): + """A reversed target keeps its data pointer, strides and accessible span.""" + source = np.arange(6, dtype=np.float64) + reversed_view = source[::-1] - with pytest.raises(ValueError, match="pointer descriptor rank 1 does not match declared handle rank 2"): - handle.to_numpy() + view = _numpy_view_from_descriptor_facts(_descriptor_facts_for_array(reversed_view), np.float64) + assert view.shape == (6,) + assert view.strides == (-8,) + np.testing.assert_allclose(view, source[::-1]) + view[0] = np.float64(42.0) + assert source[5] == np.float64(42.0) -def test_pointer_descriptor_view_policy_maps_null_decoded_descriptor_to_none(): - source = np.arange(4, dtype=np.float64) - descriptor = {**_pointer_descriptor_for_array(source), "base_addr": 0} - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "to_numpy": lambda _handle: descriptor, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, - to_numpy_policy="descriptor_view", - ) - assert handle.to_numpy() is None - - -def test_pointer_descriptor_view_policy_rejects_null_descriptor_for_present_state(): - source = np.arange(4, dtype=np.float64) - descriptor = {**_pointer_descriptor_for_array(source), "base_addr": 0} - handle = PointerArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (4,), - "to_numpy": lambda _handle: descriptor, - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, - to_numpy_policy="descriptor_view", - ) +def test_view_from_facts_reports_absent_storage_and_a_disagreeing_element_width(): + assert _numpy_view_from_descriptor_facts((0, 8, 1, 0, 0, 8), np.float64) is None - with pytest.raises(TypeError, match="null descriptor for present descriptor state"): - handle.to_numpy() + with pytest.raises(ValueError, match="does not match NumPy dtype itemsize"): + _numpy_view_from_descriptor_facts((1024, 4, 1, 1, 2, 4), np.float64) diff --git a/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py b/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py index fd2c2bacf..ef2560845 100644 --- a/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py +++ b/tests/fortran/pointers/runtime/test_pointer_handle_protocol.py @@ -6,58 +6,60 @@ AllocatableArray, NativeArrayHandleBase, PointerArray, - _native_array_actual_for_binding, - _native_array_descriptor_for_binding, - _native_array_handle_from_generated_ops, + _native_array_handle_from_generated_dispatch, + _numpy_view_from_descriptor_facts, ) from tests.fortran._support.native_array_handles import ( _ArrayState, _common_ops, - _handoff, - _pointer_descriptor_for_array, - _required_handoff_ops, + _descriptor_facts_for_array, + _generated_handle_dispatch, + _handle_dispatch, ) -def test_pointer_to_numpy_short_circuits_unassociated_state_before_unsupported_policy(): +def test_pointer_to_numpy_reports_unassociated_state_before_an_unsupported_policy(): + """There is nothing to expose either way when the target is not there. + + A policy that blocks extraction still has to answer an unassociated + pointer with None rather than refusing, because no view is being withheld. + """ handle = PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "associated": lambda _handle: False, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ) assert handle.to_numpy() is None -def test_shape_short_circuits_absent_descriptor_state_before_generated_shape(): - def fail_shape(_handle): - pytest.fail("absent descriptor state must not call generated shape") +def test_shape_reports_absent_descriptor_state_without_being_asked_first(): + def fail_state(_handle): + pytest.fail("the shape inquiry reads the descriptor, which records absence itself") allocatable = AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": fail_shape, - "allocated": lambda _handle: False, - }, + **_handle_dispatch({"shape": lambda _handle: None, "allocated": fail_state}), to_numpy_policy="unsupported", ) pointer = PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": fail_shape, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "associated": fail_state, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ) @@ -71,13 +73,14 @@ def test_to_numpy_contiguous_view_policy_rejects_non_contiguous_storage(): handle = PointerArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: strided.shape, - "to_numpy": lambda _handle: strided, - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: strided.shape, + "to_numpy": lambda _handle: strided, + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="contiguous_view", ) @@ -90,13 +93,14 @@ def test_to_numpy_descriptor_view_policy_never_copies_storage(): handle = PointerArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: source.shape, - "to_numpy": lambda _handle: source, - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: source.shape, + "to_numpy": lambda _handle: source, + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="descriptor_view", ) @@ -116,14 +120,15 @@ def test_to_numpy_rejects_generated_non_numpy_results(policy: str): handle = AllocatableArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: [1.0, 2.0], - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (2,), + "to_numpy": lambda _handle: [1.0, 2.0], + "allocated": lambda _handle: True, + "deallocate": lambda _handle: None, + "resize": lambda _handle, _shape: None, + } + ), to_numpy_policy=policy, ) @@ -131,32 +136,17 @@ def test_to_numpy_rejects_generated_non_numpy_results(policy: str): handle.to_numpy() -def test_to_numpy_rejects_generated_none_for_present_descriptor_state(): - handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: None, - "allocated": lambda _handle: True, - }, - ) - - with pytest.raises(TypeError, match="returned None for present descriptor state"): - handle.to_numpy() - - def test_to_numpy_rejects_generated_array_with_wrong_rank_or_dtype(): wrong_rank = AllocatableArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: np.zeros((1, 2), dtype=np.float64), - "allocated": lambda _handle: True, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (2,), + "to_numpy": lambda _handle: np.zeros((1, 2), dtype=np.float64), + "allocated": lambda _handle: True, + } + ), ) with pytest.raises(ValueError, match="to_numpy result rank 2 does not match declared rank 1"): wrong_rank.to_numpy() @@ -164,53 +154,37 @@ def test_to_numpy_rejects_generated_array_with_wrong_rank_or_dtype(): wrong_dtype = AllocatableArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: np.zeros(2, dtype=np.int32), - "allocated": lambda _handle: True, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (2,), + "to_numpy": lambda _handle: np.zeros(2, dtype=np.int32), + "allocated": lambda _handle: True, + } + ), ) with pytest.raises(TypeError, match="to_numpy result dtype"): wrong_dtype.to_numpy() -def test_runtime_handle_shapes_reject_negative_extents_before_binding_handoff(): +def test_runtime_handle_shapes_reject_negative_extents(): handle = AllocatableArray( dtype=np.dtype(np.float64), rank=1, - ops={ - "shape": lambda _handle: (-1,), - "allocated": lambda _handle: True, - "array_actual": lambda _handle: pytest.fail("negative shape must block native handoff"), - "descriptor": lambda _handle: pytest.fail("negative shape must block descriptor handoff"), - }, + **_handle_dispatch( + { + "shape": lambda _handle: (-1,), + "allocated": lambda _handle: True, + "resize": lambda _handle, _shape: None, + } + ), to_numpy_policy="unsupported", ) with pytest.raises(ValueError, match="non-negative"): _ = handle.shape - with pytest.raises(ValueError, match="non-negative"): - _native_array_actual_for_binding(handle) - with pytest.raises(ValueError, match="non-negative"): - _native_array_descriptor_for_binding(handle, descriptor_kind="allocatable") with pytest.raises(ValueError, match="non-negative"): handle.resize(-1) - valid_handle = AllocatableArray( - dtype=np.dtype(np.float64), - rank=1, - ops={ - "array_actual": lambda _handle: _handoff(228), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - "descriptor": lambda _handle: _handoff(229), - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(ValueError, match="non-negative"): - valid_handle._descriptor_for_binding(expected_shape=(-1,)) - def test_pointer_handle_uses_common_base_and_nullify_operation(): state = _ArrayState(shape=(5,), value=np.zeros(5, dtype=np.int32)) @@ -225,7 +199,7 @@ def nullify(_handle): "nullify": nullify, "destroy": lambda _handle: None, } - handle = PointerArray(dtype="int32", rank=1, ops=ops, descriptor_ownership="owned") + handle = PointerArray(dtype="int32", rank=1, **_handle_dispatch(ops), descriptor_ownership="owned") assert isinstance(handle, NativeArrayHandleBase) assert handle.descriptor_kind == "pointer" @@ -240,37 +214,31 @@ def nullify(_handle): assert handle.to_numpy() is None -def test_pointer_associate_accepts_reassociation_and_an_unassociated_source(): +def test_pointer_associate_copies_the_target_as_it_stands_and_does_not_follow_it(): + """A pointer assignment snapshots the source's target, it does not track it.""" first_value = np.arange(3, dtype=np.float64) second_value = np.arange(4, dtype=np.float64) - destination_state = {"descriptor": _pointer_descriptor_for_array(first_value)} - source_state = {"descriptor": _pointer_descriptor_for_array(second_value)} + absent = (0, 8, 1, 0, 0, 8) def pointer(state): return PointerArray( dtype="float64", rank=1, - ops={ - "shape": lambda _handle: tuple(dimension["extent"] for dimension in state["descriptor"]["dim"]), - "array_actual": lambda _handle: _handoff(state["descriptor"]["base_addr"]), - "descriptor": lambda _handle: state["descriptor"], - "to_numpy": lambda _handle: state["descriptor"], - "associated": lambda _handle: state["descriptor"]["base_addr"] != 0, - "associate": lambda _handle, descriptor: state.update(descriptor=descriptor), - "nullify": lambda _handle: state.update( - descriptor={ - "base_addr": 0, - "elem_len": 8, - "rank": 1, - "dim": [{"lower_bound": 0, "extent": 0, "sm": 8}], - } - ), - }, + **_handle_dispatch( + { + "shape": lambda _handle: (state["facts"][4],) if state["facts"][0] else None, + "descriptor": lambda _handle: state["facts"], + "to_numpy": lambda _handle: _numpy_view_from_descriptor_facts(state["facts"], "float64"), + "associated": lambda _handle: state["facts"][0] != 0, + "associate": lambda _handle, facts: state.update(facts=facts), + "nullify": lambda _handle: state.update(facts=absent), + } + ), to_numpy_policy="descriptor_view", ) - destination = pointer(destination_state) - source = pointer(source_state) + destination = pointer({"facts": _descriptor_facts_for_array(first_value)}) + source = pointer({"facts": _descriptor_facts_for_array(second_value)}) destination.associate(source) assert destination.associated is True @@ -278,55 +246,47 @@ def pointer(state): np.testing.assert_array_equal(destination.to_numpy(), second_value) source.nullify() + assert destination.associated is True destination.associate(source) assert destination.associated is False -def test_generated_pointer_associate_packs_standard_descriptor_facts(): +def test_generated_pointer_associate_hands_over_flat_descriptor_facts(): value = np.arange(6, dtype=np.float64)[::2] - source_state = {"descriptor": _pointer_descriptor_for_array(value)} source = PointerArray( dtype="float64", rank=1, - ops={ - "shape": lambda _handle: value.shape, - "array_actual": lambda _handle: _handoff(value.ctypes.data), - "descriptor": lambda _handle: source_state["descriptor"], - "to_numpy": lambda _handle: source_state["descriptor"], - "associated": lambda _handle: True, - "associate": lambda _handle, descriptor: source_state.update(descriptor=descriptor), - "nullify": lambda _handle: None, - }, - to_numpy_policy="descriptor_view", + **_handle_dispatch( + { + "shape": lambda _handle: value.shape, + "descriptor": lambda _handle: _descriptor_facts_for_array(value), + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + "associate": lambda _handle, _facts: None, + } + ), + to_numpy_policy="unsupported", ) received = [] - destination = _native_array_handle_from_generated_ops( + operations = { + "shape": lambda: None, + "descriptor": lambda: None, + "associated": lambda: False, + "associate": lambda facts: received.append(facts), + "nullify": lambda: None, + } + destination = _native_array_handle_from_generated_dispatch( "pointer", "float64", 1, - { - "shape": lambda: None, - "array_actual": lambda: 1, - "descriptor": lambda: 1, - "associated": lambda: False, - "associate": lambda facts: received.append(facts), - "nullify": lambda: None, - }, + _generated_handle_dispatch(operations), + operations, to_numpy_policy="unsupported", ) destination.associate(source) - assert received == [ - ( - int(value.ctypes.data), - 8, - 1, - 1, - 3, - 16, - ) - ] + assert received == [(int(value.ctypes.data), 8, 1, 1, 3, 16)] @pytest.mark.parametrize( @@ -337,12 +297,13 @@ def test_generated_pointer_associate_packs_standard_descriptor_facts(): PointerArray( dtype="int32", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "associated": lambda _handle: False, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ), TypeError, @@ -352,12 +313,13 @@ def test_generated_pointer_associate_packs_standard_descriptor_facts(): PointerArray( dtype="float64", rank=2, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "associated": lambda _handle: False, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ), ValueError, @@ -369,13 +331,14 @@ def test_pointer_associate_rejects_incompatible_sources(other, error, message): destination = PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: None, - "associated": lambda _handle: False, - "associate": lambda _handle, _descriptor: None, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: None, + "associated": lambda _handle: False, + "associate": lambda _handle, _descriptor: None, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ) @@ -383,16 +346,18 @@ def test_pointer_associate_rejects_incompatible_sources(other, error, message): destination.associate(other) -def test_pointer_allocation_operations_are_policy_gated_by_ops_table(): +def test_pointer_allocation_operations_are_policy_gated_by_capabilities(): state = _ArrayState(shape=(1,), value=object()) handle = PointerArray( dtype="float64", rank=1, - ops={ - **_common_ops(state), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + **_common_ops(state), + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + } + ), ) with pytest.raises(NotImplementedError, match="pointer handle operation 'allocate' is not available"): @@ -403,7 +368,7 @@ def test_pointer_allocation_operations_are_policy_gated_by_ops_table(): handle.resize((4,)) -def test_pointer_allocation_operations_route_when_policy_ops_exist(): +def test_pointer_allocation_operations_route_when_capabilities_exist(): state = _ArrayState(shape=None, value=None) def allocate(_handle, shape): @@ -421,14 +386,16 @@ def resize(_handle, shape): handle = PointerArray( dtype="float64", rank=2, - ops={ - **_common_ops(state), - "associated": lambda _handle: state.shape is not None, - "nullify": lambda _handle: deallocate(_handle), - "allocate": allocate, - "deallocate": deallocate, - "resize": resize, - }, + **_handle_dispatch( + { + **_common_ops(state), + "associated": lambda _handle: state.shape is not None, + "nullify": lambda _handle: deallocate(_handle), + "allocate": allocate, + "deallocate": deallocate, + "resize": resize, + } + ), ) assert handle.associated is False @@ -448,12 +415,13 @@ def test_pointer_to_numpy_reports_missing_descriptor_extraction(): handle = PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2,), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (2,), + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ) @@ -465,13 +433,14 @@ def test_to_numpy_policy_unsupported_reports_completed_policy_block(): handle = PointerArray( dtype=np.dtype(np.float64), rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (2,), - "to_numpy": lambda _handle: pytest.fail("unsupported policy must not call generated extraction"), - "associated": lambda _handle: True, - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (2,), + "to_numpy": lambda _handle: pytest.fail("unsupported policy must not call generated extraction"), + "associated": lambda _handle: True, + "nullify": lambda _handle: None, + } + ), to_numpy_policy="unsupported", ) @@ -483,14 +452,15 @@ def test_common_shape_dispatch_validates_rank(): handle = AllocatableArray( dtype="float64", rank=2, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (4,), - "to_numpy": lambda _handle: None, - "allocated": lambda _handle: True, - "deallocate": lambda _handle: None, - "resize": lambda _handle, _shape: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (4,), + "to_numpy": lambda _handle: None, + "allocated": lambda _handle: True, + "deallocate": lambda _handle: None, + "resize": lambda _handle, _shape: None, + } + ), ) with pytest.raises(ValueError, match="shape rank 1 does not match declared rank 2"): @@ -502,46 +472,23 @@ def test_common_handle_rejects_invalid_descriptor_kind(): NativeArrayHandleBase( dtype="float64", rank=1, - ops={}, + invoke=lambda _operation: None, + capabilities=(), descriptor_kind="target", descriptor_ownership="borrowed", ) -def test_common_handle_rejects_invalid_generated_operation_table(): - with pytest.raises(TypeError, match="operation names must be strings"): - AllocatableArray(dtype="float64", rank=1, ops={1: lambda _handle: None}) - with pytest.raises(TypeError, match="operation 'shape' must be callable"): - AllocatableArray(dtype="float64", rank=1, ops={"shape": None}) +def test_common_handle_rejects_invalid_dispatch_contract(): + with pytest.raises(TypeError, match="dispatcher must be callable"): + AllocatableArray(dtype="float64", rank=1, invoke=None, capabilities={"shape", "allocated"}) + with pytest.raises(TypeError, match="capability names must be strings"): + AllocatableArray(dtype="float64", rank=1, invoke=lambda _operation: None, capabilities={1}) def test_common_handle_requires_generated_shape_operation(): with pytest.raises(ValueError, match="requires generated operation 'shape'"): - AllocatableArray(dtype="float64", rank=1, ops={}) - - -def test_common_handle_requires_generated_handoff_operations(): - with pytest.raises(ValueError, match="requires generated operation 'array_actual'"): - AllocatableArray( - dtype="float64", - rank=1, - ops={ - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - }, - to_numpy_policy="unsupported", - ) - with pytest.raises(ValueError, match="requires generated operation 'descriptor'"): - AllocatableArray( - dtype="float64", - rank=1, - ops={ - "array_actual": lambda _handle: _handoff(244), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - }, - to_numpy_policy="unsupported", - ) + AllocatableArray(dtype="float64", rank=1, invoke=lambda _operation: None, capabilities=()) def test_extraction_enabled_handle_requires_generated_to_numpy_operation(): @@ -549,11 +496,12 @@ def test_extraction_enabled_handle_requires_generated_to_numpy_operation(): AllocatableArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "allocated": lambda _handle: True, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "allocated": lambda _handle: True, + } + ), to_numpy_policy="borrowed_view", ) @@ -563,29 +511,43 @@ def test_pointer_handle_requires_generated_associated_and_nullify_operations(): PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "nullify": lambda _handle: None, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "nullify": lambda _handle: None, + } + ), ) with pytest.raises(ValueError, match="requires generated operation 'nullify'"): PointerArray( dtype="float64", rank=1, - ops={ - **_required_handoff_ops(), - "shape": lambda _handle: (1,), - "associated": lambda _handle: True, - }, + **_handle_dispatch( + { + "shape": lambda _handle: (1,), + "associated": lambda _handle: True, + } + ), ) def test_common_handle_rejects_invalid_descriptor_ownership(): with pytest.raises(ValueError, match="descriptor_ownership must be 'borrowed' or 'owned'"): - AllocatableArray(dtype="float64", rank=1, ops={}, descriptor_ownership="temporary") + AllocatableArray( + dtype="float64", + rank=1, + invoke=lambda _operation: None, + capabilities=(), + descriptor_ownership="temporary", + ) def test_common_handle_rejects_invalid_to_numpy_policy(): with pytest.raises(ValueError, match="to_numpy_policy must be one of"): - AllocatableArray(dtype="float64", rank=1, ops={}, to_numpy_policy="maybe_copy") + AllocatableArray( + dtype="float64", + rank=1, + invoke=lambda _operation: None, + capabilities=(), + to_numpy_policy="maybe_copy", + ) diff --git a/tests/fortran/strings/policy/test_string_wrapper_policy.py b/tests/fortran/strings/policy/test_string_wrapper_policy.py index 3a2bbbe49..eb2f46d05 100644 --- a/tests/fortran/strings/policy/test_string_wrapper_policy.py +++ b/tests/fortran/strings/policy/test_string_wrapper_policy.py @@ -325,6 +325,27 @@ def test_fixed_length_allocatable_string_update_takes_the_descriptor_result_lane assert policy.results[0].updates_argument is True +@pytest.mark.parametrize("attribute", ["allocatable", "pointer"]) +def test_fixed_length_character_array_descriptor_arguments_are_blocked(tmp_path: Path, attribute: str): + module = _semantic_module_from_text( + f""" +module fixed_array_descriptor + implicit none +contains + subroutine inspect(values) + character(len=5), {attribute}, intent(in) :: values(:) + end subroutine inspect +end module fixed_array_descriptor +""", + tmp_path, + module_name="fixed_array_descriptor", + ) + policy = module.functions[0].metadata[RESOLVED_FUNCTION_WRAPPER_POLICY_METADATA] + + assert policy.supported is False + assert any("no interoperable descriptor interface" in blocker for blocker in policy.blockers) + + def test_plain_fixed_length_string_update_keeps_copy_in_out_replacement(tmp_path: Path): """A dummy with no descriptor attribute keeps the caller-buffer replacement.