Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
f7e65e5
Expose non-target module arrays as views and interoperable logicals
saidctb Sep 2, 2026
bbee891
Borrow the runtime descriptor for an allocatable actual
saidctb Sep 3, 2026
92af0d2
Publish a native entry-point table for module array handles
saidctb Sep 3, 2026
c3b272d
Take the published entry-point table when placing a descriptor argument
saidctb Sep 3, 2026
bde7120
Publish entry-point tables for field and owned result handles
saidctb Sep 3, 2026
3cc23e4
Detach a borrowed descriptor from the capsule that carried it
saidctb Sep 3, 2026
ab2b6b7
Refuse a PROTECTED module array before generating its accessors
saidctb Sep 3, 2026
24a01f9
Let a descriptor consumer define the module array it receives
saidctb Sep 3, 2026
248ff1e
Call inside the descriptor consumer for a writable allocatable dummy
saidctb Sep 4, 2026
d742fec
Place every descriptor argument inside its consumer
saidctb Sep 4, 2026
941b901
Remove the machinery for handing a copied descriptor to a binding
saidctb Sep 4, 2026
c15013f
Reach a pointer dummy through its consumer, as an allocatable one is
saidctb Sep 4, 2026
ff8d32d
Place a present optional descriptor argument like any other
saidctb Sep 4, 2026
a73b08b
Delete the machinery for rebuilding a descriptor in C
saidctb Sep 4, 2026
97ff88f
Publish an entry-point table for wrapper-owned descriptor storage
saidctb Sep 4, 2026
f6d29bf
Build an allocatable handle's NumPy view where its descriptor is
saidctb Sep 4, 2026
eb48fda
Read an array handle's storage from its table for an ordinary dummy
saidctb Sep 4, 2026
7ec432e
Normalize an empty dimension when a handle reaches an ordinary dummy
saidctb Sep 4, 2026
00e9bf6
Answer an owned handle's inquiries from the descriptor it holds
saidctb Sep 4, 2026
16d92b0
Reach an array handle's storage from C for more dummy forms
saidctb Sep 4, 2026
c19ade5
Report a refused array handle from the binding, and cover characters
saidctb Sep 4, 2026
79e7652
Publish one versioned backend for every array handle
saidctb Sep 5, 2026
abc7b3a
codex: Answer every handle inquiry from the descriptor it already has
saidctb Sep 5, 2026
b76447a
codex: Delete the lowering the descriptor route replaced, and cover t…
saidctb Sep 5, 2026
e10f5f2
Dispatch every array handle through one backend and one call
saidctb Sep 5, 2026
a2cf473
Refuse a borrowed descriptor a call cannot hold open
saidctb Sep 5, 2026
9d02d1b
Make the call inside the descriptors, always
saidctb Sep 5, 2026
3479b32
codex: Keep native array documentation current
saidctb Sep 5, 2026
3e586a4
Say which storage an owned handle operation needs
saidctb Sep 5, 2026
127e3c4
Record the capsule ABI break for extension builders
saidctb Sep 5, 2026
daa8f29
codex: Complete native array backend audit
saidctb Sep 5, 2026
b9e2d3f
Prove the two things the capsule name and an absent optional promise
saidctb Sep 5, 2026
7feba1c
Hold the record still while the version name does
saidctb Sep 5, 2026
e7a3a52
Name the capsule after the record it carries
saidctb Sep 5, 2026
ab6e467
Drop the version number the tag made redundant
saidctb Sep 5, 2026
72c1031
Describe the capsule name the code actually publishes
saidctb Sep 5, 2026
7a812cb
codex: Version native array backend semantics
saidctb Sep 6, 2026
78ef2cd
codex: Classify how each array dummy is reached, and say why a layout…
saidctb Sep 6, 2026
3ab611f
codex: Describe a NumPy array to a Fortran descriptor dummy
saidctb Sep 6, 2026
75717c9
codex: Format the descriptor constructor
saidctb Sep 6, 2026
6e431b5
codex: Reach a descriptor dummy from a handle and from NumPy alike
saidctb Sep 6, 2026
2aa5423
codex: Hand every descriptor-capable array over as a descriptor
saidctb Sep 6, 2026
ebad154
codex: Cover every source and shape a signed-stride handoff accepts
saidctb Sep 6, 2026
20d4f74
codex: Complete and harden descriptor array handoff
saidctb Sep 6, 2026
795717c
codex: Take a fixed-width character array by the only dummy that can …
saidctb Sep 6, 2026
9b46574
codex: Distinguish native entities from descriptor projections
saidctb Sep 6, 2026
22a2c81
codex: Take a default-kind LOGICAL array at its real width in the LAP…
saidctb Sep 6, 2026
d76bc7a
update docs
saidctb Sep 6, 2026
a7b491a
codex: Unify character-array handles on Fortran-owned storage
saidctb Sep 8, 2026
12a9d68
codex: complete signed stride array handling
saidctb Sep 8, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ compilation should use the focused owners under
`tests/fortran/infrastructure/building/compiling/` as applicable. Include the
relevant end-to-end feature tests whenever a generated or compiled mechanism
changes; run a broader suite when behavior spans multiple stages.
Run pytest with at most `-n 2`. Never `-n 4`, `-n 8`, or `-n auto`. The development machine has 12 cores but only about 7 GB of RAM, and every xdist worker loads NumPy while the Fortran end-to-end tests fork gfortran and cc per test on top of `pytest-monitor` profiling each one. Higher parallelism exhausts memory and thrashes swap, which has hard-frozen the machine and forced a reboot. Prefer the narrowest owning test path over a full suite run, and commit verified work promptly rather than batching it behind a long run.
Do not run LAPACK wrapper tests locally unless the user explicitly asks for them. Local verification may run everything else, including BLAS-only real-library tests; leave LAPACK coverage to GitHub Actions by default.
Do not run the full coverage workflow for routine changes. Run focused tests plus the required static-analysis suite. Reserve the complete CI-style coverage workflow for explicit pre-merge or pull-request verification, or when the user specifically requests it.
When investigating coverage failures, mirror the GitHub Actions workflow before deciding the fix: run coverage with `COVERAGE_PROCESS_START=pyproject.toml`, combine parallel data with `python3 -m coverage combine`, then run `python3 -m coverage report`. Do not assume a plain local coverage run matches CI, especially when subprocess tests are involved.
Expand Down
61 changes: 61 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,67 @@ release tags add a leading `v` to the package version.

## Unreleased

- **Breaking (native ABI):** the native array handle capsule is now
`prik.native_array_backend.v2`. Extensions built with an earlier PRIK publish
an incompatible capsule, so rebuild extensions that exchange handles.

- Allocatable and pointer character-array arguments now accept matching
caller-created handles. Deferred-length allocation and resizing use
`element_length=...`; deferred-length pointer handles require ifx or GNU
Fortran 14 or newer and do not expose `to_numpy()`.

- Returned character-array handles use Fortran-owned storage, including
deferred-length allocatables, and can be reused in matching calls. Output
arguments transfer their allocation directly; function results use normal
Fortran assignment semantics without an additional C-side data copy.

- Fixed Fortran allocatable and pointer descriptor arguments to use the
compiler's live descriptor. This works on Intel ifx as well as GNU Fortran,
preserves lower bounds, allocation and association changes, and covers
required and optional dummies, multiple descriptor dummies per call,
`intent(out)` and status-bearing calls, and handles from module variables,
fields, results, and caller-created storage.

- Descriptor-backed native calls honor `@nogil` while the native procedure is
running, including calls with multiple allocatable or pointer arguments.

- Generated Fortran allocatable and pointer handles can be passed directly to
matching ordinary array arguments. Numeric assumed-shape and assumed-rank
arguments accept representable forward or reversed Fortran sections from
either handles or NumPy arrays, including direct `bind(C)` procedures.
Optional arrays apply the same layout rules when present and accept omission
or `None` as absence. Explicit-shape and assumed-size arrays retain their
declared layout. C array arguments accept NumPy arrays.

- Fixed- and assumed-width character array arguments accept strided views,
forward or reversed, from NumPy arrays and from generated handles, and write
back through them. Previously a strided character view reached native code as
the whole buffer it was cut from, and a reversed one was refused.

- Array-valued functions whose result extents depend on descriptor arguments
return initialized NumPy arrays on Intel ifx and GNU Fortran.

- Descriptor-backed NumPy views preserve native byte strides, including
negative strides, non-contiguous pointer targets, and zero-sized dimensions.

- Writable module and derived-field allocatable arrays support allocation and
reassignment. PRIK rejects `protected` module arrays when the generated API
would require writable access.

- Module array views cover fixed, target, allocatable, pointer, shifted-bound,
character, and logical storage. Ordinary fixed-shape module arrays expose
live NumPy views with or without `target`.

- PRIK now selects interoperable Fortran logical storage on Intel and
PGI/NVIDIA compilers. Use `--no-standard-logicals` or
`standard_logicals=False` only when linking Intel objects compiled without
that option. Wider logical arrays are exposed with their matching integer
dtype, including caller-created allocatable and pointer handles;
`logical(c_bool)` remains `numpy.bool_`. A wider logical array argument is
now aliased rather than converted, so it takes that integer dtype where a
one-byte `numpy.bool_` buffer was previously copied in and out. Logical
scalars are unaffected and stay Python `bool` in every kind.

## 0.4.3 — 2026-08-31

- Republishes 0.4.2. That tag carried the previous package version, so the
Expand Down
183 changes: 183 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

PRIK (Python Runtime Interop Kit) generates native Python bindings from Fortran
projects: importable CPython extensions plus editable `.pyi` contracts for
reshaping the generated Python API. The active codebase is entirely Python
(`prik/`); Fortran/C fixture and source files (`*.f90`, `*.f95`, `*.for`, `*.c`,
`*.h`) are inputs/outputs of the tool, not implementation — don't spend
analysis effort on them unless explicitly asked to.

This repo also has an `AGENTS.md` with detailed, load-bearing contributor
policy (test philosophy, changelog rules, policy-completion boundary, QA
gating). Read it — the summary below pulls out what matters most day to day,
but AGENTS.md is authoritative for edge cases.

## Commands

Install with dev/QA extras:

```bash
python3 -m pip install -e ".[qa]"
```

Run the full suite:

```bash
PYTHONPATH=. python3 -m pytest -q
```

Run a single test / focused owner (always prefer the narrowest owning path over the full suite):

```bash
python3 -m pytest -q path/to/tests
python3 -m pytest -q path/to/test_file.py::test_name
```

The five independently-collecting suite roots:

```bash
python3 -m pytest -q tests/fortran -m "not real_library"
python3 -m pytest -q tests/c
python3 -m pytest -q tests/docs
python3 -m pytest -q tests/tools
python3 -m pytest -q tests/workflows
```

For documentation-only changes, run only:

```bash
python3 -m pytest -q tests/docs
git diff --check
```

Static analysis suite (blocking for any code/test/build/tooling change; run before pushing):

```bash
python3 -m ruff check .
python3 -m ruff format --check .
python3 tools/check_static_analysis_versions.py
python3 tools/check_codegen_complexity.py # advisory
python3 -m bandit -c pyproject.toml -r prik --severity-level medium --confidence-level medium
python3 -m vulture
python3 tools/check_radon_policy.py --base-ref auto # blocking (changed code)
python3 -m radon cc prik -n C -s --total-average # advisory, still run
python3 -m radon mi prik -s # advisory, still run
```

Enable the tracked pre-push hook once per clone (`git config core.hooksPath .githooks`) — it runs the static-analysis gate plus focused doc/wrapper/tools/workflows smoke tests.

Full CI-style coverage (only for explicit pre-merge/PR verification or investigating a coverage failure — mirror this exactly, a plain local `coverage run` does not match CI):

```bash
COVERAGE_PROCESS_START=pyproject.toml PYTHONPATH=. python3 -m coverage run -m pytest -q --randomly-seed=1
python3 -m coverage combine
python3 -m coverage report # fail_under = 90
```

Run pytest with at most `-n 2` — never `-n 4`, `-n 8`, or `-n auto`. This
machine has 12 cores but only ~7 GB of RAM; each xdist worker loads NumPy
while the Fortran end-to-end tests fork gfortran and cc per test, so higher
parallelism exhausts memory, thrashes swap, and has hard-frozen the machine.
Prefer the narrowest owning test path, and commit verified work promptly
rather than batching it behind a long run.

Don't run LAPACK wrapper tests locally unless explicitly asked (leave to GitHub Actions); ordinary local runs also exclude `real_library` generally. `examples/blas`, `examples/lapack`, `examples/fftpack`, `examples/minpack` are full-library correctness projects with their own workflows.

Alternate-compiler toolchain lane (when changing compiler portability/native generation):

```bash
python3 tools/run_fortran_toolchain_lane.py --compiler=/path/to/compiler
```

## Architecture

PRIK is a strict pipeline: source facts flow forward through owned stages, and
**meaning moves forward only** — a downstream stage implements an upstream
decision, it never reinterprets or overrides it.

```
preprocessing/ -> parsers/ -> semantics/ -> policy/ -> planning/ -> codegen/ -> printers/ -> compiler/ -> runtime/
```

| Package | Owns |
| --- | --- |
| `preprocessing/` | Source prep, provenance, includes, compiler-derived target/type probes |
| `parsers/` | Syntax facts only (`fortran/`, `pyi/` — a deferred `c/` frontend exists but isn't the published contributor architecture) |
| `semantics/` | Language-neutral semantic IR (`SemanticModule`); shared meaning, not a Python API or emitted code |
| `policy/` | **Every** interoperability decision: object kind, ownership, transfer, destruction, mutability/writeback, nullability, output projection, release responsibility, storage mode (stack/heap/alias), getter/setter behavior, support |
| `planning/` | Projects policy-complete IR into a deterministic, backend-neutral `ModulePlan` (`WrapperPlanner.build()`) — orders/names/validates, invents nothing |
| `codegen/` | Dispatches the plan into named C-binding and Fortran-bridge lowering mechanisms; backend scalar projection |
| `printers/` | Serializes formed nodes to C/Fortran/`.pyi` text — no behavior decisions |
| `compiler/` | Compiler commands, compile objects, native-support install, linking |
| `runtime/` | Python runtime objects + bundled native support used by generated extensions |
| `pipeline/` | End-to-end build orchestration (`build.py`, `wrapper.py`, `pyi.py`) tying the stages together |
| `contracts/` | Public names usable in semantic `.pyi` contracts (deliberately public — its import path is part of `.pyi` syntax) |
| `naming/` | Shared public-name and generated-symbol policy |
| `utilities/` | Stage-neutral helpers only (parsing/normalization/rendering/evaluation/visitor) |

Two input routes converge at `SemanticModule` and share everything after it:
Fortran source (`preprocessing` → `parsers/fortran` → `semantics/fortran2ir.py`)
and semantic `.pyi` contracts (`parsers/pyi` → `semantics/pyi2ir.py`).

**The hard boundary is before `WrapperPlanner.build()`.** By that point policy
must be fully decided. Binding/bridge generators (`codegen/`) must never infer
or override policy from datatype, Fortran `intent`, alias shape, storage
layout, or a local memory check, and must not add a silent fallback — if a
decision is missing, that's a bug in `policy/`, not something to patch around
in codegen. When changing behavior, prefer expressing it in completed policy
or the shared wrapper plan; touch binding/bridge lowering only when the plan
already requires a genuinely new emitted-code mechanism.

**Deciding binding vs. bridge, or any ABI question:** ask how it would work for
a `bind(C)` procedure, where there is no bridge. A direct entrypoint has only
the binding and the user's C ABI symbol, so whatever the direct route must do
is binding-owned; the bridge owns exactly the remainder that makes an ordinary
non-`bind(C)` procedure reachable through the same completed plan. When a form
*cannot* be `bind(C)` at all — e.g. a deferred-length `character(len=:)` dummy,
which the standard rejects there because `bind(C)` character dummies must have
length 1 — that proves a generated Fortran adapter is mandatory and names what
it must construct. See AGENTS.md for the full rule.

Array declaration expressions specifically cross packages in a fixed order:
`utilities/declaration_expressions.py` (parse/normalize text) → `semantics/`
(record native callable provenance) → `policy/` (complete support) →
`codegen/` (consume the completed plan only).

Root entry points: `prik.__init__` exposes `build_fortran_extension`,
`build_pyi_extension`, and `__version__` only; `prik/cli.py` is the `python3 -m prik`
dispatcher into the same stage owners. Deeper docs: `docs/developer/architecture.md`,
`docs/developer/codebase-map.md`, `docs/developer/packages/*.md`.

## Test tree

Tests mirror the pipeline and are organized `tests/<language>/<documented-feature>/<owning-stage>/`
(stage names: `parsing/`, `probes/`, `preprocessing/`, `semantics/`, `policy/`,
`codegen/`, `printers/`, `compiling/`, `pipeline/`, `runtime/`, `end_to_end/`).
Give each test one primary invariant, placed at the earliest stage that can
prove it; add `end_to_end/` only when generation/compilation/import/runtime
behavior contributes a distinct claim. Genuinely internal (non-public-behavior)
mechanisms live under `tests/fortran/infrastructure/<production-package>/`,
mirroring the production module. See `tests/README.md` for the full stage
table and markers (`fortran_end_to_end`, `real_library`, `toolchain_smoke`, `property`, `regression`, `slow`, `benchmark`).

## Working conventions (see AGENTS.md for full detail)

- Update `CHANGELOG.md` under **Unreleased** for any user/maintainer-visible
change (public APIs, features, examples, CI/build workflow, benchmark
methodology, documented limitations). Skip it for invisible internal cleanup.
- When asked to move/change an API, import path, command, or behavior, remove
the old path — do not add compatibility shims, aliases, or fallbacks unless
explicitly asked to keep them.
- Tests are evidence for a named invariant (observable behavior, a public API,
a documented diagnostic/serialized format, ABI/ownership/lifetime/build
behavior, a stage boundary, or tooling-consumed structure) — not a freeze on
prose, private names, file inventories, or incidental layout. Remove tests
that only pin removed behavior; don't add tests that only prevent refactors.
- After finishing an implementation task, summarize which pipeline stages
actually changed (parsing / semantic IR / policy / planning / codegen /
bridge / compilation / docs) and what changed there, plus the tests
touched and how they were verified.
6 changes: 6 additions & 0 deletions docs/developer/packages/codegen/c-binding.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@ shared entrypoint records.
holders, or module proxies. `CBindingNames` keeps its private C symbols aligned
with the binding helpers. Public names still come from the plan.

Array lowering follows the ABI completed by policy and carried by the plan.
Numeric assumed-shape and assumed-rank arguments use descriptors;
explicit-shape, assumed-size, raw C-pointer, and character arguments use the
address ABI. A bridge adapts non-`bind(C)` procedures to the same entrypoint
contract used by direct calls.

## Run A Minimal Manual Plan

This is the smallest complete plan: a public Python `ping()` that calls the
Expand Down
Loading
Loading