Skip to content

refactor(fast): the canopy's "film" is surface water in code and docs - #59

Merged
xiangtaoxu merged 24 commits into
EDmodel:betafrom
xiangtaoxu:refactor/veg-surface-water
Oct 30, 2026
Merged

xiangtaoxu merged 24 commits into
EDmodel:betafrom
xiangtaoxu:refactor/veg-surface-water

Conversation

@xiangtaoxu

@xiangtaoxu xiangtaoxu commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What and why

A follow-up to the output stack (#55–#58), from the owner's review of #58: the water the leaves and
wood hold on their surfaces was called a "film" in the code and docs, which does not read
intuitively. Its state, config keys and outputs already said surf_water / veg_surface
(leaf_surf_water_co, leaf_surf_water_max, mean_veg_surface_water_py), so this renames the rest.
Stacked on #58: review the last two commits, 4ab572d and 6798c2b.

  • Identifiers (4ab572d):

    • frozen%film → frozen%veg_surface;
    • film_evap_leaf/_wood → leaf_surface_evap/wood_surface_evap;
    • cohort_film_water → cohort_surface_water, limit_film_evaporation → limit_surface_evaporation,
      clamp_canopy_film → clamp_surface_water, leaf_film_coeff → leaf_surface_evap_coeff;
    • g_film_leaf/g_film_w → g_surf_leaf/g_surf_wood;
    • the tests' locals and test names.

    Only code was touched, never comments.

  • Prose (6798c2b): comments, test messages, TOML comments and the science docs say "surface
    water". vegetation_energy_dynamics.md's section "The canopy's interception film" is now "The
    canopy's surface water", and every reference to it follows.

    • Kept: soil_biophysics.md's "film flow", which is soil physics.
    • Kept: example 04's recorded calibration/fit.json, which copies its text from
      parameters.toml and is rewritten by the next calibration.

No config key, state-file variable or output name changes.

Design: the owner's decision in the review of #58 (a separate refactor PR after the stack). Drafted
with Claude Code (see Credit).

Effect on results (tick exactly one; CI checks)

  • Bit-for-bit: output is identical to the base commit's in every run listed under
    Verification. A new option that is off by default counts when the model is unchanged with it off.
  • Round-off: differences at round-off level only, e.g. from reordered sums.
  • Answer-changing: results move; give the largest changes below.

User-facing changes (tick all that apply; each needs an "Upgrading" note in the changelog)

  • None
  • Config keys added, renamed or removed, or a value in the shipped config files changed
  • Output variables or output files changed
  • Input files (forcing, maps, initial conditions) or their format changed
  • Building or running changed: command-line or CMake options, dependencies, the Python API
  • State files written by the last release no longer load

The state file's two long_name attributes now read "water on the leaf surfaces" and "water on the
wood surfaces"; the variable names are unchanged, so every state file loads as before.

Verification

  • Base commit: feat(output): CF time with bounds; clearer names, labels and defaults #58's head, 1104855.
  • Runs compared with the base commit: the reference config with the shipped census, 3 months, F
    on, dev builds, ifx 2026.1 and gfortran. Bit for bit on both: every variable on F, D, M and Y, and
    the state, 95/95.
  • ctest: 98/98 on ifx and on gfortran dev builds; lint clean; ruff check clean.
  • git grep -niw "films\?": only the two kept mentions above.
  • The long tests (label long): not run.
  • Other builds: not run (production, nvfortran).

Credit and AI use

  • Collaborators:
  • AI assistance: Claude Code (Claude Opus 5.5) renamed the identifiers, reworded the prose (with a
    sub-agent for the comments and docs) and ran the verification above.
  • I answer for this pull request: I can explain every change in it, and the verification above was actually run

Not in this PR

🤖 Generated with Claude Code

xiangtaoxu and others added 22 commits October 29, 2026 17:57
…writes

The cohort order of a month's records. The stand is re-sorted every slow step
(daily growth and mortality), but the output folded cohort slabs by slot, so a
monthly cohort record averaged two cohorts in one slot after a mid-month swap,
and staged fast-loop rows were reduced with the post-sort weights. Every cohort
value of a month is now folded into its cohort's slot of the month (keyed by
global_cohort_id: the live state, the diagnostic rows and the staged rows, whose
ids and weights the fast loop stages once per step), and every record of the
month is written in the month's end order (write_order, before the I/O phase).
The axes' bookkeeping is also written on the F tier, read from the stand.

No overwrites. A run lists every diagnostic file it would write and stops before
its first step if one exists; the writer creates files with NC_NOCLOBBER.
RunConfig.remove_diagnostics() removes a configuration's files; the examples'
scripts and calibrate_fast's trials call it before they run again.

file_chunk honoured. A tier with a cohort or patch variable was cut to monthly
files (F: daily) whatever its file_chunk said. The cohort and patch axes are now
unlimited and grow within a file; each record names its cohorts and patches.
"run" is renamed "all"; a monthly chunk of a day and an annual one shorter than a
year are refused.

Model state is bit for bit (ifx and gfortran dev builds, two runs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion, dew

Four polygon outputs split ET by where the vapour comes from, as the canopy
air's vapour ledger booked it on each step:

- mean_transp_py       transpiration
- mean_evap_canopy_py  evaporation of the intercepted water (leaf and wood films)
- mean_evap_ground_py  ground evaporation: bare soil and snow pack
- mean_dew_py          condensate (dew, fog) sent to soil layer 1

ET = transp + evap_canopy + evap_ground - dew - the canopy air's vapour gain.
Default tiers D M Y, F on request (they are patch-block fast rows).

The films' evaporation is b-weighted into the step's boundary-flux amounts
(stage_bflux_t / boundary_flux_t, both integrators); close_ledgers exports the
four amounts in step_exports_t, and patch_diag_row turns them into rates.
PA_TRANSP held le_flux / L_v (ET) under a transpiration label and no output
read it; it now holds transpiration. The patch rows after it move up by three.

test_fast_integ_step: on both schemes, each step's sources less the dew less
ET equal the canopy air's net vapour inflow (worst 1.5e-16 ESDIRK2, 2.1e-16
RK45, relative), the morning rain's films evaporate, and the day's canopy and
ground evaporation agree between ESDIRK2 and RK45.

Bit for bit: example 04's calibrated BCI run on the production ifx build
matches the archived v0.4.0 run in all 24 variables both write (43,824 hourly
and 1,826 daily records). ctest 95/95 on ifx and gfortran dev builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-daily values everywhere

Every term the whole-column water and energy ledgers book is now on the patch row and an output
variable, so both budgets close from the files on any tier (docs/science/diagnostics.md §4, The
budgets): shed water, film excess, store gain and the water's paths and stores; bottom heat,
advected enthalpy, the stores' energy gains and H by source. Adds Ra, the respiration split and
Rh on the patch row, ecosystem carbon, absorbed PAR, the leaves' wet fraction and patch ET.

ET and runoff are read from the fast loop's exports (the polygon accumulators are gone), and the
formerly sub-daily-only variables get coarse sources, so every fast-loop quantity is on F and on
D/M/Y. A per-cohort row reduced to the patch or polygon is reduced on F from the staged rows.
New pytest output_budgets closes both budgets from a run's D and F files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y record

Every mean_<x> with a sub-daily value has a diel_<x>: its mean at each time of day over the
record, in bins of the new [output].diel_interval ("1 hour" to "4 hours"), on a diel axis (the
last) with the coordinate diel_hour and its bounds. Each staged sub-step is folded into its bin
of the coarse tiers' open records as the F tier reads it, before the step closes them; a cohort's
cycle goes to its slot of the month. D, M and Y (D and M on the cohort and patch axes), never F;
off by default, so the output is bit for bit. Needs the fast loop and a dt_fast that divides the
bin. MAX_OUTPUT_VARS rises to 1024 for the ~150 new names.

New pytest output_diel: a daily record is, bin by bin, its day's sub-daily records, on the
polygon, patch, soil and cohort axes, at 1- and 3-hour bins; a diel_ variable on F is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
time is now each record's start in days since the run's start (proleptic Gregorian, standard_name
time), with time_bnds: the start and the end of the record's last step, so a partial record
carries its true span and xarray and CDO decode the axis. The year/month/day companions stay.

mean_transp_leaf_py -> mean_leaf_transp_py (ecophys, off by default) and mean_transp_co ->
mean_leaf_transp_co, so the per-leaf-area transpiration has one name at every level apart from
mean_transp_py. mean_abs_par_co's long name says it is the incident-equivalent PAR;
mean_cas_co2_pa joins its polygon twin in the energy group; mean_soil_temp_top_pa is on D by
default like its siblings; diagnostics.md §7 no longer promises an albedo. ex05's plot script and
README read the new time axis. Model state is bit for bit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A run still stops before its first step when a file it would write exists, unless the new
[output].overwrite = true (default false) lets it replace them (NC_CLOBBER, no start-up check).
The examples' configs and the Python scripts set it, so rerunning an example replaces its own
files; RunConfig.remove_diagnostics() and the scripts' clean-up helpers are gone. The output_files
test reruns with overwrite on and checks every file is written again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s enthalpy split by direction

Review of EDmodel#58: "film" and "canopy" read unclearly, and "rh"/"ra" read as relative humidity.
- mean_canopy_water_py -> mean_veg_surface_water_py, mean_evap_canopy_py -> mean_veg_surface_evap_py,
  mean_film_excess_py -> mean_veg_surface_overflow_py, mean_evap_ground_py -> mean_ground_evap_py,
  mean_ra_py -> mean_raut_py; the patch rows and exports this PR added follow (PA_RHET, PA_RAUT, ...).
- mean_enth_advect_py ("advected" suggests horizontal transport) is split by direction:
  mean_precip_enth_py (the rain and snow's in) and mean_outflow_enth_py (runoff, drainage,
  overflow and the exported vapour's beyond LE, out). The budget test closes on the pair.
Model state is bit for bit with the previous head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…CO2 and pressure removed

Review of EDmodel#58:
- mean_rh_py/_pa -> mean_rhet_py/_pa (FLD_P_RH -> FLD_P_RHET): "rh" read as relative humidity.
- mean_ca_co and mean_pressure_co are removed with their rows (CO_CA, CO_PRESSURE): they repeated,
  for every cohort, the patch's canopy-air CO2 at the sub-step's start and the forcing's air
  pressure. Only edmore.canopy's docstring named them; it now points the offline leaf replay at
  mean_cas_co2_pa (the previous sub-step's) and mean_psurf_py. 29 fast cohort rows, 39 in all.
Model state is bit for bit; mean_rhet_* equals the old mean_rh_* record for record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The water the leaves and wood hold on their surfaces was called a "film" in the code; its state,
config keys and outputs already say surf_water / veg_surface. Identifiers now follow:
frozen%film -> frozen%veg_surface, film_evap_leaf/_wood -> leaf_/wood_surface_evap,
cohort_film_water -> cohort_surface_water, limit_film_evaporation -> limit_surface_evaporation,
clamp_canopy_film -> clamp_surface_water, leaf_film_coeff -> leaf_surface_evap_coeff,
g_film_leaf/_w -> g_surf_leaf/_wood, and the tests' locals and names. No config key, state-file
variable or output changes; comments and docs follow in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ter"

The prose follows the identifiers of the previous commit: the water the leaves and wood hold on
their surfaces is "surface water" throughout the comments, the tests' messages, the TOML comments
and the science docs (vegetation_energy_dynamics.md's section is now "The canopy's surface
water"). Kept: soil_biophysics.md's "film flow" (soil physics), and example 04's recorded fit.json,
which the next calibration rewrites from parameters.toml. Bit for bit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@xiangtaoxu
xiangtaoxu requested a review from a team as a code owner October 30, 2026 01:59
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@xiangtaoxu
xiangtaoxu merged commit 62e24d8 into EDmodel:beta Oct 30, 2026
4 checks passed
xiangtaoxu added a commit to xiangtaoxu/ED-MORE that referenced this pull request Oct 30, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
xiangtaoxu added a commit that referenced this pull request Oct 30, 2026
…es and residuals (#60)

* fix(output): one cohort order per month, file_chunk honoured, no overwrites

The cohort order of a month's records. The stand is re-sorted every slow step
(daily growth and mortality), but the output folded cohort slabs by slot, so a
monthly cohort record averaged two cohorts in one slot after a mid-month swap,
and staged fast-loop rows were reduced with the post-sort weights. Every cohort
value of a month is now folded into its cohort's slot of the month (keyed by
global_cohort_id: the live state, the diagnostic rows and the staged rows, whose
ids and weights the fast loop stages once per step), and every record of the
month is written in the month's end order (write_order, before the I/O phase).
The axes' bookkeeping is also written on the F tier, read from the stand.

No overwrites. A run lists every diagnostic file it would write and stops before
its first step if one exists; the writer creates files with NC_NOCLOBBER.
RunConfig.remove_diagnostics() removes a configuration's files; the examples'
scripts and calibrate_fast's trials call it before they run again.

file_chunk honoured. A tier with a cohort or patch variable was cut to monthly
files (F: daily) whatever its file_chunk said. The cohort and patch axes are now
unlimited and grow within a file; each record names its cohorts and patches.
"run" is renamed "all"; a monthly chunk of a day and an annual one shorter than a
year are refused.

Model state is bit for bit (ifx and gfortran dev builds, two runs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): cite #55

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(output): ET's sources: transpiration, canopy and ground evaporation, dew

Four polygon outputs split ET by where the vapour comes from, as the canopy
air's vapour ledger booked it on each step:

- mean_transp_py       transpiration
- mean_evap_canopy_py  evaporation of the intercepted water (leaf and wood films)
- mean_evap_ground_py  ground evaporation: bare soil and snow pack
- mean_dew_py          condensate (dew, fog) sent to soil layer 1

ET = transp + evap_canopy + evap_ground - dew - the canopy air's vapour gain.
Default tiers D M Y, F on request (they are patch-block fast rows).

The films' evaporation is b-weighted into the step's boundary-flux amounts
(stage_bflux_t / boundary_flux_t, both integrators); close_ledgers exports the
four amounts in step_exports_t, and patch_diag_row turns them into rates.
PA_TRANSP held le_flux / L_v (ET) under a transpiration label and no output
read it; it now holds transpiration. The patch rows after it move up by three.

test_fast_integ_step: on both schemes, each step's sources less the dew less
ET equal the canopy air's net vapour inflow (worst 1.5e-16 ESDIRK2, 2.1e-16
RK45, relative), the morning rain's films evaporate, and the day's canopy and
ground evaporation agree between ESDIRK2 and RK45.

Bit for bit: example 04's calibrated BCI run on the production ifx build
matches the archived v0.4.0 run in all 24 variables both write (43,824 hourly
and 1,826 daily records). ctest 95/95 on ifx and gfortran dev builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(output): the water and energy budgets close from the output; sub-daily values everywhere

Every term the whole-column water and energy ledgers book is now on the patch row and an output
variable, so both budgets close from the files on any tier (docs/science/diagnostics.md §4, The
budgets): shed water, film excess, store gain and the water's paths and stores; bottom heat,
advected enthalpy, the stores' energy gains and H by source. Adds Ra, the respiration split and
Rh on the patch row, ecosystem carbon, absorbed PAR, the leaves' wet fraction and patch ET.

ET and runoff are read from the fast loop's exports (the polygon accumulators are gone), and the
formerly sub-daily-only variables get coarse sources, so every fast-loop quantity is on F and on
D/M/Y. A per-cohort row reduced to the patch or polygon is reduced on F from the staged rows.
New pytest output_budgets closes both budgets from a run's D and F files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): the default files' new variables, an upgrading note

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): cite #56

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(output): diel_, the mean diel cycle of a daily, monthly or yearly record

Every mean_<x> with a sub-daily value has a diel_<x>: its mean at each time of day over the
record, in bins of the new [output].diel_interval ("1 hour" to "4 hours"), on a diel axis (the
last) with the coordinate diel_hour and its bounds. Each staged sub-step is folded into its bin
of the coarse tiers' open records as the F tier reads it, before the step closes them; a cohort's
cycle goes to its slot of the month. D, M and Y (D and M on the cohort and patch axes), never F;
off by default, so the output is bit for bit. Needs the fast loop and a dt_fast that divides the
bin. MAX_OUTPUT_VARS rises to 1024 for the ~150 new names.

New pytest output_diel: a daily record is, bin by bin, its day's sub-daily records, on the
polygon, patch, soil and cohort axes, at 1- and 3-hour bins; a diel_ variable on F is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): cite #57

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(output): CF time with bounds; clearer names, labels and defaults

time is now each record's start in days since the run's start (proleptic Gregorian, standard_name
time), with time_bnds: the start and the end of the record's last step, so a partial record
carries its true span and xarray and CDO decode the axis. The year/month/day companions stay.

mean_transp_leaf_py -> mean_leaf_transp_py (ecophys, off by default) and mean_transp_co ->
mean_leaf_transp_co, so the per-leaf-area transpiration has one name at every level apart from
mean_transp_py. mean_abs_par_co's long name says it is the incident-equivalent PAR;
mean_cas_co2_pa joins its polygon twin in the energy group; mean_soil_temp_top_pa is on D by
default like its siblings; diagnostics.md §7 no longer promises an albedo. ex05's plot script and
README read the new time axis. Model state is bit for bit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): cite #58

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(output): [output].overwrite, an opt-in to replace existing output

A run still stops before its first step when a file it would write exists, unless the new
[output].overwrite = true (default false) lets it replace them (NC_CLOBBER, no start-up check).
The examples' configs and the Python scripts set it, so rerunning an example replaces its own
files; RunConfig.remove_diagnostics() and the scripts' clean-up helpers are gone. The output_files
test reruns with overwrite on and checks every file is written again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(output): veg_surface, ground_evap and raut names; the water's enthalpy split by direction

Review of #58: "film" and "canopy" read unclearly, and "rh"/"ra" read as relative humidity.
- mean_canopy_water_py -> mean_veg_surface_water_py, mean_evap_canopy_py -> mean_veg_surface_evap_py,
  mean_film_excess_py -> mean_veg_surface_overflow_py, mean_evap_ground_py -> mean_ground_evap_py,
  mean_ra_py -> mean_raut_py; the patch rows and exports this PR added follow (PA_RHET, PA_RAUT, ...).
- mean_enth_advect_py ("advected" suggests horizontal transport) is split by direction:
  mean_precip_enth_py (the rain and snow's in) and mean_outflow_enth_py (runoff, drainage,
  overflow and the exported vapour's beyond LE, out). The budget test closes on the pair.
Model state is bit for bit with the previous head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(output): regenerate ed_io_config.toml after the renames

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(output): mean_rhet_*, and the cohort repeats of the patch's CO2 and pressure removed

Review of #58:
- mean_rh_py/_pa -> mean_rhet_py/_pa (FLD_P_RH -> FLD_P_RHET): "rh" read as relative humidity.
- mean_ca_co and mean_pressure_co are removed with their rows (CO_CA, CO_PRESSURE): they repeated,
  for every cohort, the patch's canopy-air CO2 at the sub-step's start and the forcing's air
  pressure. Only edmore.canopy's docstring named them; it now points the offline leaf replay at
  mean_cas_co2_pa (the previous sub-step's) and mean_psurf_py. 29 fast cohort rows, 39 in all.
Model state is bit for bit; mean_rhet_* equals the old mean_rh_* record for record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(fast): "film" identifiers become veg_surface / surface water

The water the leaves and wood hold on their surfaces was called a "film" in the code; its state,
config keys and outputs already say surf_water / veg_surface. Identifiers now follow:
frozen%film -> frozen%veg_surface, film_evap_leaf/_wood -> leaf_/wood_surface_evap,
cohort_film_water -> cohort_surface_water, limit_film_evaporation -> limit_surface_evaporation,
clamp_canopy_film -> clamp_surface_water, leaf_film_coeff -> leaf_surface_evap_coeff,
g_film_leaf/_w -> g_surf_leaf/_wood, and the tests' locals and names. No config key, state-file
variable or output changes; comments and docs follow in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(fast): "film" in comments, messages and docs becomes "surface water"

The prose follows the identifiers of the previous commit: the water the leaves and wood hold on
their surfaces is "surface water" throughout the comments, the tests' messages, the TOML comments
and the science docs (vegetation_energy_dynamics.md's section is now "The canopy's surface
water"). Kept: soil_biophysics.md's "film flow" (soil physics), and example 04's recorded fit.json,
which the next calibration rewrites from parameters.toml. Bit for bit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): cite #59

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(output): one source for the canopy-air depth; per-patch ET sources and residuals

The leftovers of #58:
- mean_cas_depth_py reads the patch row, the depth the fast loop used, as mean_cas_depth_pa does,
  so the polygon value is the patches' area-weighted sum; it joins mean_cas_depth_pa in the forcing
  group. It moves by at most 6 mm; the model is bit for bit.
- mean_transp_pa, mean_veg_surface_evap_pa, mean_ground_evap_pa, mean_dew_pa, mean_resid_energy_pa
  and mean_resid_water_pa, off by default.
mean_sw_up_py stays (calibrate_fast scores albedo with it), and example 04's composite stays
hand-built (it skips the tower's gaps, which a diel_ record cannot).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): cite #60

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@xiangtaoxu
xiangtaoxu deleted the refactor/veg-surface-water branch October 30, 2026 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant