Skip to content

Fix GFR treatment leaf scale and weighted tau_0 / adaptive coding in bcf() - #12

Merged
dwachsmuth merged 4 commits into
cma-rfxfrom
claude/bcf-fixes
Nov 9, 2026
Merged

dwachsmuth merged 4 commits into
cma-rfxfrom
claude/bcf-fixes

Conversation

@dwachsmuth

Copy link
Copy Markdown

Fixes two upstream bugs in R/bcf.R found while adding count outcomes. No C++ changes. NASH fits (Gaussian, no observation_weights, no adaptive coding, treatment leaf scale not sampled) go through none of the changed paths. The test gaussian_intercept below checks this bitwise.

Base: current cma-rfx (84f5ccb). #8, #9 and #10 were already merged into it, so this branch is on top of the count code and the count PRs need no rebase. The count path's weighted tau_0 / adaptive-coding sums now go through the same helpers, with the same arithmetic.

Bug A: GFR updated the wrong forest's leaf scale

With treatment_effect_forest_params = list(sample_sigma2_leaf = TRUE), each GFR iteration drew a treatment-forest leaf scale. It then called forest_model_config_mu$update_leaf_model_scale(current_leaf_scale_mu), so the treatment forest kept its initial scale for every GFR sweep. The MCMC loop was already correct. Fix: update forest_model_config_tau with current_leaf_scale_tau.

Test: createForestModelConfig() is traced to record every update_leaf_model_scale() call. With num_gfr = 4, keep_gfr = TRUE and 3 MCMC draws, the treatment config receives exactly c(gfr draws, last gfr draw (chain reset), mcmc draws) of sigma2_leaf_tau_samples, and holds each value afterwards. The prognostic config gets the same for sigma2_leaf_mu_samples. On the pre-fix build this test fails.

Bug B: Gaussian tau_0 and adaptive coding ignored observation_weights

The C++ leaf statistics read observation_weights as variance multipliers v_i (sum_w += 1/VarWeightValue). The R-side tau_0 update and the b_0/b_1 update used unweighted Z'r, Z'Z, s_tt, s_ty. The two halves of the Gibbs sampler therefore conditioned on different likelihoods.

Fix: use precisions w_i = 1/v_i: Z'W r, Z'W Z, and weighted s_tt / s_ty. The posterior is N(Σ Z'Wr/σ², Σ) with Σ = (Z'WZ/σ² + I/τ²)⁻¹. The adaptive-coding prior (+ 2σ²) is unchanged; only the data terms are weighted. The sums live in two small internal helpers, bcf_tau_0_posterior() and bcf_coding_suff_stats(), shared by the GFR and MCMC loops and the count path. current_precision is NULL without weights or for probit, which keeps the old unweighted expressions.

Bitwise evidence

fixtures/make-bcf-fixes-reference.R captured seeded draws on 84f5ccb before either fix (commit 1). test-bcf-fixes.R compares current draws against them.

  • Identical (expect_identical on y_hat, mu_hat, tau_hat train and test, sigma², both leaf scales, tau_0, b_0, b_1, rfx, sigma²(x)):
    • Gaussian with sample_intercept = TRUE (default)
    • Gaussian with adaptive_coding = TRUE (no intercept)
    • Gaussian with both
    • Gaussian with both, plus rfx and 2 chains
    • Gaussian with a variance forest and adaptive coding
    • MCMC-only (num_gfr = 0) with the treatment leaf scale sampled
    • Weighted probit with intercept and adaptive coding
  • Differ:
    • GFR with the treatment leaf scale sampled
    • Weighted Gaussian with intercept, with adaptive coding, and with both
  • The existing count-PR Gaussian/probit reference (bcf-reference.rds) still matches bitwise, except for gaussian_dose_weighted. That fit carries observation_weights, so its draws change on purpose and it is now excluded from the bitwise comparison.

Recovery evidence

Unit checks:

  • bcf_tau_0_posterior() matches the closed form for p = 1, and for p = 2 it matches the Bayesian WLS solved as OLS on rows scaled by 1/sqrt(v σ²) with prior pseudo-rows.
  • bcf_coding_suff_stats() matches direct weighted sums.
  • Unit weights reproduce the NULL path bitwise.

Wiring check: both helpers are traced during a weighted adaptive-coding fit. On every GFR and MCMC iteration, the precision passed is exactly 1/observation_weights. The conditional mean used for tau_0 equals the exact WLS conditional mean given that iteration's forests, Σ b r/v / (Σ b²/v + σ²/τ²). Unweighted Gaussian and weighted probit fits pass NULL.

End-to-end study (scratch script, not committed): n = 800, 20% of rows with 400× noise variance passed as weights, constant effect 1, 10 seeds, 10 GFR + 100 burn-in + 200 draws.

before after
posterior-mean tau_0 error, no adaptive coding (max / RMSE) 67.6 / 36.2 0.72 / 0.40
ATE error, no adaptive coding (max / RMSE) 0.087 / 0.045 0.080 / 0.042
ATE error, adaptive coding (max / RMSE) 0.75 / 0.37 0.080 / 0.048

Before the fix, the weighted forest offset the wandering tau_0, so the ATE looked fine without adaptive coding. With adaptive coding it did not. With adaptive coding, tau_0 alone is not identified (only tau_0 · (b_1 − b_0) is), so the table reports the ATE there.

A lighter committed version (n = 500, one seed) asserts |mean tau_0 − 1| < 1. The value is 0.44 after the fix and 38.7 before. It also asserts the ATE is within 0.25.

Test suite

NOT_CRAN=true, installed into a scratch library:

  • Before (84f5ccb): FAIL 0 | WARN 13 | SKIP 2 | PASS 1841
  • After: FAIL 0 | WARN 13 | SKIP 2 | PASS 1900

Per file, the only changes are:

  • new test-bcf-fixes.R (60 expectations)
  • test-bcf-count.R: 167 → 166, from dropping the gaussian_dose_weighted comparison

Known and not fixed

These are also listed in NASH-FORK.md under "Known upstream bugs":

  • Probit BCF with sample_intercept = TRUE omits tau_0·b(z) from the latent step's truncated-normal mean and from the residual handed to the forests. Fixing it changes probit draws, so it is deliberately left for a separate decision.
  • Probit tau_0 and adaptive coding still ignore observation_weights. The probit latent step draws z with unit variance whatever the weights.
  • The R-side tau_0 and adaptive-coding updates ignore a variance forest's per-row variances.
  • Any zero observation_weights make Gaussian bcf() return NaN draws, because of the C++ variance-multiplier convention. The input checks still describe all-zero weights as "prior sampling mode", and the roxygen text describes weights as precisions. Both are pre-existing.

🤖 Generated with Claude Code

dwachsmuth and others added 4 commits November 8, 2026 18:26
…g fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With treatment_effect_forest_params$sample_sigma2_leaf = TRUE, each GFR
iteration drew a new treatment-forest leaf scale but then handed the
prognostic scale to the prognostic forest's config, so the treatment forest
kept its initial scale for every GFR sweep. Update the treatment config with
the treatment draw, as the MCMC loop already does. Fits that do not sample
the treatment leaf scale, or that run no GFR iterations, are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The C++ leaf statistics read observation_weights as variance multipliers
v_i (sum_w += 1 / v_i), but bcf() sampled tau_0 and the adaptive-coding
parameters in R from unweighted sums, so with weights the two halves of the
Gibbs sampler conditioned on different likelihoods and tau_0 drifted far from
the effect while the weighted forest offset it. Use precisions 1 / v_i in
Z'W r, Z'W Z and the s_tt / s_ty sums, through two helpers the count path
now shares (bcf_tau_0_posterior, bcf_coding_suff_stats). Without weights,
and for probit, the unweighted arithmetic is kept and draws are bitwise
unchanged. The count tests' weighted Gaussian reference is no longer
compared bitwise, since its draws change on purpose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bugs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dwachsmuth
dwachsmuth marked this pull request as ready for review November 9, 2026 00:10
@dwachsmuth
dwachsmuth merged commit 2eab02b into cma-rfx Nov 9, 2026
2 checks passed
@dwachsmuth
dwachsmuth deleted the claude/bcf-fixes branch November 9, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant