Skip to content

Make observation_weights precision weights, as documented - #17

Merged
dwachsmuth merged 1 commit into
mainfrom
claude/observation-weights-as-precision
Nov 13, 2026
Merged

dwachsmuth merged 1 commit into
mainfrom
claude/observation-weights-as-precision

Conversation

@dwachsmuth

Copy link
Copy Markdown

Summary

bart() and bcf() document observation_weights as precision weights (y_i ~ N(mu(X_i), sigma^2 / w_i): a larger weight gives a row more influence). They did the opposite. The R layer handed w unchanged to createForestDataset(variance_weights = ), and the C++ layer reads that slot as a variance multiplier everywhere it is used: leaf sufficient statistics (sum_w += 1 / VarWeightValue), random effects (cwiseInverse()), and the global variance's posterior scale (r^2 / weights(i)). It is the same slot a variance forest scales by exp(s(x)) and a count outcome sets to 1 / omega.

This PR keeps the C++ convention and makes the R layer pass 1 / w.

Found on 12 Nov 2026 while adding noise-aware row weights to nash for NASH6 (nash_plus#100). A toy fit at ff3eb820: X carries no signal, half the rows sit at y = 1 and half at y = 0.

weights mean prediction mean sigma^2
none 0.492 0.256
100 on the y = 1 rows 0.005 (now 0.987) 0.006
4 on every row 0.495 0.064 (now 1.020)

Upstream: stochtree has the same mismatch at 737f397: R/bart.R passes variance_weights = observation_weights, and its leaf model divides by the weight. Nothing has been reported upstream; that is left to David.

Base: main (f805419), since that is where the fix has to live. nash currently pins ff3eb820. The nash draft PR that moves its pin to this branch states what that requires.

Changes

  • One conversion, at every entry point. bart() and bcf() compute observation_variance_weights <- 1 / w once, after validation. That value is used for:
    • the forest datasets (Gaussian and probit);
    • count outcomes: base variance weights, which become 1 / (w omega) after the Pólya-Gamma step;
    • BCF's R-side precision for the tau_0 and adaptive-coding updates (now w itself);
    • random effects.
      bcf_count.R only validates. No other R function takes observation_weights.
  • Random effects get the weights. A Gaussian or probit fit with random effects now passes the weights to its random-effects dataset. Before, it passed NULL, so only the forest was weighted. Count outcomes already passed theirs.
  • Zero weights. 1 / 0 = Inf, an infinite variance multiplier: the row drops out of the leaf and random-effects statistics and the residual sum of squares. The weighted GlobalHomoskedasticVarianceModel::PosteriorShape now counts only rows with a finite variance weight, so a zero-weight row is absent from the likelihood altogether. The documented "all weights zero = prior sampling" mode therefore samples the priors with num_gfr = 0; the existing validation (num_gfr > 0 refused) is unchanged. Under the old code a zero weight became a zero variance multiplier, which divides by zero. The outcome is still standardized, and default priors calibrated, over all rows without weights. The docs now say so.
  • Unchanged: the C++ variance-weight convention, the variance-forest path, and the count path's use of the slot.
  • Bookkeeping: docs in R/bart.R, R/bcf.R and their Rd files; NEWS, NASH-FORK.md; version 0.4.6.9013.

Tests

New tests in tests/testthat/test-observation-weights.R. Each fails under the old behaviour:

  • heavy weights pull a BART fit toward their rows, and small weights pull it away;
  • a constant weight c multiplies sigma^2 by c (ratio between 3.2 and 5 for c = 4);
  • random effects see the weights: two groups with opposite weight patterns over the same labels; the group intercepts differ by 0.98 with the fix and by -0.003 at ff3eb820;
  • zero-weight rows match a fit without them (mean, and sigma^2 within a factor of 1.5);
  • heavy weights pull a BCF fit toward their rows.

Fixtures and tests that passed weights in the old meaning now pass 1 / w:

  • make-outcome-reference.R, make-negbin-reference.R, make-bcf-fixes-reference.R and make-bcf-reference.R;
  • test-bcf-fixes.R, which passed variances v as weights in two tests.

For the bitwise reference fits, 1 / (1 / w) differs from w in the last bit, so the four weighted reference entries now match to about 1e-15 and are compared with tolerance = 1e-10:

  • gaussian_weighted;
  • root_weighted (NB);
  • probit_weighted_intercept_adaptive;
  • gaussian_dose_weighted, which was already skipped.

Every unweighted reference entry is still compared bitwise and passes.

Full suite (macOS arm64, R 4.6, installed package, NOT_CRAN=true): 209 tests, 2159 expectations, 0 failures, 2 skipped (the suite's own slow-test skips).

Identity: seeded unweighted fits at ff3eb820 and at this commit are bitwise identical in every draw. That covers BART, BART with a random intercept, probit BART, importance retention, and BCF with a random intercept: train and test predictions, sigma^2, leaf scales, random effects, and posterior predictive draws. The saved JSON differs only in stochtree_version.

Not verified

  • Probit with weights: semantics are unchanged apart from the inversion. The probit latent step was not reviewed for weights.
  • Multi-threaded fits: all checks ran at one thread.
  • Other platforms: the bitwise reference fixtures only run on the platform that generated them.

Draft: do not merge before the nash side is reviewed.

🤖 Generated with Claude Code

bart() and bcf() documented observation_weights as y_i ~ N(mu, sigma^2 / w_i)
but handed w unchanged to the C++ datasets, whose variance-weight slot is a
variance multiplier wherever it is read. A larger weight therefore gave a row
less influence, and a constant weight c divided sigma^2 by c. Upstream
stochtree (737f397) has the same mismatch.

The R layer now converts once (1 / w) and uses that for every forest and
random-effects dataset, for count outcomes (1 / (w omega)) and for BCF's R-side
precision. Gaussian and probit random effects now receive the weights too. The
weighted PosteriorShape counts only rows with a finite variance weight, so a
zero weight removes a row from the likelihood. Unweighted fits are bitwise
unchanged; weighted reference fixtures pass 1 / w and match to rounding.

Co-Authored-By: Claude <noreply@anthropic.com>
@dwachsmuth
dwachsmuth marked this pull request as ready for review November 13, 2026 08:05
@dwachsmuth
dwachsmuth merged commit f348480 into main Nov 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant