Repository navigation
Count outcomes in bcf() via Pólya-Gamma augmentation - #10
Merged
Merged
Conversation
…counts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Polya-Gamma step turns the count part into the weighted Gaussian regression bcf() already fits: prognostic forest, basis-regression treatment forest (binary or continuous dose), propensity covariate, adaptive coding, tau_0, random effects, GFR and multiple chains run on the pseudo-outcome with variance weights 1 / omega. tau_0 and the coding parameters, sampled in R, use the precision weights; tau_c is a log rate ratio with a prior set by log_rate_ratio_prior_sd (default 0.5). family = 'zinb' adds probit zero-part prognostic and treatment forests on the structural-zero indicators, sharing the step of bart(). Variance forests and a sampled global variance are refused. predict.bcfmodel() gains the rate, response and zero_probability scales, the zero-part terms and an offset; computeContrastBCFModel() and sampleBCFPosteriorPredictive() handle counts; JSON carries the family, dispersion draws and zero-part forests. Gaussian and probit draws are bitwise unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recovery of a log rate ratio under confounded assignment (NB and ZINB) and of the zero-part effect, with thresholds calibrated over 16 fits; Gaussian and probit BCF draws bitwise against a reference; predict scales and terms, contrasts, posterior predictive, JSON round trips, multiple chains, warm starts, continuous dose, adaptive coding, weights and validation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dwachsmuth
marked this pull request as ready for review
November 8, 2026 23:14
This was referenced Nov 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #9 (
cma-zinb, ZINB BART), which is stacked on #8 (cma-nb). Adds count outcomes tobcf(): plain negative binomial and zero-inflated negative binomial (ZINB). This covers the BCF part ofnash_experiments/docs/nashtree-zinb/feasibility-20261108.md§1.Model
general_params = list(outcome_model = OutcomeModel("count", "log", family = "negbin" | "zinb")).y_i ~ NB(mu_i, r)withlog mu_i = o_i + c + mu_c(x_i, pi_hat_i) + [tau_0 + tau_c(x_i)] b(z_i) [+ rfx].b(z)is the treatment basis:z, or the adaptive codingb_0 (1 - z) + b_1 z.tau_0 + tau_c(x)is a log rate ratio, per unit of dose for a continuous treatment.cis the fixed centering constant, as in the NB / ZINB BART.offset_train/offset_testtake the log exposureo.zinb).pi_i = Phi(c0 + mu_z(x_i, pi_hat_i) + tau_z(x_i) z_i). That makes four forests in all.z. Both use probit calibrations with the leaf scale held fixed:2 / mfor the prognostic forest, the probit-BCFdelta_maxformula withdelta_max = 0.5for the treatment forest.zero_prognostic_forest_paramsandzero_treatment_effect_forest_params.Gibbs scheme
Each iteration runs the following steps.
s.omega_i ~ PG(y_i + r, psi_i). Then:resid_traingets the pseudo-outcome(y - r) / (2 omega) + log r - o, the role the latentzplays for probit.1 / omega. Under ZINB, structural zeros getInf.c + mu + [tau_0 + tau(x)] b(z) + rfx.tau_0and adaptive coding. These are sampled in R from unweighted sums in the existing code. The count path recomputes them with the precision weightsomega(0 for structural zeros); the Gaussian and probit code paths are untouched.r. Slice step over the non-structural rows.Refused with counts: variance forest,
sample_sigma2_global = TRUE, a multivariate treatment withzinb,observation_weightswithzinb. Withnegbin,observation_weightsmultiply the variance weights.Prior calibration
(range(log1p y) / (4 sqrt(m)))^2.treatment_effect_forest_params$log_rate_ratio_prior_sd(default 0.5).sigma2_leaf = sd^2 / num_trees, so one prior sd oftau(x)is a rate ratio of 0.61 to 1.65.tau_0:N(0, 1)by default.API
predict.bcfmodel()offsetargument."linear":y_hat = log mu,mu/prognostic_functionare log baseline rates,tau/cateare log rate ratios."rate": every term exponentiated: rates, rate ratios, multiplicative rfx."response": like"rate", excepty_hat = (1 - pi) mu."zero_probability":y_hat = piplus the zero-part terms.zero_prognostic,zero_tauandzero_forest. These are probit indices on"linear"; on the other scaleszero_prognosticandzero_forestare probabilities andzero_taustays a probit-scale coefficient.computeContrastBCFModel()takesoffset_0/offset_1and the four count scales. The linear contrast equals the CATE (tested).sampleBCFPosteriorPredictive()draws NB counts with the matchedrdraw, then zeroes each draw with probabilitypi.log_rate_ratio_prior_sd, and the zero forests (forest_2,forest_3).print,summaryandextractParameterreport counts. New outputsnegbin_dispersion_samples,zero_mu_hat_*,zero_tau_hat_*andzero_hat_*.Recovery
tests/testthat/test-bcf-count.R. Simulated data: n = 3,000, r = 3, offset.pi(x) = 0.15 + 0.7 plogis(3 (x1 - 0.5)), and x1 also drives the prognostic part.tau_c(x) = 0.3 + 0.4 x2, mean log rate ratio ≈ 0.50.tau_z = -0.5on the probit scale, 29–32% zeros.Calibration used 8 data sets per family (16 fits, about 16–17 s each):
pi(x,1) - pi(x,0)(truth −0.147): max error / coverageThe NB coverage miss is seed 2002: interval 0.504–0.618 against a truth of 0.503.
Test thresholds, applied to the first seed of each family:
Timing
n = 5,000, default forests (250 prognostic + 100 treatment trees), 100 MCMC iterations from root, median of 3 runs, single thread, ms per iteration:
Tests
Full suite,
NOT_CRAN=true:cma-zinb, after #9)Every pre-existing file has an identical count;
test-bcf-count.Radds 167 expectations. Intest-negbin.Randtest-zinb.R, the one expectation thatbcf()refuses counts now checks that it refuses a variance forest with counts.Bitwise check.
fixtures/bcf-reference.rdswas captured from the build beforebcf.Rwas edited (make-bcf-reference.R), and current draws match it bitwise. It covers:intercept_plus_treatmentrfx, 2 chains, GFR and test set, plus prediction and contrast;The #8 and #9 references still pass.
Other coverage: predict scales and terms, contrasts, the posterior predictive mean and zero mass, the 5 JSON round trips, multiple chains (GFR and root), combined JSON, warm starts, continuous dose, adaptive coding with unit and non-unit weights, and validation.
Found and left unchanged
These are pre-existing, upstream code. They are also written up in
NASH-FORK.md.tau_0. Withsample_intercept = TRUE(the default),tau_0 b(z)is left out of both the truncated-normal mean and the residual handed to the forests in the latent step. The count path includes it. Fixing probit would change its draws, so it is out of scope here.forest_model_config_mu$update_leaf_model_scale(current_leaf_scale_mu), i.e. the prognostic config with the prognostic scale. The MCMC loop is correct.tau_0and adaptive coding ignoreobservation_weights.Not verified
intercept_plus_treatmentand custom rfx bases;negbin.tau_zprior default (delta_max = 0.5) andlog_rate_ratio_prior_sd = 0.5are reasoned defaults, not tuned.num_threads > 1is untested.No GPL sources were read or copied. Everything was installed into a scratch library only.
🤖 Generated with Claude Code