Skip to content
41 changes: 41 additions & 0 deletions NASH-FORK.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,3 +256,44 @@ Pre-existing in upstream code and not fixed here:
- `observation_weights` of zero are infinite precision under the C++
convention: any zero weight makes Gaussian `bcf()` return `NaN` draws,
although the input checks describe all-zero weights as prior sampling.

## Random intercept on the ZINB zero part (0.4.6.9011)

Branch `claude/zinb-zero-rfx`, on `cma-bcf-count` (d8a0cb7b), merged with
`cma-rfx` 2eab02bb (the #12 fixes). With
`rfx_group_ids_train` and `family = "zinb"`, the zero part of `bart()` and
`bcf()` gets its own random effects: `pi_i = Phi(c0 + g(x_i) [+ tau_z(x_i) z_i]
+ w_i' v_k(i))` on the count part's groups, basis and `random_effects_params`
priors, with separate parameters `v`. Default on;
`general_params$zero_random_effects = FALSE` gives the previous model draw for draw.

**Scheme**: a second `RandomEffectsModel` / `RandomEffectsDataset` (no
variance weights) / `RandomEffectsTracker` / `RandomEffectSamples`, sampled
right after the zero forest(s) on the zero part's latent residual
`z - c0 - forests - w'v` with `global_variance = 1`, the way probit `bart()`
samples its rfx. `v` enters the structural-zero draw and the truncated-normal
mean. Chain resets, warm starts and GFR deletion mirror the count rfx. Output:
`zero_rfx_samples`, `zero_rfx_preds_train/test`; `zero_hat_*` are now the full
index. `predict()` term `"zero_rfx"` (probit scale on every scale); `y_hat` on
the response / zero-probability scales includes `v` and needs group ids. JSON:
flag `has_zero_rfx` and `random_effect_container_1`. Unseen groups follow
`unseen_groups` in `bart()`; `bcf()` refuses them on both parts (no policy
there, unchanged).

**Recovery** (60 tracts x 30 rows, intercept SD 0.4 count / 0.7 zero, r = 5,
8 seeds): `bart()` pi correlation 0.882-0.927 with vs 0.535-0.677 without,
response RMSE ratio <= 0.834, intercept correlation 0.870-0.924. `bcf()` with
treatment correlated 0.35-0.49 with the zero intercept: zero-part effect error
0.076-0.296 with vs 0.556-0.824 without; the remainder is leakage through the
partially pooled intercepts (error 0.01-0.2 when treatment is unconfounded).

**Mixing** (`bcf()`, same panel, 4 chains x 1,000 draws, 3 seeds, GFR and root
starts): split R-hat of the response-scale ATE <= 1.008 with vs <= 1.005
without; bulk ESS 468-660 vs 520-851 of 4,000. Zero-part effect: R-hat <= 1.035
vs <= 1.025, ESS 133-192 vs 173-243. **Timing** (n = 5,000, 100 tracts, default
trees, ms per iteration): `bart()` ZINB 32.2 -> 32.6; `bcf()` ZINB 69-71 ->
68.5-68.8 (within run-to-run noise).

The seeded references `tests/testthat/fixtures/zinb-rfx-reference.rds` pin
ZINB without group ids, ZINB with the switch off (`bart()` and `bcf()`) and NB
with group ids (`bart()` and `bcf()`) to d8a0cb7b bitwise.
18 changes: 18 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,23 @@
# nashtree 0.4.6.9011

* The zero part of zero-inflated count models (`family = "zinb"`) in `bart()`
and `bcf()` gets random effects of its own whenever `rfx_group_ids_train` is
given: `pi = Phi(c0 + g(X) [+ tau_z(X) Z] + w' v_group)`, on the count part's
groups, basis (`model_spec`) and `random_effects_params` priors but with
separate parameters, sampled after the zero forest(s) on the zero part's
probit latent residual with unit variance. The intercept enters `pi`
everywhere: the structural-zero draw, the latent step, `zero_hat_train` /
`zero_hat_test` (now the full probit index), `predict()` (new term
`"zero_rfx"`, on the probit scale; `y_hat` on the `"response"` and
`"zero_probability"` scales now needs `rfx_group_ids`), the posterior
predictive, contrasts, and JSON (a second random-effects container). Unseen
groups follow `random_effects_params$unseen_groups` in `bart()`; `bcf()`
refuses them on both parts, as before. New outputs `zero_rfx_samples`,
`zero_rfx_preds_train`, `zero_rfx_preds_test`.
`general_params$zero_random_effects = FALSE` turns it off and reproduces the
previous draws bitwise; models without group ids, negative-binomial models
and saved models without the field are unchanged.

* `bcf()` fits count outcomes: `OutcomeModel("count", "log")` (negative
binomial) and `OutcomeModel("count", "log", family = "zinb")`. The count part
is `log(mu) = offset + c + mu(X, pi_hat) + [tau_0 + tau(X)] b(Z) [+ rfx]`, so
Expand Down
Loading
Loading