Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,24 @@
path = deps/fast_double_parser
url = https://github.com/lemire/fast_double_parser.git
branch = master
shallow = true
shallow = true
[submodule "deps/fmt"]
path = deps/fmt
url = https://github.com/fmtlib/fmt.git
branch = master
shallow = true
shallow = true
[submodule "deps/boost_math"]
path = deps/boost_math
url = https://github.com/boostorg/math.git
branch = master
shallow = true
shallow = true
[submodule "deps/eigen"]
path = deps/eigen
url = https://gitlab.com/libeigen/eigen.git
branch = 3.4
shallow = true
branch = 5.0
shallow = true
[submodule "deps/pybind11"]
path = deps/pybind11
url = https://github.com/pybind/pybind11.git
branch = v2.12
shallow = true
shallow = true
1 change: 1 addition & 0 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Authors@R:
person("Pedro", "Lima", role = "ctb"),
person("Entejar", "Alam", role = "ctb"),
person("David", "Wachsmuth", role = "ctb"),
person("Michael", "Chirico", role = "ctb"),
person("stochtree", "contributors", role = c("cph")),
person("Eigen", "contributors", role = c("cph"), comment = "C++ source uses the Eigen library for matrix operations, see inst/COPYRIGHTS"),
person("xgboost", "contributors", role = c("cph"), comment = "C++ tree code and related operations include or are inspired by code from the xgboost library, see inst/COPYRIGHTS"),
Expand Down
37 changes: 19 additions & 18 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -53,21 +53,22 @@ include deps/eigen/Eigen/SparseCore
include deps/eigen/Eigen/SparseQR
include deps/eigen/Eigen/misc
include deps/eigen/Eigen/plugins
recursive-include deps/eigen/Eigen/src/Cholesky *.h
recursive-include deps/eigen/Eigen/src/Core *.h
recursive-include deps/eigen/Eigen/src/Dense *.h
recursive-include deps/eigen/Eigen/src/Eigenvalues *.h
recursive-include deps/eigen/Eigen/src/Geometry *.h
recursive-include deps/eigen/Eigen/src/Householder *.h
recursive-include deps/eigen/Eigen/src/IterativeLinearSolvers *.h
recursive-include deps/eigen/Eigen/src/Jacobi *.h
recursive-include deps/eigen/Eigen/src/LU *.h
recursive-include deps/eigen/Eigen/src/OrderingMethods *.h
recursive-include deps/eigen/Eigen/src/QR *.h
recursive-include deps/eigen/Eigen/src/SVD *.h
recursive-include deps/eigen/Eigen/src/Sparse *.h
recursive-include deps/eigen/Eigen/src/SparseCholesky *.h
recursive-include deps/eigen/Eigen/src/SparseCore *.h
recursive-include deps/eigen/Eigen/src/SparseQR *.h
recursive-include deps/eigen/Eigen/src/misc *.h
recursive-include deps/eigen/Eigen/src/plugins *.h
include deps/eigen/Eigen/Version
recursive-include deps/eigen/Eigen/src/Cholesky *.h *.inc
recursive-include deps/eigen/Eigen/src/Core *.h *.inc
recursive-include deps/eigen/Eigen/src/Dense *.h *.inc
recursive-include deps/eigen/Eigen/src/Eigenvalues *.h *.inc
recursive-include deps/eigen/Eigen/src/Geometry *.h *.inc
recursive-include deps/eigen/Eigen/src/Householder *.h *.inc
recursive-include deps/eigen/Eigen/src/IterativeLinearSolvers *.h *.inc
recursive-include deps/eigen/Eigen/src/Jacobi *.h *.inc
recursive-include deps/eigen/Eigen/src/LU *.h *.inc
recursive-include deps/eigen/Eigen/src/OrderingMethods *.h *.inc
recursive-include deps/eigen/Eigen/src/QR *.h *.inc
recursive-include deps/eigen/Eigen/src/SVD *.h *.inc
recursive-include deps/eigen/Eigen/src/Sparse *.h *.inc
recursive-include deps/eigen/Eigen/src/SparseCholesky *.h *.inc
recursive-include deps/eigen/Eigen/src/SparseCore *.h *.inc
recursive-include deps/eigen/Eigen/src/SparseQR *.h *.inc
recursive-include deps/eigen/Eigen/src/misc *.h *.inc
recursive-include deps/eigen/Eigen/src/plugins *.h *.inc
6 changes: 6 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,17 @@
* Fix the move-type and node-selection probabilities in the MCMC grow / prune Metropolis-Hastings acceptance ratios, so that the tree sampler targets the correct posterior over trees [#426](https://github.com/StochasticTree/stochtree/pull/426)
* Fix ordering of missing (`NaN`) covariate values when presorting features for the grow-from-root (GFR) sampler, which now sorts `NaN`s last, consistent with how numeric splits route them [#427](https://github.com/StochasticTree/stochtree/pull/427)
* Fix the conditional posterior of the random effects variance components, which was missing factors of one half in its inverse gamma shape and scale [#428](https://github.com/StochasticTree/stochtree/pull/428)
* Fix an uninitialized starting value when initializing cloglog forests in R, which could crash or corrupt the outcome data for binary and ordinal models with a cloglog link [#420](https://github.com/StochasticTree/stochtree/pull/420)

Fixes in [#425](https://github.com/StochasticTree/stochtree/pull/425), [#426](https://github.com/StochasticTree/stochtree/pull/426), and [#428](https://github.com/StochasticTree/stochtree/pull/428) change sampler output, as does [#427](https://github.com/StochasticTree/stochtree/pull/427) for covariates with missing values, so results for a given random seed will differ from previous versions.

## Documentation and Other Maintenance

* Remove unused variables flagged by compiler warnings [#417](https://github.com/StochasticTree/stochtree/pull/417)
* Mark `Log::Fatal` as `[[noreturn]]`, which silences "may be used uninitialized" compiler warnings [#418](https://github.com/StochasticTree/stochtree/pull/418)
* Explicitly initialize variables in the R forest initialization and sampling code [#421](https://github.com/StochasticTree/stochtree/pull/421)
* Use shallow clones and canonical `.git` URLs for git submodules [#422](https://github.com/StochasticTree/stochtree/pull/422)
* Updated Eigen version to 5.0.1, which also fixes builds on PowerPC without VSX support [#424](https://github.com/StochasticTree/stochtree/pull/424)
* Add absolute tolerance to several python floating point comparison tests [#430](https://github.com/StochasticTree/stochtree/pull/430).
* Add exact-posterior tests of the MCMC tree sampler in R and Python, which check that a long chain on a small problem reproduces the analytically enumerated posterior over trees [#431](https://github.com/StochasticTree/stochtree/pull/431)

Expand Down
23 changes: 17 additions & 6 deletions cran-bootstrap.R
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,13 @@ if (pkgdown_build) {
if (include_tests) {
test_files_src <- list.files("test/R", recursive = TRUE, full.names = TRUE)
# Exclude fixture JSON files: large test-only snapshots, not needed on CRAN
test_files_src <- test_files_src[!grepl("/fixtures/.*\\.json$", test_files_src)]
test_files_src <- test_files_src[
!grepl("/fixtures/.*\\.json$", test_files_src)
]
# Exclude backward-compat tests: they depend on jsonlite and fixture files, not suitable for CRAN
test_files_src <- test_files_src[!grepl("test-serialization-compat\\.R$", test_files_src)]
test_files_src <- test_files_src[
!grepl("test-serialization-compat\\.R$", test_files_src)
]
test_files_dst <- file.path(cran_dir, gsub("test/R", "tests", test_files_src))
pkg_core_files <- c(pkg_core_files, test_files_src)
pkg_core_files_dst <- c(pkg_core_files_dst, test_files_dst)
Expand Down Expand Up @@ -431,7 +435,8 @@ eigen_modules <- c(
"SparseLU",
"SparseQR",
"misc",
"plugins"
"plugins",
"Version"
)
eigen_files_to_vendor_src <- c()
eigen_files_to_vendor_dst <- c()
Expand Down Expand Up @@ -515,11 +520,16 @@ if (all(file.exists(eigen_files_to_vendor_src))) {
}

# Clean up pragmas that suppress warnings in Eigen and JSON headers
# File 1: Eigen "DisableStupidWarnings" header
cran_eigen_suppress_warnings <- file.path(
# Eigen files: the "DisableStupidWarnings" header, plus the AltiVec (PowerPC)
# packet math header, which gained a GCC diagnostic block in Eigen 5
cran_eigen_pragma_files <- file.path(
cran_dir,
"src/include/Eigen/src/Core/util/DisableStupidWarnings.h"
c(
"src/include/Eigen/src/Core/util/DisableStupidWarnings.h",
"src/include/Eigen/src/Core/arch/AltiVec/PacketMath.h"
)
)
for (cran_eigen_suppress_warnings in cran_eigen_pragma_files) {
eigen_suppress_warnings_lines <- readLines(cran_eigen_suppress_warnings)
for (i in 1:length(eigen_suppress_warnings_lines)) {
line <- eigen_suppress_warnings_lines[i]
Expand Down Expand Up @@ -555,3 +565,4 @@ for (i in 1:length(eigen_suppress_warnings_lines)) {
)
}
writeLines(eigen_suppress_warnings_lines, cran_eigen_suppress_warnings)
}
2 changes: 1 addition & 1 deletion deps/eigen
Submodule eigen updated from 9df21d to bc3b39
6 changes: 3 additions & 3 deletions include/stochtree/random_effects.h
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ class MultivariateRegressionRandomEffectsModel {
group_parameters_ = group_parameters;
}
void SetGroupParameter(Eigen::VectorXd& group_parameter, int32_t group_id) {
group_parameters_(Eigen::all, group_id) = group_parameter;
group_parameters_(Eigen::placeholders::all, group_id) = group_parameter;
}
void SetWorkingParameterCovariance(Eigen::MatrixXd& working_parameter_covariance) {
working_parameter_covariance_ = working_parameter_covariance;
Expand Down Expand Up @@ -211,7 +211,7 @@ class MultivariateRegressionRandomEffectsModel {
std::int32_t group_ind;
for (int i = 0; i < n; i++) {
group_ind = tracker.CategoryNumber(group_labels[i]);
output[i] = X(i, Eigen::all) * alpha_diag * group_parameters_(Eigen::all, group_ind);
output[i] = X(i, Eigen::placeholders::all) * alpha_diag * group_parameters_(Eigen::placeholders::all, group_ind);
}
}

Expand All @@ -238,7 +238,7 @@ class MultivariateRegressionRandomEffectsModel {
std::int32_t group_ind;
for (int i = 0; i < n; i++) {
group_ind = tracker.CategoryNumber(group_labels[i]);
new_pred = X(i, Eigen::all) * alpha_diag * group_parameters_(Eigen::all, group_ind);
new_pred = X(i, Eigen::placeholders::all) * alpha_diag * group_parameters_(Eigen::placeholders::all, group_ind);
new_resid = residual.GetElement(i) - new_pred;
residual.SetElement(i, new_resid);
tracker.SetPrediction(i, new_pred);
Expand Down
1 change: 1 addition & 0 deletions man/stochtree-package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 12 additions & 12 deletions src/random_effects.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ void RandomEffectsTracker::ResetFromSample(MultivariateRegressionRandomEffectsMo
for (int i = 0; i < n; i++) {
group_ind = CategoryNumber(group_labels[i]);
prev_pred = GetPrediction(i);
new_pred = X(i, Eigen::all) * alpha_diag * xi(Eigen::all, group_ind);
new_pred = X(i, Eigen::placeholders::all) * alpha_diag * xi(Eigen::placeholders::all, group_ind);
new_resid = residual.GetElement(i) - new_pred + prev_pred;
residual.SetElement(i, new_resid);
SetPrediction(i, new_pred);
Expand Down Expand Up @@ -123,7 +123,7 @@ void MultivariateRegressionRandomEffectsModel::SampleGroupParameters(RandomEffec
for (int i = 0; i < num_groups; i++) {
posterior_mean = GroupParameterMean(dataset, residual, rfx_tracker, global_variance, i);
posterior_covariance = GroupParameterVariance(dataset, residual, rfx_tracker, global_variance, i);
group_parameters_(Eigen::all, i) = normal_sampler_.SampleEigen(posterior_mean, posterior_covariance, gen);
group_parameters_(Eigen::placeholders::all, i) = normal_sampler_.SampleEigen(posterior_mean, posterior_covariance, gen);
}
}

Expand Down Expand Up @@ -155,9 +155,9 @@ Eigen::VectorXd MultivariateRegressionRandomEffectsModel::WorkingParameterMean(R
Eigen::MatrixXd xi = group_parameters_;
for (int i = 0; i < num_groups; i++) {
observation_indices = rfx_tracker.NodeIndicesInternalIndex(i);
X_group = X(observation_indices, Eigen::all);
y_group = y(observation_indices, Eigen::all);
xi_group = xi(Eigen::all, i);
X_group = X(observation_indices, Eigen::placeholders::all);
y_group = y(observation_indices, Eigen::placeholders::all);
xi_group = xi(Eigen::placeholders::all, i);
posterior_denominator += ((xi_group).asDiagonal() * X_group.transpose() * X_group * (xi_group).asDiagonal()) / global_variance;
posterior_numerator += (xi_group).asDiagonal() * X_group.transpose() * y_group / global_variance;
}
Expand All @@ -178,9 +178,9 @@ Eigen::MatrixXd MultivariateRegressionRandomEffectsModel::WorkingParameterVarian
Eigen::MatrixXd xi = group_parameters_;
for (int i = 0; i < num_groups; i++) {
observation_indices = rfx_tracker.NodeIndicesInternalIndex(i);
X_group = X(observation_indices, Eigen::all);
y_group = y(observation_indices, Eigen::all);
xi_group = xi(Eigen::all, i);
X_group = X(observation_indices, Eigen::placeholders::all);
y_group = y(observation_indices, Eigen::placeholders::all);
xi_group = xi(Eigen::placeholders::all, i);
posterior_denominator += ((xi_group).asDiagonal() * X_group.transpose() * X_group * (xi_group).asDiagonal()) / (global_variance);
}
return posterior_denominator.inverse();
Expand All @@ -195,8 +195,8 @@ Eigen::VectorXd MultivariateRegressionRandomEffectsModel::GroupParameterMean(Ran
Eigen::MatrixXd posterior_denominator = group_parameter_covariance_.inverse();
Eigen::VectorXd posterior_numerator = Eigen::VectorXd::Zero(num_components);
std::vector<data_size_t> observation_indices = rfx_tracker.NodeIndicesInternalIndex(group_id);
Eigen::MatrixXd X_group = X(observation_indices, Eigen::all);
Eigen::VectorXd y_group = y(observation_indices, Eigen::all);
Eigen::MatrixXd X_group = X(observation_indices, Eigen::placeholders::all);
Eigen::VectorXd y_group = y(observation_indices, Eigen::placeholders::all);
posterior_denominator += ((alpha).asDiagonal() * X_group.transpose() * X_group * (alpha).asDiagonal()) / (global_variance);
posterior_numerator += (alpha).asDiagonal() * X_group.transpose() * y_group / global_variance;
return posterior_denominator.inverse() * posterior_numerator;
Expand All @@ -211,8 +211,8 @@ Eigen::MatrixXd MultivariateRegressionRandomEffectsModel::GroupParameterVariance
Eigen::MatrixXd posterior_denominator = group_parameter_covariance_.inverse();
// Eigen::VectorXd posterior_numerator = Eigen::VectorXd::Zero(num_components);
std::vector<data_size_t> observation_indices = rfx_tracker.NodeIndicesInternalIndex(group_id);
Eigen::MatrixXd X_group = X(observation_indices, Eigen::all);
// Eigen::VectorXd y_group = y(observation_indices, Eigen::all);
Eigen::MatrixXd X_group = X(observation_indices, Eigen::placeholders::all);
// Eigen::VectorXd y_group = y(observation_indices, Eigen::placeholders::all);
posterior_denominator += ((alpha).asDiagonal() * X_group.transpose() * X_group * (alpha).asDiagonal()) / (global_variance);
// posterior_numerator += (alpha).asDiagonal() * X_group.transpose() * y_group;
return posterior_denominator.inverse();
Expand Down
30 changes: 15 additions & 15 deletions test/cpp/test_random_effects.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ TEST(RandomEffects, Construction) {
Eigen::MatrixXd sigma(test_dataset.rfx_basis_cols, test_dataset.rfx_basis_cols);
alpha << 1.5;
xi << 2, 4;
Eigen::VectorXd xi0 = xi(Eigen::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::all, 1);
Eigen::VectorXd xi0 = xi(Eigen::placeholders::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::placeholders::all, 1);
sigma << 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand All @@ -148,8 +148,8 @@ TEST(RandomEffects, Construction) {
// Change values and push a second "sample" to the container
alpha << 2.0;
xi << 1, 3;
xi0 = xi(Eigen::all, 0);
xi1 = xi(Eigen::all, 1);
xi0 = xi(Eigen::placeholders::all, 0);
xi1 = xi(Eigen::placeholders::all, 1);
sigma << 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand Down Expand Up @@ -200,9 +200,9 @@ TEST(RandomEffects, Computation) {
Eigen::MatrixXd sigma(test_dataset.rfx_basis_cols, test_dataset.rfx_basis_cols);
alpha << 1., 1.;
xi << 1., 1., 1., 1., 1., 1.;
Eigen::VectorXd xi0 = xi(Eigen::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::all, 1);
Eigen::VectorXd xi2 = xi(Eigen::all, 2);
Eigen::VectorXd xi0 = xi(Eigen::placeholders::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::placeholders::all, 1);
Eigen::VectorXd xi2 = xi(Eigen::placeholders::all, 2);
sigma << 1, 0, 0, 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand Down Expand Up @@ -256,8 +256,8 @@ TEST(RandomEffects, Predict) {
Eigen::MatrixXd sigma(test_dataset.rfx_basis_cols, test_dataset.rfx_basis_cols);
alpha << 1.5;
xi << 2, 4;
Eigen::VectorXd xi0 = xi(Eigen::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::all, 1);
Eigen::VectorXd xi0 = xi(Eigen::placeholders::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::placeholders::all, 1);
sigma << 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand All @@ -270,8 +270,8 @@ TEST(RandomEffects, Predict) {
// Change values and push a second "sample" to the container
alpha << 2.0;
xi << 1, 3;
xi0 = xi(Eigen::all, 0);
xi1 = xi(Eigen::all, 1);
xi0 = xi(Eigen::placeholders::all, 0);
xi1 = xi(Eigen::placeholders::all, 1);
sigma << 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand Down Expand Up @@ -318,8 +318,8 @@ TEST(RandomEffects, Serialization) {
Eigen::MatrixXd sigma(test_dataset.rfx_basis_cols, test_dataset.rfx_basis_cols);
alpha << 1.5;
xi << 2, 4;
Eigen::VectorXd xi0 = xi(Eigen::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::all, 1);
Eigen::VectorXd xi0 = xi(Eigen::placeholders::all, 0);
Eigen::VectorXd xi1 = xi(Eigen::placeholders::all, 1);
sigma << 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand All @@ -332,8 +332,8 @@ TEST(RandomEffects, Serialization) {
// Change values and push a second "sample" to the container
alpha << 2.0;
xi << 1, 3;
xi0 = xi(Eigen::all, 0);
xi1 = xi(Eigen::all, 1);
xi0 = xi(Eigen::placeholders::all, 0);
xi1 = xi(Eigen::placeholders::all, 1);
sigma << 1;
model.SetWorkingParameter(alpha);
model.SetGroupParameter(xi0, 0);
Expand Down
Loading