diff --git a/API_STABILITY.md b/API_STABILITY.md index eafa88b..33c0c60 100644 --- a/API_STABILITY.md +++ b/API_STABILITY.md @@ -107,6 +107,13 @@ If you depend on `masque` before 1.0, set a version floor in `DESCRIPTION` (`Imports: masque (>= 0.14.0)`) and record the exact version with `renv` (`renv::snapshot()`). +- `0.17.0`: new exports `control_spec()`, `controls()`, `control_table()`, + `masque_plan()`, `clone_plan()`, `save_plan()`, `read_plan()`, `handover()`. + `mask(conditional = )` also takes `"design"` or column names, and `roles` + also takes a plan. **A site's coordinate displacement changes under the same + seed**, because the draw is now keyed on the site's value, so a rebuild is a + new release of those sites. No signature breaks. + ## 1.0 and after From `1.0.0`, `masque` adopts a **frozen API**: diff --git a/CITATION.cff b/CITATION.cff index 4c222eb..afcd24c 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -20,7 +20,7 @@ authors: repository-code: "https://github.com/max578/masque" url: "https://max578.github.io/masque/" license: MIT -version: 0.16.0 +version: 0.17.0 keywords: - synthetic-data - data-masking diff --git a/DESCRIPTION b/DESCRIPTION index 67a301c..44bfd08 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,6 +1,6 @@ Package: masque Title: Structurally Faithful Development Surrogates for Tabular Data -Version: 0.16.0 +Version: 0.17.0 Authors@R: c( person("Max", "Moldovan", email = "max.moldovan@gmail.com", @@ -29,7 +29,7 @@ Depends: Imports: cli, data.table (>= 1.15.0), - digest, + digest (>= 0.6.21), Matrix, S7, stats, @@ -42,6 +42,7 @@ Suggests: ggplot2, knitr, maps, + openssl, readxl, rmarkdown, spelling, diff --git a/NAMESPACE b/NAMESPACE index 16104aa..f5f8df3 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -5,15 +5,22 @@ S3method(print,masque_conformance) export(apply_recipe) export(audit_mask) export(clean_table) +export(clone_plan) export(conform_table) +export(control_spec) +export(control_table) +export(controls) export(data_card) export(detect_design) +export(handover) export(jitter_coordinates) export(mask) export(mask_set) export(masque) +export(masque_plan) export(plot_design_summary) export(propose_roles) +export(read_plan) export(read_recipe) export(read_set) export(recipe) @@ -21,6 +28,7 @@ export(reveal_maps) export(role_options) export(roles) export(roles_validate) +export(save_plan) export(save_recipe) export(set_role) export(synthesise_geospatial) diff --git a/NEWS.md b/NEWS.md index c31d076..750ff95 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,81 @@ +# masque 0.17.0 + +## Behaviour changes + +* The site displacement of `jitter_coordinates()` and `mask(coords = )` is now + keyed on the site's value and the seed, so one seed gives a site one + displacement even after rows are shuffled, added or removed. In 0.16.0 the + advice to reuse the seed held only for a byte-identical table. **A clone built + by an earlier version moves under the new draw, so rebuilding it counts as a + second release of its sites: retire the old clone.** +* The rebuild table in `?jitter_coordinates` is corrected: 14.6, 9.4, 6.3, 4.4 + and 3.1 km at 1, 2, 4, 8 and 16 kept rebuilds, and seven rebuilds, not eight, + fall below the 5 km floor. +* Rows are one site when their coordinates agree to nine decimal places, so a + table written to a file with 15 significant digits and read back keeps its draw + at nearly every site. +* The audit grades a re-simulated number that tracks the original row by row: + HIGH at r of 0.9 or more on the rows of small levels, with the means of large + terms taken out. MEDIUM at 0.7 or more there, or at 0.95 or more overall, which + big environment means alone can reach. +* Under the hierarchy ladder a design level must hold three rows to carry a + shift. A level of two rows gave a shift close to those rows' own values: a clone + correlated 0.99 with the original at its rows, and the audit said low. Conditional + clones with such a column change. Levels of three or four rows still carry a + shift, since real alpha blocks are that small, and the audit grades them. +* A private `seed` in `controls()` no longer appears in the manifest. With a + plan and its recipe, the owner's permutations now come from SHA-256 of a + 256-bit key from the system's random source, because a derived integer seed + could be searched against a per-replicate statistic. A private integer `seed` + keeps its 31 bits. Release nothing computed from a single replicate, and check + the permutation p-value as the result of the test on the real table. +* `apply_recipe()` and `controls()` with a recipe now refuse a table whose values + differ from those the recipe was built on, not only its NA mask. A table read + back from a file at 15 digits still matches, as do its column types. +* In a plan, the default control unit passes over a column with too few units for + a 5 percent level. It warns when several columns qualify + (`masque_control_unit_ambiguous`): name the unit `on` was assigned at. +* New warnings mark units and strata with too few arrangements for a 5 percent + level (`masque_control_unit_small`) and a whole-valued target whose rounding + changes the declared shape (`masque_control_shape_rounded`). + `masque_control_strata_weak` now fires when `on` is constant in over 10 percent + of the strata (was 50). +* `handover(allow_high = TRUE)` names the overridden findings in `README.txt`. + The refusal text now says what each writer records: only `masque()` writes to + the recipe. +* The averaged-row figure in `?jitter_coordinates` is corrected to 0.64 km for + 360 rows. +* masque needs digest 0.6.21 or later. +* The range statement in `vignette("confidentiality")` is corrected. A whole + number in a collaborate clone can fall one full step outside the observed + range, not half a step as 0.16.0 stated, so a column whose minimum is 0 can + show -1. A conditional clone can go further. + +## New features + +* `masque_plan()` holds the roles and every named spec in one object. + `mask(df, plan)` reads them from it, an argument passed as well overrides the + plan, and each control spec is resolved on the source and written back to the + recipe. `save_plan()` and `read_plan()` keep a plan as YAML. +* `clone_plan()` gives the plan in clone vocabulary. `controls(clone, plan)` runs + the declared controls for the analyst. `controls(real, plan, recipe = rec)` + runs them for the owner: cleaning and conformance are replayed, a table with a + different schema or NA mask is refused, a spec changed since `mask()` is refused + and the seed is derived privately from the recipe. +* `handover(m, dir)` writes the clone, the clone plan, the data card and a note + to one folder after the release gate. It never writes the recipe. +* `mask(conditional = )` takes `"design"`, which conditions on the retained + design columns alone: the clone keeps design means and block variance and + carries no treatment effect. It also takes column names, an explicit + partition that runs through the same ladder. +* Known-truth controls. `control_spec()` declares what a control permutes and + plants; `controls()` derives a null, `n` negative and `n` positive control + datasets from any table with a role table, by restricted permutation at + the declared unit within declared design strata and a per-level shift in + target-SD units; `control_table()` regenerates any of them from its seed. + The same spec runs on the clone and, with a private seed, on the + original. Vignette "Known-truth controls". + # masque 0.16.0 ## Behaviour changes diff --git a/R/audit.R b/R/audit.R index d2eb710..92011eb 100644 --- a/R/audit.R +++ b/R/audit.R @@ -89,6 +89,12 @@ #' (`grain = FALSE`, or a collaborate grain with fewer than five units) carries #' a note saying the clone does not keep that grain. #' +#' The grade of a re-simulated number compares rows. A shift on a design level +#' of three or four rows, with within-level noise as large as the signal, can +#' grade low while the correlation of the level means stays near 0.85. A low +#' class means the rows are not copies. It does not mean the level means are +#' hidden. +#' #' As a reference point, a collaborate-mode clone of `agridat::besag.met` #' (yield as the outcome, design columns kept) scores `low` on every #' column, and none of its yields equals the original value. diff --git a/R/check_integrity.R b/R/check_integrity.R index 9028ac0..2604eca 100644 --- a/R/check_integrity.R +++ b/R/check_integrity.R @@ -1,5 +1,5 @@ -#' Internal: verify the recipe's recorded NA-mask fingerprint against -#' the candidate `original` data frame. +#' Internal: verify the recipe's recorded NA-mask and value fingerprints +#' against the candidate `original` data frame. #' #' Called from [apply_recipe()] when `check_integrity = TRUE` (the #' default). Aborts with guidance if the fingerprint does not match. @@ -32,7 +32,78 @@ "changed since the recipe was built), pass ", "{.code check_integrity = FALSE} to {.fun apply_recipe}." ) - )) + ), class = c("masque_integrity_refusal", "orchestra_refusal")) + } + recorded <- .prop_or(rec, "content_fp", list()) + if (length(recorded) && !.content_matches(recorded, .content_fingerprint(original))) { + cli::cli_abort(c( + paste0( + "Recipe integrity check failed: the values of {.arg original} ", + "do not match the recipe's recorded fingerprint." + ), + i = "The recipe was built against a table with the same missingness and different values.", + "*" = paste0( + "If this is intentional, pass {.code check_integrity = FALSE} to ", + "{.fun apply_recipe}. {.fun controls} has no such switch: it runs on ", + "the table the clone was built on." + ) + ), class = c("masque_integrity_refusal", "orchestra_refusal")) } invisible(TRUE) } + +# What the recipe records of the values: two weighted sums per numeric column +# and a digest of the rest. The tolerance covers a file at 15 digits. The sum of +# logs guards small values beside a very large one. +.content_fingerprint <- function(df) { + cols <- sort(names(df), method = "radix") + canon <- lapply(cols, function(cl) .content_canonical(df[[cl]])) + names(canon) <- cols + numeric_col <- vapply(canon, is.numeric, logical(1L)) + list( + sums = lapply(canon[numeric_col], function(x) { + ok <- is.finite(x) + w <- sin(seq_along(x) * sqrt(2)) + list( + value = sum(x[ok] * w[ok]), + scale = sum(abs(x[ok] * w[ok])), + log_value = sum(sign(x[ok]) * log1p(abs(x[ok])) * w[ok]), + log_scale = sum(log1p(abs(x[ok])) * abs(w[ok])), + other = digest::digest(list(which(!ok & !is.na(x)), x[!ok & !is.na(x)]), algo = "sha256") + ) + }), + text = digest::digest(canon[!numeric_col], algo = "sha256") + ) +} + +# One form for a column whatever a file reader made of it: a number, where every +# value reads as one, and otherwise text. Logical values count as 0 and 1. +.content_canonical <- function(x) { + if (is.numeric(x) && !is.object(x) || inherits(x, "integer64")) { + return(as.double(x)) + } + if (is.logical(x)) { + return(as.double(x)) + } + # c() drops the growable bit data.table may set, which serialize() would hash. + text <- c(enc2utf8(as.character(x))) + if (all(text[!is.na(text)] %in% c("TRUE", "FALSE"))) { + return(as.double(text == "TRUE")) + } + num <- suppressWarnings(as.numeric(text)) + if (any(is.na(num) & !is.na(text))) text else num +} + +.content_matches <- function(recorded, actual) { + if (!identical(recorded$text, actual$text) || + !setequal(names(recorded$sums), names(actual$sums))) { + return(FALSE) + } + all(vapply(names(recorded$sums), function(cl) { + a <- recorded$sums[[cl]] + b <- actual$sums[[cl]] + identical(a$other, b$other) && + abs(a$value - b$value) <= 1e-14 * max(a$scale, b$scale) && + abs(a$log_value - b$log_value) <= 1e-14 * max(a$log_scale, b$log_scale) + }, logical(1L))) +} diff --git a/R/control_spec.R b/R/control_spec.R new file mode 100644 index 0000000..af94102 --- /dev/null +++ b/R/control_spec.R @@ -0,0 +1,296 @@ +# control_spec.R -- control_spec(): the declaration behind controls(). + +#' Internal: the masque_control_spec S7 class +#' +#' A container for one validated declaration; constructed by +#' [control_spec()] only. +#' +#' @keywords internal +#' @noRd +masque_control_spec <- S7::new_class( + "masque_control_spec", + properties = list( + on = S7::new_property( + class = S7::new_union(S7::class_character, NULL), default = NULL + ), + target = S7::new_property( + class = S7::new_union(S7::class_character, NULL), default = NULL + ), + # A named numeric vector of level shifts in outcome-SD units, or + # `list(sd = )` until the spec meets a table and the vector is drawn. + effect = S7::new_property( + class = S7::new_union(S7::class_list, S7::class_numeric, NULL), + default = NULL + ), + strata = S7::new_property( + class = S7::class_character, default = character() + ), + unit = S7::new_property(class = S7::class_character, default = character()), + grain = S7::new_property(class = S7::class_list, default = list()), + n = S7::new_property(class = S7::class_integer, default = 1L), + seed = S7::class_integer + ) +) + +#' Declare the controls to derive from a table +#' +#' A control spec says what [controls()] permutes and plants: the column +#' the negative control reassigns (`on`), the outcome the +#' positive control plants an effect on (`target`), the planted effect per +#' level of `on` in outcome-SD units (`effect`), the design strata within +#' which `on` is permuted and the unit it is permuted at, the other +#' unit-grained columns, the number of replicates and the seed. The spec is +#' declared once, before any control table exists, and is the record of +#' what was planted; print it to read the declaration. +#' +#' The null control needs nothing beyond the seed. The negative control +#' needs `on`, which defaults to the single treatment column of the table. +#' The positive control needs `target` and `effect` together. +#' +#' @param on The name of the column the negative control reassigns, by +#' permuting it within `strata` at its `unit`. The outcome is not changed, +#' so any real effect of `on` stays in it as unexplained variation. Must be +#' a treatment or covariate column, categorical or +#' whole-valued. `NULL` (the default) means the single treatment column of +#' the table; a table with several or no treatment columns needs `on` +#' named. +#' @param target The name of one or more numeric outcome columns the +#' positive control plants the effect on, or `NULL` for no positive +#' control. A logical, factor, date, event or two-valued column is +#' refused when the spec meets the table. +#' @param effect The planted effect per level of `on`, in units of the +#' target's standard deviation in the table at hand. That standard deviation +#' includes the design variance and any real effect of `on`, so the size is +#' not a multiple of an analysis's residual. Give it as a named numeric +#' vector (`c(G01 = 0.5, G02 = -0.5)`; a level not named shifts by zero), +#' or `list(sd = s)` to draw one shift per level from a normal +#' distribution with standard deviation `s`, under the spec's seed, when +#' the spec first meets a table. Required with `target`. On a whole-valued +#' target the shift is rounded, which can change the ratios between levels; +#' `shift_realised` in `@truth` holds the effect that is in the table, and a +#' warning (class `masque_control_shape_rounded`) marks a changed shape. +#' @param strata Design columns within which `on` is permuted, or `NULL` +#' for no stratification. Every unit of `on` must lie within one stratum. +#' Prefer complete replicates to incomplete blocks; see [controls()]. +#' @param unit Design columns within which `on` is constant. `on` is then +#' permuted across those units, so name the unit `on` was assigned at. +#' `NULL` (the default) permutes rows, and in a plan picks the coarsest design +#' column on which `on` is constant and which has enough units for a 5 percent +#' level, with a warning (class `masque_control_unit_ambiguous`) when several +#' qualify. +#' @param grain A named list giving, for other analysed columns constant +#' within a design unit, the unit columns: `list(rain = "trial")`. Such a +#' column is permuted across its units in the null control, so it stays +#' constant within each unit. +#' @param n The number of negative (and positive) replicates. +#' @param seed The seed of the control draws. Required. +#' +#' @returns A `masque_control_spec` object, with a print method. +#' +#' @examples +#' spec <- control_spec( +#' on = "gen", target = "yield", effect = c(G01 = 0.5, G02 = -0.5), +#' strata = "rep", n = 19L, seed = 2026L +#' ) +#' spec +#' +#' @seealso [controls()], [control_table()]. +#' +#' @export +control_spec <- function( + on = NULL, + target = NULL, + effect = NULL, + strata = NULL, + unit = NULL, + grain = NULL, + n = 1L, + seed = NULL +) { + .check_control_seed(seed) + on <- .check_control_names(on, "on", single = TRUE) + target <- .check_control_names(target, "target", single = FALSE) + effect <- .check_control_effect(effect, target) + strata <- .check_control_cols(strata, "strata") + unit <- .check_control_cols(unit, "unit") + grain <- .check_control_grain(grain) + n <- .check_control_n(n) + masque_control_spec( + on = on, + target = target, + effect = effect, + strata = strata, + unit = unit, + grain = grain, + n = n, + seed = as.integer(seed) + ) +} + +# Refusal helper ------------------------------------------------------------- + +# A classed refusal: `masque_control__refusal` under the shared marker. +# The message is interpolated in the caller's frame, where its values live. +.control_refuse <- function(msg, what, .envir = parent.frame()) { + cli::cli_abort( + msg, + class = c(paste0("masque_control_", what, "_refusal"), "orchestra_refusal"), + .envir = .envir + ) +} + +# Argument checks ------------------------------------------------------------ + +# The seed is required: a control without a seed cannot be declared. +.check_control_seed <- function(seed) { + if (is.null(seed)) { + .control_refuse( + c( + "`seed` is required for a control spec.", + i = "A control is reproducible from its declaration, so the seed is part of it." + ), + "seed" + ) + } + .check_seed(seed) +} + +# `NULL`, or a character vector of column names, single when `single`. +.check_control_names <- function(x, arg, single) { + if (is.null(x)) { + return(NULL) + } + ok <- is.character(x) && length(x) >= 1L && !anyNA(x) && all(nzchar(x)) && + !anyDuplicated(x) && (!single || length(x) == 1L) + if (!ok) { + what <- if (single) "a single column name" else "one or more column names" + .control_refuse("`{arg}` must be {what}, or `NULL`.", arg) + } + x +} + +# `NULL` or a character vector of design column names. +.check_control_cols <- function(x, arg) { + if (!length(x)) { + return(character()) + } + .check_control_names(x, arg, single = FALSE) +} + +# `effect` and `target` come together: a named numeric vector of finite +# level shifts, or `list(sd = s)` with a single positive `s`. +.check_control_effect <- function(effect, target) { + if (is.null(effect) && is.null(target)) { + return(NULL) + } + if (is.null(effect) || is.null(target)) { + .control_refuse( + "`target` and `effect` are declared together: a positive control needs both.", + "effect" + ) + } + if (is.numeric(effect)) { + nms <- names(effect) + ok <- length(effect) >= 1L && !is.null(nms) && all(nzchar(nms)) && + !anyDuplicated(nms) && all(is.finite(effect)) + if (!ok) { + .control_refuse( + "A numeric `effect` must be a named vector of finite level shifts, one name per level of `on`.", + "effect" + ) + } + return(stats::setNames(as.double(effect), nms)) + } + if (is.list(effect)) { + ok <- identical(names(effect), "sd") && is.numeric(effect$sd) && + length(effect$sd) == 1L && is.finite(effect$sd) && effect$sd > 0 + if (ok) { + return(list(sd = as.double(effect$sd))) + } + } + .control_refuse( + "`effect` must be a named numeric vector of level shifts or `list(sd = s)` with a single positive `s`.", + "effect" + ) +} + +# `NULL` or a named list of character vectors: column -> its unit columns. +.check_control_grain <- function(grain) { + if (is.null(grain)) { + return(list()) + } + nms <- names(grain) + ok <- is.list(grain) && length(grain) >= 1L && !is.null(nms) && + all(nzchar(nms)) && !anyDuplicated(nms) && + all(vapply(grain, function(u) { + is.character(u) && length(u) >= 1L && !anyNA(u) && all(nzchar(u)) + }, logical(1L))) + if (!ok) { + .control_refuse( + "`grain` must be a named list of unit column names, as in `list(rain = \"trial\")`.", + "grain" + ) + } + grain +} + +# A single whole number of replicates, at least one. +.check_control_n <- function(n) { + ok <- is.numeric(n) && length(n) == 1L && !is.na(n) && is.finite(n) && + n == round(n) && n >= 1L + if (!ok) { + .control_refuse("`n` must be a single whole number of replicates, at least 1.", "n") + } + as.integer(n) +} + +# Print ---------------------------------------------------------------------- + +# local(): see the plot method in detect_plot.R. +local(S7::method(print, masque_control_spec) <- function(x, ...) { + cli::cli_h1("masque_control_spec") + cli::cli_bullets(c( + "*" = sprintf("Permuted column (on): %s", .spec_on_line(x)), + "*" = sprintf("Planted-effect target: %s", .spec_target_line(x)), + "*" = sprintf("Effect (target-SD units): %s", .spec_effect_line(x)), + "*" = sprintf("Strata: %s", .spec_cols_line(x@strata, "none")), + "*" = sprintf("Unit of on: %s", .spec_cols_line(x@unit, "row")), + "*" = sprintf("Unit-grained columns: %s", .spec_grain_line(x)), + "*" = sprintf("Replicates: %d", x@n), + "*" = "Seed: present" + )) + invisible(x) +}) + +.spec_on_line <- function(x) { + if (is.null(x@on)) "the treatment column of the table" else x@on +} + +.spec_target_line <- function(x) { + if (is.null(x@target)) "none" else paste(x@target, collapse = ", ") +} + +.spec_effect_line <- function(x) { + if (is.null(x@effect)) { + return("none (no positive control)") + } + if (is.list(x@effect)) { + return(sprintf( + "one shift per level drawn from N(0, %s^2) when the spec meets a table", + format(x@effect$sd) + )) + } + paste(sprintf("%s %+.3g", names(x@effect), x@effect), collapse = ", ") +} + +.spec_cols_line <- function(cols, empty) { + if (length(cols)) paste(cols, collapse = " + ") else empty +} + +.spec_grain_line <- function(x) { + if (!length(x@grain)) { + return("none declared") + } + units <- vapply(x@grain, paste, character(1L), collapse = " + ") + paste(sprintf("%s within %s", names(x@grain), units), collapse = "; ") +} diff --git a/R/controls.R b/R/controls.R new file mode 100644 index 0000000..3ea8c6e --- /dev/null +++ b/R/controls.R @@ -0,0 +1,868 @@ +# controls.R -- controls() and control_table(): null, negative and positive +# control datasets derived from a table by restricted permutation and planted +# effects. Nothing is re-simulated. + +#' Internal: the masque_controls S7 class +#' +#' The resolved declaration plus everything needed to regenerate a control +#' table: the owned table, the validated roles, the seeds, the analysed +#' columns, the event pairs, the per-row stratum and unit ids, the unit ids +#' of grained columns, the manifest, the truth table and the notes. +#' +#' @keywords internal +#' @noRd +masque_controls <- S7::new_class( + "masque_controls", + properties = list( + table = S7::class_data.frame, + roles = S7::class_data.frame, + spec = masque_control_spec, + seeds = S7::class_integer, + # The owner's digest key; when set, `seeds` is NA and no integer is used. + key = S7::new_property(class = S7::class_character, default = character()), + analysed = S7::class_character, + events = S7::class_list, + strata_id = S7::class_integer, + unit_id = S7::class_integer, + grain_units = S7::class_list, + manifest = S7::class_data.frame, + truth = S7::class_data.frame, + notes = S7::class_character + ) +) + +# Offset that roots the planted-effect draw away from the permutation stream. +.CONTROL_SALT <- 7919L + +#' Derive null, negative and positive controls from a table +#' +#' Builds the control datasets a [control_spec()] declares, from any table +#' with a role table: a synthetic clone, a comparator's clone, or, on the +#' owner's side, the original. Nothing is re-simulated. Three kinds of +#' control exist, each with a known truth: +#' +#' * `null`: every analysed column (outcome, covariate and date columns +#' with action `scramble` or `alias`) is permuted independently over its +#' non-missing rows; a column declared unit-grained is permuted across its +#' units and stays constant within each. Design, treatment and kept +#' columns, the NA mask and every column's multiset of values are kept. +#' No association among analysed columns, or with the treatment, remains. +#' Use it as a smoke test: a pipeline that finds signal here builds that +#' signal itself. A leak already in the table, such as a covariate that +#' copies the outcome, is shuffled away with the rest and passes. +#' * `negative`: the declared column `on` alone is permuted within the +#' declared design strata, at its unit, while the design structure and all +#' other associations are kept. The outcome is not changed, so any real +#' effect of `on` stays in it as unexplained variation. Choose strata that +#' every level of `on` can reach. Complete replicates qualify. Within incomplete blocks each block +#' keeps its set of levels, so part of the real effect of `on` survives. +#' With `n` replicates it gives a permutation null for a whole pipeline: a +#' false-positive rate, a permutation p-value. +#' * `positive`: the same permutation as the negative replicate, plus the +#' declared shift per level of `on` on each target, in units of the +#' target's standard deviation in the table at hand. Positive minus +#' negative is the planted truth, exactly. +#' +#' @section Declaration before data: +#' The spec is resolved against the table here: `on` defaults to the single +#' treatment column; strata and unit are checked for nesting; an `effect` +#' given as `list(sd = )` is drawn under the spec's seed, once, so the same +#' spec plants the same vector on every table it meets. The resolved spec +#' and the truth table are on the returned object and in its print. +#' +#' @section Two sides, two seeds: +#' An analyst who holds only the clone runs this on the clone. The owner +#' runs it on the original, offline, on a private draw (the planted vector +#' still comes from the spec's seed). A public seed would let anyone who knows +#' the design, which the clone shares, rebuild every permutation and read the +#' original outcome out of per-replicate statistics. +#' +#' With a plan and its `recipe`, the permutations come from SHA-256 of a +#' 256-bit key the recipe holds, so there is no seed to search. A private +#' integer `seed` is the weaker route: it has at most 31 bits, and anyone given +#' a per-replicate statistic can try each value, because the true seed fits the +#' statistic exactly and a wrong one does not. +#' +#' Control tables built from an original never leave the owner. What may leave +#' goes through the owner's usual output checking, classed by what it +#' estimates. The false-positive rate says little about the real outcome. The +#' rate at which the positives find the shift depends on the residual variance. +#' The permutation p-value is the result of the analyst's test on the real +#' table. A vector of per-replicate means or contrasts is not an aggregate, +#' under either route, and every further pipeline run on the original is +#' another set of queries. +#' +#' @section Limits: +#' A kept column is structure and is never permuted, so leakage through +#' kept columns, ids or row order passes the null. No control can detect +#' confounding. A planted effect is clean: uniform within a level, with no +#' confounding and no interaction. +#' Nothing is clipped, so a shift can take a non-negative outcome below +#' zero; [control_table()] warns when it does. The null collapses the +#' design means of row-grained outcomes, so false-positive calibration +#' comes from the negative control, not the null. +#' +#' A negative control is a test of the stated size for a comparison that uses +#' only contrasts within the strata, so the analysis needs the strata in its +#' model. Permuting within replicates and fitting no replicate term puts the +#' replicate variance in the residual and rejects far less often than the +#' nominal level. A shared rate on the clone and the original shows that the two +#' agree, and a rate of zero on both says nothing about calibration. A warning +#' (class `masque_control_unit_small`) marks units and strata with too few +#' arrangements for a 5 percent level, and another (class +#' `masque_control_strata_weak`) marks strata that hold one level of `on`. +#' +#' @param df The table to derive controls from: a data frame or data.table. +#' @param roles Its role table, as returned by [roles()] for a clone or by +#' [propose_roles()] and [set_role()] for any table. Or a plan: the analyst +#' passes [clone_plan()] with the clone, the owner passes the plan and the +#' `recipe` with the real table. +#' @param spec A [control_spec()]. With a plan in `roles`, the name of one of +#' its specs, or `NULL` when it holds exactly one. +#' @param events Linked time/status pairs among the outcome columns, in the +#' form [mask()] takes. The null permutes each pair as one block. `NULL` +#' when there are none. +#' @param seed A private integer seed for the permutations that overrides the +#' spec's seed, for the owner's side; `NULL` uses the spec's seed, or with a +#' `recipe` the recipe's key. A value equal to the spec's seed is refused +#' there. See "Two sides, two seeds" for what an integer can and cannot hide. +#' @param recipe With a plan in `roles`, the recipe of the clone. `df` is then +#' the real table: its cleaning and conformance are replayed as in +#' [apply_recipe()], a table with a different schema, NA mask or values is +#' refused, +#' and a spec that differs from the one fixed at [mask()] is refused. +#' +#' @returns A `masque_controls` object: the resolved spec (`@spec`), the +#' manifest (`@manifest`, one row per dataset), the truth table +#' (`@truth`, one row per target and level of `on`) and the notes +#' (`@notes`). Datasets are produced by [control_table()]. +#' +#' @examples +#' trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +#' r <- propose_roles(trial, detect = FALSE) +#' r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +#' r <- set_role(r, "gen", role = "treatment", action = "keep") +#' r <- set_role(r, "yield", role = "outcome", action = "scramble") +#' r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +#' spec <- control_spec( +#' on = "gen", target = "yield", effect = list(sd = 0.5), +#' strata = "rep", n = 5L, seed = 2026L +#' ) +#' ctl <- controls(trial, r, spec) +#' ctl +#' head(control_table(ctl, "positive", 1L)) +#' +#' @seealso [control_spec()], [control_table()], [mask()], [roles()]. +#' +#' @export +controls <- function(df, roles, spec = NULL, events = NULL, seed = NULL, + recipe = NULL) { + withr::local_preserve_seed() + plan_notes <- character() + key <- character() + if (S7::S7_inherits(roles, .plan_class)) { + from_plan <- .controls_from_plan(df, roles, spec, events, seed, recipe) + df <- from_plan$df + roles <- from_plan$roles + spec <- from_plan$spec + events <- from_plan$events + seed <- from_plan$seed + key <- from_plan$key + plan_notes <- from_plan$notes + } else if (!is.null(recipe)) { + .control_refuse("`recipe` goes with a plan in `roles`: {.code controls(real, plan, recipe = rec)}.", "recipe") + } + .check_control_table(df) + .check_control_spec_class(spec) + if (!is.null(seed)) { + .check_seed(seed) + } + df <- .as_owned_dt(df) + roles <- roles_validate(roles, df = df) + + # Resolve the declaration against the table ------------------------------ + res <- .resolve_control_spec(spec, df, roles) + events <- .normalise_events(events, df, roles) + .check_control_target_events(res$spec, events) + + # Seeds: one for the null, one per replicate ----------------------------- + seeds <- if (length(key)) { + rep(NA_integer_, res$spec@n + 1L) + } else { + as.integer(.sub_seeds( + if (is.null(seed)) spec@seed else as.integer(seed), res$spec@n + 1L + )) + } + + manifest <- .control_manifest(res, seeds) + # A manifest is what the owner reads before a release: no private seed in it. + if (!is.null(seed) || length(key)) { + manifest$seed <- NA_integer_ + } + masque_controls( + table = df, + roles = .as_owned_dt(roles), + spec = res$spec, + seeds = seeds, + key = key, + analysed = res$analysed, + events = events, + strata_id = res$strata_id, + unit_id = res$unit_id, + grain_units = res$grain_units, + manifest = manifest, + truth = res$truth, + notes = c(res$notes, plan_notes) + ) +} + +#' Produce one control dataset +#' +#' Regenerates a control dataset of a [controls()] object from its seed: the +#' null, or negative or positive replicate `i`. Positive replicate `i` is +#' negative replicate `i` plus the planted shift. The result carries a +#' `masque_control` attribute naming its kind and replicate; its values +#' alone do not tell it from the table it came from, so keep the manifest +#' with it. +#' +#' @param ctl A `masque_controls` object from [controls()]. +#' @param kind One of `"null"`, `"negative"`, `"positive"`. +#' @param i The replicate, from 1 to `n` of the spec; ignored for the null. +#' +#' @returns A data.table with the columns, classes, NA mask and row order of +#' the input table. +#' +#' @examples +#' trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +#' r <- propose_roles(trial, detect = FALSE) +#' r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +#' r <- set_role(r, "gen", role = "treatment", action = "keep") +#' r <- set_role(r, "yield", role = "outcome", action = "scramble") +#' r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +#' spec <- control_spec(on = "gen", strata = "rep", n = 3L, seed = 1L) +#' ctl <- controls(trial, r, spec) +#' neg <- control_table(ctl, "negative", 2L) +#' table(neg$block, neg$gen)[1:2, 1:6] +#' +#' @seealso [controls()], [control_spec()]. +#' +#' @export +control_table <- function(ctl, kind = c("null", "negative", "positive"), i = 1L) { + withr::local_preserve_seed() + .check_controls_class(ctl) + kind <- match.arg(kind) + i <- .check_control_index(i, ctl@spec@n) + if (identical(kind, "positive") && is.null(ctl@spec@target)) { + .control_refuse( + "The spec declares no `target`, so there is no positive control.", + "kind" + ) + } + + out <- data.table::copy(ctl@table) + stream <- if (identical(kind, "null")) 0L else i + permute <- if (identical(kind, "null")) .permute_analysed else .permute_on + if (length(ctl@key)) { + permute(out, ctl, .keyed_draw(ctl@key, stream)) + } else { + with_rng_state(ctl@seeds[1L + stream], permute(out, ctl, sample.int)) + } + if (!identical(kind, "null")) { + if (identical(kind, "positive")) { + .plant_effect(out, ctl) + } + } + data.table::setattr(out, "masque_control", list( + kind = kind, + replicate = if (identical(kind, "null")) NA_integer_ else i + )) + out[] +} + +# Resolution ---------------------------------------------------------------- + +# Resolve a spec against a table and its roles. +.resolve_control_spec <- function(spec, df, roles) { + n <- nrow(df) + notes <- character() + design <- intersect( + names(df), roles$col[roles$role == "design" & roles$action != "drop"] + ) + analysed <- intersect(names(df), roles$col[ + roles$role %in% c("outcome", "covariate", "date") & + roles$action %in% c("scramble", "alias") + ]) + if (!length(analysed)) { + .control_refuse( + "The table has no analysed column: an outcome, covariate or date column with action `scramble` or `alias`.", + "analysed" + ) + } + kept_outcome <- roles$col[roles$role == "outcome" & roles$action == "keep"] + if (length(kept_outcome)) { + warning(warningCondition( + sprintf( + "Kept outcome column%s %s %s structure and %s not permuted by any control.", + if (length(kept_outcome) > 1L) "s" else "", + paste(kept_outcome, collapse = ", "), + if (length(kept_outcome) > 1L) "are" else "is", + if (length(kept_outcome) > 1L) "are" else "is" + ), + class = "masque_control_kept_outcome" + )) + } + + # The permuted column and its unit ------------------------------------ + on <- .resolve_control_on(spec@on, df, roles, spec@target) + unit <- .check_design_cols(spec@unit, "unit", design) + unit_id <- if (length(unit)) { + as.integer(.unit_id(df, unit)) + } else { + seq_len(n) + } + if (length(unit) && !.constant_within(unit_id, df[[on]])) { + .control_refuse( + "`on` ({.field {on}}) varies within its declared unit ({.field {unit}}); a unit must hold one value of `on`.", + "unit" + ) + } + + # Strata: nested above the unit, and `on` must vary somewhere ------------ + strata <- .check_design_cols(spec@strata, "strata", design) + strata_id <- if (length(strata)) { + as.integer(.unit_id(df, strata)) + } else { + rep(1L, n) + } + if (length(strata) && !.constant_within(unit_id, strata_id)) { + .control_refuse( + "A unit of `on` ({.field {unit}}) spans more than one stratum ({.field {strata}}); strata must be nested above the unit.", + "strata" + ) + } + constant_share <- .strata_constant_share(df[[on]], strata_id) + if (constant_share >= 1) { + .control_refuse( + c( + "`on` ({.field {on}}) is constant within every stratum ({.field {strata}}), so a restricted permutation changes nothing.", + i = "Coarsen the strata or drop them." + ), + "degenerate" + ) + } + if (constant_share > 0.1) { + warning(warningCondition( + sprintf( + "`on` (%s) is constant within %.0f%% of the strata; the negative control does not permute those strata, and a model that ignores them can reject on every replicate.", + on, 100 * constant_share + ), + class = "masque_control_strata_weak" + )) + } + if (!.level_attainable(unit_id, df[[on]], strata_id)) { + warning(warningCondition( + sprintf( + "`on` (%s) has too few distinct arrangements across its units and strata for a 5 percent randomisation level, so a rejection rate on the negative controls cannot read as a test size.", + on + ), + class = "masque_control_unit_small" + )) + } + + # Grain of other analysed columns ---------------------------------------- + grain_units <- list() + for (cl in names(spec@grain)) { + if (!cl %in% analysed) { + .control_refuse( + "`grain` names {.field {cl}}, which is not an analysed column of the table.", + "grain" + ) + } + u <- .check_design_cols(spec@grain[[cl]], "grain", design) + gid <- as.integer(.unit_id(df, u)) + if (.constant_within(gid, df[[cl]])) { + grain_units[[cl]] <- gid + } else { + notes <- c(notes, sprintf( + "%s is not constant within %s in this table; permuted at the row.", + cl, paste(u, collapse = " + ") + )) + } + } + + # Targets and the planted truth ----------------------------------------- + target <- .check_control_targets(spec@target, df, roles, on, spec@grain) + levels <- .control_levels(df[[on]]) + effect <- .resolve_control_effect(spec@effect, levels, spec@seed) + truth <- .control_truth(df, target, effect, strata_id) + + spec_out <- masque_control_spec( + on = on, + target = target, + effect = effect, + strata = strata, + unit = unit, + grain = spec@grain, + n = spec@n, + seed = spec@seed + ) + list( + spec = spec_out, + analysed = analysed, + strata_id = strata_id, + unit_id = unit_id, + n_strata = max(strata_id), + constant_share = constant_share, + grain_units = grain_units, + truth = truth, + notes = notes + ) +} + +# `on`: the single treatment column when NULL; present, a treatment or +# covariate, categorical or whole-valued, and not a target. +.resolve_control_on <- function(on, df, roles, target) { + if (is.null(on)) { + trt <- intersect( + names(df), + roles$col[roles$role == "treatment" & roles$action != "drop"] + ) + if (length(trt) != 1L) { + .control_refuse( + c( + "`on` is `NULL` and the table has {length(trt)} treatment column{?s}; name `on`.", + i = "A negative control needs exactly one column to permute." + ), + "on" + ) + } + on <- trt + } + if (!on %in% names(df)) { + .control_refuse("`on` ({.field {on}}) is not a column of the table.", "on") + } + role <- roles$role[match(on, roles$col)] + kind <- roles$kind[match(on, roles$col)] + if (!role %in% c("treatment", "covariate")) { + .control_refuse( + "`on` ({.field {on}}) has role {.val {role}}; it must be a treatment or covariate column.", + "on" + ) + } + # An aliased numeric is a set of labels, on both sides. + action <- roles$action[match(on, roles$col)] + categorical <- kind %in% .categorical_kinds() || identical(action, "alias") + whole <- kind %in% .numeric_kinds() && .is_whole_valued(df[[on]]) + if (!categorical && !whole) { + .control_refuse( + "`on` ({.field {on}}) must be categorical or whole-valued; a continuous column has no levels to shift.", + "on" + ) + } + if (on %in% target) { + .control_refuse("`on` ({.field {on}}) cannot also be a `target`.", "on") + } + on +} + +# Columns named for strata, unit or grain must be retained design columns. +.check_design_cols <- function(cols, arg, design) { + if (!length(cols)) { + return(character()) + } + bad <- setdiff(cols, design) + if (length(bad)) { + .control_refuse( + "`{arg}` names {.field {bad}}, which {?is/are} not {?a/} retained design column{?s} of the table.", + arg + ) + } + cols +} + +# Targets: present, outcome or covariate, numeric with more than two values, +# not `on`, and not declared unit-grained. +.check_control_targets <- function(target, df, roles, on, grain) { + if (is.null(target)) { + return(NULL) + } + missing <- setdiff(target, names(df)) + if (length(missing)) { + .control_refuse( + "`target` names {.field {missing}}, not {?a/} column{?s} of the table.", + "target" + ) + } + for (tg in target) { + role <- roles$role[match(tg, roles$col)] + kind <- roles$kind[match(tg, roles$col)] + y <- df[[tg]] + ok <- role %in% c("outcome", "covariate") && kind %in% .numeric_kinds() && + data.table::uniqueN(y, na.rm = TRUE) > 2L + if (!ok) { + .control_refuse( + "`target` {.field {tg}} must be a numeric outcome or covariate with more than two distinct values.", + "target_kind" + ) + } + if (tg %in% names(grain)) { + .control_refuse( + "`target` {.field {tg}} is declared unit-grained; a shift by level of `on` would break its constancy within the unit.", + "target_grain" + ) + } + } + if (on %in% target) { + .control_refuse("`on` ({.field {on}}) cannot also be a `target`.", "target") + } + target +} + +# A target cannot be a declared event column. +.check_control_target_events <- function(spec, events) { + if (is.null(spec@target) || !length(events)) { + return(invisible(NULL)) + } + ev <- unlist(lapply(events, function(e) c(e$time, e$status))) + bad <- intersect(spec@target, ev) + if (length(bad)) { + .control_refuse( + "`target` {.field {bad}} is a linked-event column; events take no planted shift.", + "target_kind" + ) + } + invisible(NULL) +} + +# The levels of `on` as character, in sorted order. +.control_levels <- function(x) { + .level_key(sort(unique(x[!is.na(x)]))) +} + +# Share of strata in which `on` holds at most one distinct value. +.strata_constant_share <- function(x, strata_id) { + g <- V1 <- NULL + obs <- !is.na(x) + pairs <- data.table::data.table(g = strata_id, x = x) + per <- pairs[obs, data.table::uniqueN(x), by = g] + n_strata <- data.table::uniqueN(strata_id) + constant <- n_strata - nrow(per) + sum(per$V1 <= 1L) + constant / n_strata +} + +# The effect vector over the levels: drawn from `list(sd = )` under the +# spec's seed, or a named vector completed with zeros. +.resolve_control_effect <- function(effect, levels, seed) { + if (is.null(effect)) { + return(NULL) + } + if (is.list(effect)) { + root <- (as.numeric(seed) + .CONTROL_SALT) %% .Machine$integer.max + draw <- with_rng_state(root, stats::rnorm(length(levels), 0, effect$sd)) + return(stats::setNames(draw, levels)) + } + unknown <- setdiff(names(effect), levels) + if (length(unknown)) { + .control_refuse( + "`effect` names {.val {unknown}}, not {?a/} level{?s} of `on`.", + "effect" + ) + } + out <- stats::setNames(rep(0, length(levels)), levels) + out[names(effect)] <- effect + out +} + +# One row per target and level; a whole-valued target takes a whole shift. +.control_truth <- function(df, target, effect, strata_id) { + empty <- data.table::data.table( + target = character(), level = character(), effect_sd = double(), + sd_y = double(), sd_within_strata = double(), shift_raw = double(), + shift_realised = double(), whole_valued = logical() + ) + if (is.null(target)) { + return(empty) + } + rows <- lapply(target, function(tg) { + y <- df[[tg]] + sd_y <- stats::sd(y, na.rm = TRUE) + resid <- y - stats::ave(y, strata_id, FUN = function(v) mean(v, na.rm = TRUE)) + whole <- .is_whole_valued(y) + shift_raw <- unname(effect) * sd_y + shift_realised <- if (whole) round(shift_raw) else shift_raw + vanished <- whole & effect != 0 & shift_realised == 0 + ratio <- (shift_realised / shift_raw)[shift_raw != 0] + if (whole && !any(vanished) && length(ratio) && diff(range(ratio)) > 0.1 * mean(abs(ratio))) { + warning(warningCondition( + sprintf( + "On the whole-valued target %s, rounding changes the shape of the declared effect. `shift_realised` in `@truth` holds the effect that is in the table.", + tg + ), + class = "masque_control_shape_rounded" + )) + } + if (any(vanished)) { + .control_refuse( + c( + "On the whole-valued target {.field {tg}}, the shift for level{?s} {.val {names(effect)[vanished]}} rounds to zero.", + i = "Declare a larger effect, or a target with a finer resolution." + ), + "shift_vanishes" + ) + } + data.table::data.table( + target = tg, + level = names(effect), + effect_sd = unname(effect), + sd_y = sd_y, + sd_within_strata = stats::sd(resid, na.rm = TRUE), + shift_raw = shift_raw, + shift_realised = shift_realised, + whole_valued = whole + ) + }) + data.table::rbindlist(rows) +} + +# One manifest row per dataset: what was permuted, where, under which seed. +.control_manifest <- function(res, seeds) { + spec <- res$spec + n <- spec@n + kinds <- c("null", rep("negative", n), if (!is.null(spec@target)) rep("positive", n)) + reps <- c(NA_integer_, seq_len(n), if (!is.null(spec@target)) seq_len(n)) + dataset <- ifelse(is.na(reps), kinds, paste0(kinds, "_", reps)) + seed_of <- ifelse(is.na(reps), seeds[1L], seeds[1L + ifelse(is.na(reps), 0L, reps)]) + permuted <- ifelse( + kinds == "null", paste(res$analysed, collapse = ", "), spec@on + ) + data.table::data.table( + dataset = dataset, + kind = kinds, + replicate = reps, + seed = as.integer(seed_of), + permuted = permuted, + strata = if (length(spec@strata)) paste(spec@strata, collapse = " + ") else "none", + unit = if (length(spec@unit)) paste(spec@unit, collapse = " + ") else "row", + n_strata = as.integer(res$n_strata), + strata_constant_share = res$constant_share, + target = if (is.null(spec@target)) NA_character_ else paste(spec@target, collapse = ", ") + ) +} + +# Permutation --------------------------------------------------------------- + +# Can a permutation of `on` across units within strata reach a 5 percent level? +# True when it has at least 20 arrangements. Swapping the labels of levels with +# the same count in every stratum gives the same partition, so those count once. +.level_attainable <- function(unit_id, on_val, strata_id) { + keep <- !is.na(on_val) + keep[keep] <- !duplicated(unit_id[keep]) + counts <- table(as.character(on_val[keep]), strata_id[keep]) + log_n <- sum(lfactorial(colSums(counts))) - sum(lfactorial(counts)) + same_counts <- apply(counts, 1L, paste, collapse = " ") + log_n - sum(lfactorial(table(same_counts))) >= log(20) - 1e-8 +} + +# One permutation of 1:n by Fisher-Yates, its uniforms taken from SHA-256 of +# the label and a counter, so the key is the whole digest and not an R seed. +.keyed_perm <- function(label, n) { + perm <- seq_len(n) + if (n < 2L) { + return(perm) + } + hash <- digest::getVDigest("sha256") + h <- hash(paste(label, seq_len(n - 1L), sep = "\r"), serialize = FALSE) + u <- (strtoi(substr(h, 1L, 6L), 16L) * 16^7 + strtoi(substr(h, 7L, 13L), 16L) + 0.5) / 16^13 + for (i in n:2L) { + j <- 1L + as.integer(floor(u[i - 1L] * i)) + held <- perm[i] + perm[i] <- perm[j] + perm[j] <- held + } + perm +} + +# A drawing function for one dataset: successive calls take successive labels. +.keyed_draw <- function(key, stream) { + calls <- 0L + function(n) { + calls <<- calls + 1L + .keyed_perm(paste(key, stream, calls, sep = "\r"), n) + } +} + +# Permute the non-missing values of `x` across units within strata; an +# all-missing unit stays missing. With `unit_id = seq_along(x)` it is a row +# permutation. +.permute_within <- function(x, unit_id, strata_id, draw = sample.int) { + obs <- which(!is.na(x)) + if (length(obs) < 2L) { + return(x) + } + u_obs <- unique(unit_id[obs]) + first <- obs[match(u_obs, unit_id[obs])] + u_val <- x[first] + groups <- split(seq_along(u_obs), strata_id[first]) + perm <- integer(length(u_obs)) + for (g in groups) { + perm[g] <- g[draw(length(g))] + } + new_val <- u_val[perm] + out <- x + out[obs] <- new_val[match(unit_id[obs], u_obs)] + out +} + +# The null: every analysed column on its own draw, at its grain, no strata; +# a linked-event pair moves as one row block. +.permute_analysed <- function(out, ctl, draw) { + n <- nrow(out) + rows <- seq_len(n) + none <- rep(1L, n) + pair_cols <- unlist(lapply(ctl@events, function(e) c(e$time, e$status))) + for (cl in setdiff(ctl@analysed, pair_cols)) { + uid <- ctl@grain_units[[cl]] + if (is.null(uid)) { + uid <- rows + } + data.table::set(out, j = cl, value = .permute_within(out[[cl]], uid, none, draw)) + } + for (e in ctl@events) { + perm <- .permute_within(rows, rows, none, draw) + for (cl in c(e$time, e$status)) { + data.table::set(out, j = cl, value = out[[cl]][perm]) + } + } + invisible(out) +} + +# The negative: `on` alone, within strata, at its unit. +.permute_on <- function(out, ctl, draw) { + on <- ctl@spec@on + data.table::set( + out, + j = on, + value = .permute_within(out[[on]], ctl@unit_id, ctl@strata_id, draw) + ) + invisible(out) +} + +# The positive: each target's realised shift on the rows of each level of +# the permuted `on`; nothing is clipped. +.plant_effect <- function(out, ctl) { + on_val <- .level_key(out[[ctl@spec@on]]) + crossed <- character() + for (tg in unique(ctl@truth$target)) { + tr <- ctl@truth[ctl@truth$target == tg, ] + shift <- tr$shift_realised[match(on_val, tr$level)] + shift[is.na(shift)] <- 0 + y <- out[[tg]] + obs <- !is.na(y) + y_new <- y + y_new[obs] <- y[obs] + shift[obs] + if (is.integer(y)) { + y_new <- as.integer(round(y_new)) + } + if (all(y[obs] >= 0) && any(y_new[obs] < 0)) { + crossed <- c(crossed, tg) + } + data.table::set(out, j = tg, value = y_new) + } + if (length(crossed)) { + warning(warningCondition( + sprintf( + "The planted shift took %s below zero on a target with no negative values; nothing is clipped.", + paste(crossed, collapse = ", ") + ), + class = "masque_control_sign_crossed" + )) + } + invisible(out) +} + +# Argument checks ----------------------------------------------------------- + +.check_control_table <- function(df) { + if (!is.data.frame(df)) { + .control_refuse( + "`df` must be a data frame; got {.cls {class(df)[1]}}.", "table" + ) + } + if (nrow(df) < 2L) { + .control_refuse("`df` needs at least two rows to permute.", "table") + } + if (anyDuplicated(names(df))) { + .control_refuse("`df` has duplicated column names.", "table") + } + invisible(df) +} + +.check_control_spec_class <- function(spec) { + if (!S7::S7_inherits(spec, masque_control_spec)) { + .control_refuse( + "`spec` must be a {.cls masque_control_spec} from {.fn control_spec}; got {.cls {class(spec)[1]}}.", + "spec" + ) + } + invisible(spec) +} + +.check_controls_class <- function(ctl) { + if (!S7::S7_inherits(ctl, masque_controls)) { + .control_refuse( + "`ctl` must be a {.cls masque_controls} from {.fn controls}; got {.cls {class(ctl)[1]}}.", + "kind" + ) + } + invisible(ctl) +} + +.check_control_index <- function(i, n) { + ok <- is.numeric(i) && length(i) == 1L && !is.na(i) && i == round(i) && + i >= 1L && i <= n + if (!ok) { + .control_refuse( + "`i` must be a single replicate number from 1 to {n}.", "kind" + ) + } + as.integer(i) +} + +# Print --------------------------------------------------------------------- + +# local(): see the plot method in detect_plot.R. +local(S7::method(print, masque_controls) <- function(x, ...) { + spec <- x@spec + cli::cli_h1("masque_controls") + cli::cli_bullets(c( + "*" = sprintf( + "Table: %d rows, %d columns; %d analysed column%s", + nrow(x@table), ncol(x@table), length(x@analysed), + if (length(x@analysed) == 1L) "" else "s" + ), + "*" = sprintf("Permuted column (on): %s", spec@on), + "*" = sprintf( + "Strata: %s (%d strata; on constant in %.0f%% of them)", + .spec_cols_line(spec@strata, "none"), max(x@strata_id), + 100 * x@manifest$strata_constant_share[1L] + ), + "*" = sprintf("Unit of on: %s", .spec_cols_line(spec@unit, "row")), + "*" = sprintf( + "Datasets: 1 null, %d negative, %d positive", + spec@n, if (is.null(spec@target)) 0L else spec@n + ), + "*" = if (length(x@key)) "Draw: a private key" else "Seeds: present" + )) + if (nrow(x@truth)) { + cli::cli_h2("Planted truth (positive minus negative, per level of on)") + print(x@truth[, c( + "target", "level", "effect_sd", "sd_y", "sd_within_strata", + "shift_realised" + )]) + } + if (length(x@notes)) { + cli::cli_h2("Notes") + cli::cli_bullets(stats::setNames(x@notes, rep("!", length(x@notes)))) + } + invisible(x) +}) diff --git a/R/jitter_coordinates.R b/R/jitter_coordinates.R index f862c1b..4d14551 100644 --- a/R/jitter_coordinates.R +++ b/R/jitter_coordinates.R @@ -27,22 +27,26 @@ #' source table has one true coordinate per place. `by` chooses how a site is #' identified: #' -#' - `by = NULL` (default) treats rows that share an identical input coordinate -#' pair as one site. This needs no user action and cannot alter genuinely +#' - `by = NULL` (default) treats rows that share an input coordinate pair, to +#' nine decimal places, as one site. This needs no user action and cannot alter genuinely #' point-level data, where every input pair is already distinct. #' - `by = c("trial")` (one or more column names) treats rows sharing those #' values as one site, for the case where a site's recorded coordinates differ #' slightly between rows. Where such a group holds more than one distinct #' input coordinate, the group's row-weighted centroid is used and the #' consolidation is reported. -#' - `by = FALSE` restores the pre-0.10.0 behaviour of displacing every row -#' independently, and warns if the input carries repeated coordinate pairs. -#' It exists only so a table masked by an earlier version can be reproduced. +#' - `by = FALSE` displaces every row independently, as versions before 0.10.0 +#' did, and warns if the input carries repeated coordinate pairs. The draw +#' follows the row's position, so shuffled rows move differently. +#' +#' The draw is a function of the seed, the two column names and the site's +#' value: its coordinates, or its `by` values. Row order and the other sites +#' in the table play no part. #' #' Displacing each row of a repeated coordinate separately leaks the site. #' Donut displacement is isotropic, so the mean of many independent draws #' around one true site converges on that site: averaging the 360 rows of -#' a single trial recovers the true position to a median 0.58 km, against the +#' a single trial recovers the true position to a median 0.64 km, against the #' 5 km floor the donut was asked for. One draw per site removes the estimator #' and leaves the full displacement in place. #' @@ -55,19 +59,31 @@ #' #' \tabular{lr}{ #' Rebuilds kept \tab Median recovery error \cr -#' 1 \tab 14.2 km \cr -#' 2 \tab 10.1 km \cr -#' 4 \tab 6.4 km \cr -#' 8 \tab 4.5 km \cr +#' 1 \tab 14.6 km \cr +#' 2 \tab 9.4 km \cr +#' 4 \tab 6.3 km \cr +#' 8 \tab 4.4 km \cr #' 16 \tab 3.1 km #' } #' -#' Eight kept rebuilds fall below the 5 km floor the donut was asked for. +#' Seven kept rebuilds fall below the 5 km floor the donut was asked for. +#' +#' A known region also cuts the annulus. Someone who knows a site lies in a +#' square a few times `max_km` wide or smaller can discard the part of the +#' annulus around the released point that falls outside it, with no second +#' release needed. #' -#' The remedy is to **reuse the seed**. The same `seed` reproduces the -#' same displacement for the same sites, so any number of rebuilds yields one -#' point and nothing to average. Draw a fresh seed only when you intend to -#' supersede every earlier version, and retire the ones you replace. +#' The remedy is to **reuse the seed**. The same `seed` gives a site the same +#' displacement in every rebuild, even when rows are shuffled, added or removed, +#' so any number of rebuilds yields one point and nothing to average. Three +#' things start a new draw for a site: a different seed, a changed site value +#' (a corrected coordinate, a renamed `by` label) and renamed coordinate +#' columns. A clone built before version 0.17.0 used a different draw, so a +#' rebuild under 0.17.0 is a new release of those sites even with the same seed. +#' Treat each of these as a fresh seed: retire the versions it replaces. The seed +#' reveals every displacement, so it stays as confidential as the recipe. A seed +#' has 31 bits: where a clone shows site labels and holds two pairs at one place, +#' it can be found by search, so alias the label or declare one pair. #' #' Choose the unit carefully. It is the finest grouping that denotes **one #' physical place at one time**. For a multi-year trial series that is location @@ -106,7 +122,11 @@ #' displacement that lands in the sea, using [maps::map.where()] (requires the #' `maps` package). A function `function(lon, lat)` returning a logical vector #' supplies your own test (for example an `sf`-based high-resolution -#' coastline). `FALSE` applies no constraint. +#' coastline). `FALSE` applies no constraint. The test conditions the annulus: +#' near a coast the accepted directions are the ones on land, so the mean +#' displacement points inland and the median distance can move by kilometres, +#' while no point falls inside `min_km`. A site that cannot reach land is +#' missing. #' @param max_tries Maximum re-draws per site before giving up. Default `100`. #' A site that cannot be placed on land is set to `NA` on both axes, with a #' warning: masque never emits the true coordinate as though it were masked. @@ -198,7 +218,11 @@ jitter_coordinates <- function(df, lat_col, lon_col, land_fn <- .resolve_land_fn(on_land) withr::local_preserve_seed() - if (!is.null(seed)) set.seed(as.integer(.check_seed(seed))) + root <- if (is.null(seed)) { + sample.int(.Machine$integer.max, 1L) + } else { + as.integer(.check_seed(seed)) + } lat <- df[[lat_col]] lon <- df[[lon_col]] @@ -218,9 +242,11 @@ jitter_coordinates <- function(df, lat_col, lon_col, lat_v <- lat[valid] lon_v <- lon[valid] - # Rows are one site only when their coordinates are identical as doubles. + # Rows are one site when their coordinates agree to nine decimal places, about + # 0.1 mm, so writing the table to a file and back does not start a new draw. + # Adding zero merges -0 with 0. pair_key <- paste( - match(lat_v, unique(lat_v)), match(lon_v, unique(lon_v)), + sprintf("%.9f", lat_v + 0), sprintf("%.9f", lon_v + 0), sep = "\r" ) @@ -302,15 +328,16 @@ jitter_coordinates <- function(df, lat_col, lon_col, tries <- tries + 1L la <- src_lat[pending] lo <- src_lon[pending] + u <- .site_uniforms(root, c(lat_col, lon_col), ug[pending], tries) if (method == "donut") { # sqrt of a uniform on [min^2, max^2] is uniform by AREA over the annulus. - radius <- sqrt(stats::runif(length(pending), min_km^2, max_km^2)) - bearing <- stats::runif(length(pending), 0, 2 * pi) + radius <- sqrt(min_km^2 + u[, 1L] * (max_km^2 - min_km^2)) + bearing <- u[, 2L] * 2 * pi d_north <- radius * cos(bearing) d_east <- radius * sin(bearing) } else { - d_north <- stats::rnorm(length(pending), 0, sd_km) - d_east <- stats::rnorm(length(pending), 0, sd_km) + d_north <- stats::qnorm(u[, 1L], 0, sd_km) + d_east <- stats::qnorm(u[, 2L], 0, sd_km) } cand_lat <- la + d_north / 111.32 cand_lon <- lo + d_east / (111.32 * cos(la * pi / 180)) @@ -349,6 +376,24 @@ jitter_coordinates <- function(df, lat_col, lon_col, ) } +# Two uniforms in (0, 1) per site for one try: a pure function of the seed, +# the coordinate pair, the try and the site's value. Nothing depends on row +# order, on the other sites or on the RNG kind, so a pinned seed gives a site +# the same displacement in any table that holds it. +.site_uniforms <- function(root, pair, keys, try) { + hash <- digest::getVDigest("sha256") + h <- hash( + enc2utf8(paste(root, pair[1L], pair[2L], try, keys, sep = "\r")), + serialize = FALSE + ) + word <- function(from) { + high <- strtoi(substr(h, from, from + 5L), 16L) + low <- strtoi(substr(h, from + 6L, from + 12L), 16L) + (high * 16^7 + low + 0.5) / 16^13 + } + cbind(word(1L), word(14L)) +} + # A structured account of one coordinate-pair jitter, recorded on the recipe. .coord_report <- function(mode, n_sites, n_rows, consolidated = character(), unplaced = 0L) { @@ -579,13 +624,13 @@ jitter_coordinates <- function(df, lat_col, lon_col, } # Jitter each spec in place and return the frame with one report per spec. -# Sites are grouped on `original`; each spec draws from its own sub-stream. +# Sites are grouped on `original`; the pair's names separate its draws. .apply_coord_jitter <- function(synth, specs, seed, original = NULL) { n <- length(specs) if (!n) { return(list(df = synth, reports = list())) } - sub_seeds <- .sub_seeds(seed, n) + root <- if (is.null(seed)) .sub_seeds(NULL, 1L) else seed reports <- vector("list", n) for (i in seq_len(n)) { s <- specs[[i]] @@ -602,7 +647,7 @@ jitter_coordinates <- function(df, lat_col, lon_col, synth, lat_col = s$lat, lon_col = s$lon, by = if (isFALSE(s$by)) FALSE else NULL, method = s$method, min_km = s$min_km, max_km = s$max_km, - sd_km = s$sd_km, on_land = s$on_land, seed = sub_seeds[[i]], + sd_km = s$sd_km, on_land = s$on_land, seed = root, .group = grp ) synth <- res$df diff --git a/R/leakage.R b/R/leakage.R index 6ddf582..074708a 100644 --- a/R/leakage.R +++ b/R/leakage.R @@ -16,6 +16,7 @@ if (is.null(coarsened)) coarsened <- character() na_pattern_uniq <- .global_na_pattern_uniqueness(df) strata <- .audit_strata(df, rec) + row_terms <- .audit_row_terms(df, rec) design_cols <- intersect( rec@roles$col[rec@roles$role == "design"], names(df) ) @@ -75,6 +76,14 @@ } } + # A re-simulated number that tracks the original at its own row. + row_track <- list(all = NA_real_, within = NA_real_) + if (in_synth && kind %in% c("numeric", "integer") && + .audit_numeric_resimulated(action) && !is_grained && !two_valued && + !(col %in% linked) && comparable_n >= .MIN_STRATUM) { + row_track <- .row_tracking(x, synth[[col]], row_terms) + } + # Same-row match against a permutation baseline: categorical, date and # two-valued columns, compared in the original label space. same <- NULL @@ -122,7 +131,8 @@ two_valued = two_valued, linked = col %in% linked, by_excess = by_excess, grained = is_grained, allow_kept_outcome = allow_kept_outcome, - value_set_pct = if (is.null(value_set)) NA_real_ else value_set$pct + value_set_pct = if (is.null(value_set)) NA_real_ else value_set$pct, + row_track = row_track ) notes <- .audit_notes( @@ -134,6 +144,14 @@ row_grain = Filter(function(g) col %in% g$cols, row_grain), value_set = value_set ) + tracked <- c(row_track$within, row_track$all) + if (isTRUE(abs(row_track$within) >= .AUDIT_THRESHOLDS$row_cor_within_medium) || + isTRUE(abs(row_track$all) >= .AUDIT_THRESHOLDS$row_cor)) { + notes <- sprintf( + "tracks the original row by row (r = %.2f)", + tracked[which(!is.na(tracked))[1L]] + ) + } data.table::data.table( col = col, @@ -166,6 +184,12 @@ same_row_medium = list(excess = 10, z = 4), same_row_high = list(excess = 50, z = 4), value_set_high = 50, + # Tracking of a number on its own row: overall, and inside small levels. A real + # alpha design conditioned on 24 genotypes of three rows reaches 0.81. + row_cor = 0.95, + row_cor_within_high = 0.9, + row_cor_within_medium = 0.7, + row_cell_small = 10L, joint_medium = list(excess = 5, z = 4), joint_high = list(excess = 20, z = 4), rare_medium = list(excess = 1, z = 3), @@ -183,6 +207,63 @@ excess >= rule$z * sd) } +# The columns a clone's numbers were conditioned or shifted on, sorted into +# large terms, whose means a clone may keep, and the rows held by small levels, +# whose means sit close to those rows. A treatment's means are the point of a +# conditional clone, so its levels count as small only below the stratum floor. +.audit_row_terms <- function(df, rec) { + used <- intersect(.prop_or(rec, "conditioning_used", character()), names(df)) + shifted <- .prop_or(rec, "conditioning_shifted", character()) + cols <- intersect(unique(c(used, unlist(strsplit(shifted, ":", fixed = TRUE)))), names(df)) + treatment <- rec@roles$col[rec@roles$role == "treatment"] + small_rows <- rep(FALSE, nrow(df)) + large <- list() + # Rows per level, row by row, for a partition. + level_size <- function(partition) { + id <- .row_group_id(.cols(df, partition)) + tabulate(id)[id] + } + for (cl in cols) { + trt <- cl %in% treatment + size <- level_size(cl) + top <- if (trt) .MIN_STRATUM - 1L else .AUDIT_THRESHOLDS$row_cell_small + bottom <- if (trt) 2L else 3L + small_rows <- small_rows | (size >= bottom & size <= top) + if (stats::median(size) > top) { + large <- c(large, list(df[[cl]])) + } + } + if (length(used) >= 2L) { + size <- level_size(used) + small_rows <- small_rows | + (size >= .MIN_STRATUM & size <= .AUDIT_THRESHOLDS$row_cell_small) + } + list(large = large, small_rows = small_rows) +} + +# Correlation of a number with its original overall and, on the rows of small +# levels, after the means of the large terms are taken out. +.row_tracking <- function(x, y, terms) { + both <- !is.na(x) & !is.na(y) + r <- function(a, b) { + if (length(a) >= 2L && stats::sd(a) > 0 && stats::sd(b) > 0) stats::cor(a, b) else NA_real_ + } + a <- as.numeric(x[both]) + b <- as.numeric(y[both]) + overall <- r(a, b) + small <- terms$small_rows[both] + if (sum(small) < .MIN_STRATUM) { + return(list(all = overall, within = NA_real_)) + } + for (g in terms$large) { + g <- as.character(g[both]) + g[is.na(g)] <- ".na" + a <- a - stats::ave(a, g) + b <- b - stats::ave(b, g) + } + list(all = overall, within = r(a[small], b[small])) +} + # A scrambled numeric column is re-simulated through the copula, within the # conditioning strata when the clone is conditional. .audit_numeric_resimulated <- function(action) { @@ -583,7 +664,8 @@ exact_match_pct, freq_min, n_unique_levels, n_rows, coarsened = FALSE, same_row_excess = NA_real_, same_row_sd = NA_real_, two_valued = FALSE, linked = FALSE, by_excess = FALSE, grained = FALSE, - allow_kept_outcome = FALSE, value_set_pct = NA_real_ + allow_kept_outcome = FALSE, value_set_pct = NA_real_, + row_track = list(all = NA_real_, within = NA_real_) ) { collab <- identical(mode, "collaborate") @@ -598,6 +680,12 @@ return(if (collab) "high" else "medium") } + # 0b. A re-simulated number that tracks the original inside small conditioning + # cells, whose means are close to the rows themselves: HIGH. + if (collab && isTRUE(abs(row_track$within) >= .AUDIT_THRESHOLDS$row_cor_within_high)) { + return("high") + } + # 1b. Outcome kept as real values beside re-simulated columns in # collaborate: HIGH, MEDIUM under `allow_kept_outcome = TRUE`. if (role == "outcome" && collab && in_synth && !is.na(action) && @@ -628,6 +716,13 @@ return("high") } + # 3c. Weaker tracking inside small levels, or overall only, where the clone may + # be keeping big environment means. + if (collab && (isTRUE(abs(row_track$within) >= .AUDIT_THRESHOLDS$row_cor_within_medium) || + isTRUE(abs(row_track$all) >= .AUDIT_THRESHOLDS$row_cor))) { + return("medium") + } + # 4. Outcome with exact-match-pct > 1% in collaborate: MEDIUM. Two-valued # columns match by chance about half the time; rule 5b grades them. if (role == "outcome" && collab && !isTRUE(two_valued) && diff --git a/R/mask.R b/R/mask.R index 0bcc50c..cb29232 100644 --- a/R/mask.R +++ b/R/mask.R @@ -51,7 +51,10 @@ #' [conform_table()], whose `data` element is then masked. Its cleaning #' record and its merges and type changes go on the recipe, so #' [apply_recipe()] replays them; `clean` is not applied a second time. -#' @param roles A roles table from [propose_roles()] (possibly edited). +#' @param roles A roles table from [propose_roles()] (possibly edited), or a +#' [masque_plan()]. A plan supplies `coords`, `events`, `conditional`, +#' `ladder`, `grain` and `alias_names` where the call leaves them out, and its +#' control specs are resolved on `df` and written back to the recipe. #' Tables from masque <= 0.5.0 are upgraded with a deprecation #' warning; see [roles_validate()]. #' @param mode Either `"local"` or `"collaborate"`. When omitted, inherit @@ -82,8 +85,9 @@ #' [apply_recipe()] / [unmask()], so a pipeline written against the #' aliased synthetic round-trips. Alias the names when the schema itself #' is sensitive. -#' @param conditional Logical scalar (default `FALSE`). The -#' *collaborate-grade conditional clone*. When `FALSE`, scrambled +#' @param conditional `FALSE` (default), `TRUE`, `"design"`, or a character +#' vector of column names. The *collaborate-grade conditional clone*. +#' When `FALSE`, scrambled #' numeric columns are re-simulated from one global Gaussian copula - #' marginals and global covariance survive, but the treatment-to-outcome #' relationship does not, so a causal model fitted on the clone recovers @@ -106,6 +110,18 @@ #' pooling raises a classed `masque_conditional_degraded` warning -- #' including the case where no treatment or design column survives at #' all, in which case the clone is the pooled copula. +#' +#' `"design"` conditions on the retained design columns alone. The clone +#' keeps the design means, block and environment variance, while a treatment +#' effect is not carried: a real effect moves into the residual, as it does +#' when an owner permutes the treatment for a negative control, so the clone +#' stands in for those negatives. The residual is not the one a real fit has. +#' Linked `events` are stratified only by treatment columns of the partition, +#' so `"design"` and a design-only partition do not keep their design means. +#' A character vector of names gives the partition explicitly, from the +#' retained design and treatment columns, and runs through the same ladder; +#' treatment columns are still never dropped. The single word `"design"` +#' always means this option, even where a column has that name. #' @param ladder How the coarsening ladder orders the design columns it may #' drop, and what it keeps of a dropped one. `"hierarchy"` (the default) #' drops plot coordinates (`row`, `col`, `range`, `plot`) first, then @@ -124,7 +140,12 @@ #' how a three-replicate variety trial keeps its genotype means. A level #' with a single row is pooled with the other singletons before the fit, #' so no row's own outcome is carried. The terms carried are recorded on -#' the recipe as `conditioning_shifted`. `"levels"` +#' the recipe as `conditioning_shifted`. The interaction of a dropped column +#' with a kept treatment is not carried, so a multi-environment clone can keep +#' its genotype and environment means and lose the genotype-by-environment +#' pattern. Shrinkage of a shift uses the mean level size, so on unbalanced +#' columns the small levels are shrunk less than their own size would give. +#' `"levels"` #' is the ladder of masque 0.11.1 to 0.12.0: columns are dropped in #' decreasing order of distinct values and a dropped column leaves no #' trace in the clone. @@ -146,7 +167,8 @@ #' grouping it was coarsened under, and [apply_recipe()] retargets to the real #' coordinates. Several published jitters of the same point average toward #' it: keep the seed, which the recipe records, and do not publish more than -#' one jittered release of the same sites. +#' one jittered release of the same sites. A seed has 31 bits: where a clone +#' shows site labels and holds two pairs at one place, it can be found by search. #' @param events Optional linked time-to-event outcomes. One spec per group of #' linked events, `list(time = c(...), status = c(...))`, with the time and #' status columns listed in the order the events occur; pass several specs @@ -273,6 +295,28 @@ mask <- function(df, # Undo any RNG change made inside mask() on exit, whichever path made it. withr::local_preserve_seed() + # A plan fills the arguments the call leaves out. + plan <- NULL + if (!missing(roles) && S7::S7_inherits(roles, .plan_class)) { + plan <- roles + if (!identical(plan@vocabulary, "source")) { + .plan_refuse( + c( + "This plan is in clone vocabulary.", + i = "It describes a clone for {.fn controls}, and cannot mask a table." + ), + "vocabulary" + ) + } + roles <- plan@roles + if (missing(coords) && length(plan@coords)) coords <- plan@coords + if (missing(events) && length(plan@events)) events <- plan@events + if (missing(conditional)) conditional <- plan@conditional + if (missing(ladder)) ladder <- plan@ladder + if (missing(grain)) grain <- plan@grain + if (missing(alias_names)) alias_names <- plan@alias_names + } + # A conformance object carries its own cleaning and the judged decisions; # both go on the recipe so apply_recipe() can replay them. conform_rec <- NULL @@ -287,13 +331,7 @@ mask <- function(df, class = c("masque_bad_df_refusal", "orchestra_refusal") ) } - if (!is.logical(conditional) || length(conditional) != 1L || - is.na(conditional)) { - cli::cli_abort( - "`conditional` must be a single `TRUE` or `FALSE`.", - class = c("masque_bad_conditional_refusal", "orchestra_refusal") - ) - } + .check_conditional_arg(conditional) if (!is.logical(grain) || length(grain) != 1L || is.na(grain)) { cli::cli_abort( "`grain` must be a single `TRUE` or `FALSE`.", @@ -429,15 +467,14 @@ mask <- function(df, opts <- mode_defaults(mode) # With nothing to condition on, a conditional clone uses the global copula. - conditioning_cols <- if (isTRUE(conditional)) { - .conditioning_cols(roles) - } else { - character() - } - if (isTRUE(conditional) && !length(conditioning_cols)) { + conditioning_cols <- .resolve_conditioning(conditional, roles) + conditional_declared <- conditional + conditional <- !isFALSE(conditional) + if (conditional && !length(conditioning_cols)) { msg <- paste0( - "conditional = TRUE but no treatment or design column survives to ", - "condition on; numeric synthesis falls back to the global copula." + "A conditional clone was asked for but no treatment or design column ", + "of its partition survives; numeric synthesis falls back to the global ", + "copula." ) # Classed, so a caller can catch the loss of conditional fidelity. warning(warningCondition(msg, class = "masque_conditional_degraded")) @@ -447,7 +484,8 @@ mask <- function(df, result <- with_rng_state( seed, .mask_orchestrate( - df, roles, mode, opts, .shared_maps, conditional, ladder, + df, roles, mode, opts, .shared_maps, conditional, conditioning_cols, + ladder, events = event_specs, grain = grain_res$groups ) ) @@ -554,6 +592,7 @@ mask <- function(df, roles = .strip_roles_provenance(as.data.frame(roles)), conditional = conditional, conditioning_used = conditioning_used, + conditioning_shifted = conditioning_shifted, column_name_map = NULL, level_maps = level_maps, storage_classes = list(), @@ -641,8 +680,22 @@ mask <- function(df, allow_unmasked_coords = isTRUE(allow_unmasked_coords), allow_kept_outcome = allow_kept_outcome, warnings = warnings_acc, + content_fp = .content_fingerprint(df), integrity_fp = digest::digest(is.na(df), algo = "sha256") ) + if (!is.null(plan)) { + rec@owner_key <- .new_owner_key() + if (!length(rec@owner_key)) { + warning(warningCondition( + "No system random source was found, so this recipe holds no key for the owner's controls. Install openssl, or pass a private `seed` to controls().", + class = "masque_owner_key_unavailable" + )) + } + rec@plan <- .plan_resolve(plan, df, roles, rec, list( + coords = coords, events = events, conditional = conditional_declared, + ladder = ladder, grain = grain, alias_names = alias_names + )) + } masque_obj( synthetic = .as_owned_dt(synth), @@ -703,7 +756,7 @@ mask <- function(df, if (length(dropped)) { parts <- c(parts, sprintf( paste0( - "conditional = TRUE: cells of the finest conditioning stratum held ", + "Conditional clone: cells of the finest conditioning stratum held ", "fewer than %d rows, so the ladder coarsened it by dropping %s. ", "Conditioning on: %s." ), @@ -718,7 +771,7 @@ mask <- function(df, shifted <- report$shifted if (length(shifted)) { parts <- c(parts, sprintf( - "The main effect of %s is carried into the clone as an additive shift.", + "The main effect of %s is carried into the clone as an additive shift; its interaction with a kept treatment is not.", paste(shifted, collapse = ", ") )) } @@ -739,7 +792,9 @@ mask <- function(df, # Internal: per-action synthesis, with the RNG already seeded. The action # column decides what happens to each column; mode only adds numeric jitter. .mask_orchestrate <- function(df, roles, mode, opts, shared_maps = list(), - conditional = FALSE, ladder = "hierarchy", + conditional = FALSE, + conditioning_cols = character(), + ladder = "hierarchy", events = list(), grain = list()) { synth <- df level_maps <- list() @@ -772,14 +827,15 @@ mask <- function(df, if (isTRUE(conditional)) { conditional_report <- .conditioning_ladder( df, - cond_cols = .conditioning_cols(roles), + cond_cols = conditioning_cols, protect_cols = roles$col[role == "treatment" & action != "drop"], min_stratum = .MIN_STRATUM, ladder = ladder, x_num = x_num ) shifts <- .design_shifts( - x_num, df, conditional_report$used, conditional_report$shifted + x_num, df, conditional_report$used, conditional_report$shifted, + treatment = roles$col[role == "treatment"] ) shift <- shifts$shift conditional_report$shifted <- shifts$shifted @@ -822,7 +878,9 @@ mask <- function(df, # Linked events: status patterns permuted, gaps re-simulated per pattern. event_reports <- list() if (length(events)) { - trt_cols <- roles$col[role == "treatment" & action != "drop"] + trt_cols <- intersect( + roles$col[role == "treatment" & action != "drop"], conditioning_cols + ) strata <- if (isTRUE(conditional) && length(trt_cols)) { do.call(paste, c(lapply(.cols(df, trt_cols), as.character), sep = "/")) } diff --git a/R/mask_set.R b/R/mask_set.R index 1ab5c72..aad4ebb 100644 --- a/R/mask_set.R +++ b/R/mask_set.R @@ -41,7 +41,7 @@ #' @param alias_names Hide column names. `FALSE` (default) keeps them; #' `TRUE` aliases every non-link column (linked join keys keep their #' names so the synthetic set stays joinable). -#' @param conditional Logical scalar (default `FALSE`). Passed through to +#' @param conditional `FALSE` (default), `TRUE` or `"design"`. Passed through to #' each per-table [mask()] call: when `TRUE`, every table's numeric #' block is re-simulated within its own treatment-by-design strata so #' the treatment-to-outcome relationship survives the clone. See @@ -109,6 +109,12 @@ mask_set <- function(input, .refuse_unused_dots("mask_set", ...) withr::local_preserve_seed() ladder <- match.arg(ladder) + if (is.list(roles) && any(vapply(roles, S7::S7_inherits, logical(1L), .plan_class))) { + .plan_refuse( + "{.fn mask_set} does not take a plan yet. Call {.code mask(df, plan)} on each table.", + "unsupported" + ) + } # Check the shape of `roles` before reading its mode, so a wrong argument # is reported as such. if (!is.null(roles)) .check_roles_shape(roles) diff --git a/R/masque.R b/R/masque.R index 9a28f03..9eaccb4 100644 --- a/R/masque.R +++ b/R/masque.R @@ -60,7 +60,8 @@ #' @param clean Hygiene mode passed to [clean_table()] (`"auto"`, #' `"report"`, `"off"`). #' @param alias_names Hide column names; see [mask()] / [mask_set()]. -#' @param conditional Logical scalar (default `FALSE`). The conditional +#' @param conditional `FALSE` (default), `TRUE`, `"design"` or column names. +#' The conditional #' clone mode passed through to [mask()] / [mask_set()]: when `TRUE`, #' numeric columns are re-simulated within each treatment-by-design #' stratum so the treatment-to-outcome relationship survives the clone. @@ -127,6 +128,12 @@ masque <- function(input, clean <- match.arg(clean) ladder <- match.arg(ladder) + if (S7::S7_inherits(roles, .plan_class)) { + .plan_refuse( + "{.fn masque} does not take a plan yet. Call {.code mask(df, plan)}.", + "unsupported" + ) + } is_set <- .masque_is_set(input) if (!is_set && (!is.null(sheets) || !is.null(info))) { cli::cli_abort(c( diff --git a/R/plan.R b/R/plan.R new file mode 100644 index 0000000..aff11c7 --- /dev/null +++ b/R/plan.R @@ -0,0 +1,1005 @@ +# plan.R -- masque_plan(): the roles table and every named spec in one object. + +.plan_class <- S7::new_class( + "masque_plan", + properties = list( + roles = S7::class_data.frame, + coords = S7::class_list, + events = S7::class_list, + conditional = S7::new_union(S7::class_logical, S7::class_character), + ladder = S7::class_character, + grain = S7::class_logical, + alias_names = S7::new_union(S7::class_logical, S7::class_character), + controls = S7::class_list, + vocabulary = S7::class_character, + mode = S7::new_union(S7::class_character, NULL) + ) +) + +#' A masking plan: roles and named specs in one object +#' +#' Holds what a [mask()] call needs apart from the table: the roles, the +#' coordinate and event declarations, the conditioning choice and any named +#' [control_spec()]s. `mask(df, plan)` reads them from the plan; an argument +#' passed to [mask()] as well overrides the plan's value for that call. +#' +#' At `mask()` each control spec is resolved on the source table and written +#' back: its unit, its strata, the grain of its columns and its effect vector. +#' The clone does not depend on the specs. The resolved plan is on the recipe, +#' and [clone_plan()] gives the same plan in clone vocabulary for the analyst. +#' +#' A spec's `unit`, where not declared, resolves to the coarsest design +#' partition within which `on` is constant, or to the row. Its `strata`, where +#' not declared, resolve to the finest design partition above the unit in which +#' every stratum holds every level of `on`, or to none. A stratum that misses +#' levels of `on` would keep part of the real effect in a negative control, so +#' incomplete blocks are not chosen by default. Declare a whole-plot unit where +#' few plots hold more than one row: it is taken only when a quarter of the rows +#' share a unit. +#' +#' A drawn effect, `list(sd = )`, is drawn over the clone's labels and shows +#' nothing of the source. A declared effect is written to the clone plan as +#' given, so values that rise with a level reveal that level's rank. +#' +#' @param roles A roles table from [propose_roles()]. +#' @param coords,events,conditional,ladder,grain,alias_names As in [mask()]. +#' @param controls `NULL`, one [control_spec()], or a named list of them. A +#' single spec is named `default`. +#' +#' @returns A `masque_plan` object. +#' +#' @examples +#' trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +#' r <- propose_roles(trial, detect = FALSE) +#' r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +#' r <- set_role(r, "gen", role = "treatment", action = "alias") +#' r <- set_role(r, "yield", role = "outcome", action = "scramble") +#' r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +#' p <- masque_plan( +#' r, +#' conditional = "design", +#' controls = control_spec( +#' on = "gen", target = "yield", effect = list(sd = 0.5), n = 5L, seed = 7L +#' ) +#' ) +#' p +#' m <- mask(trial, p, mode = "local", seed = 1L) +#' clone_plan(m) +#' +#' @seealso [mask()], [clone_plan()], [save_plan()], [controls()], [handover()]. +#' +#' @export +masque_plan <- function( + roles, + coords = NULL, + events = NULL, + conditional = FALSE, + ladder = c("hierarchy", "levels"), + grain = TRUE, + alias_names = FALSE, + controls = NULL +) { + if (!is.data.frame(roles)) { + .plan_refuse("`roles` must be a data frame; got {.cls {class(roles)[1]}}.", "roles") + } + need <- setdiff(c("col", "role", "action", "kind"), names(roles)) + if (length(need)) { + .plan_refuse("`roles` lacks column{?s} {.field {need}}.", "roles") + } + .check_conditional_arg(conditional) + ladder <- match.arg(ladder) + if (!is.logical(grain) || length(grain) != 1L || is.na(grain)) { + .plan_refuse("`grain` must be a single `TRUE` or `FALSE`.", "grain") + } + names_ok <- if (is.character(alias_names)) { + !anyNA(alias_names) + } else { + is.logical(alias_names) && length(alias_names) == 1L && !is.na(alias_names) + } + if (!names_ok) { + .plan_refuse("`alias_names` must be `TRUE`, `FALSE` or column names.", "alias_names") + } + .plan_class( + roles = roles, + coords = .plan_wrap(coords, c("lat", "lon")), + events = .plan_wrap(events, c("time", "status")), + conditional = conditional, + ladder = ladder, + grain = grain, + alias_names = alias_names, + controls = .plan_controls(controls), + vocabulary = "source", + mode = attr(roles, "mode") + ) +} + +.plan_refuse <- function(msg, what, .envir = parent.frame()) { + cli::cli_abort( + msg, + class = c(paste0("masque_plan_", what, "_refusal"), "orchestra_refusal"), + .envir = .envir + ) +} + +# One spec, or a list of them, as a list of specs. +.plan_wrap <- function(x, keys) { + if (is.null(x)) { + return(list()) + } + if (is.character(x) && length(names(x))) { + x <- as.list(x) + } + if (is.list(x) && any(keys %in% names(x))) { + x <- list(x) + } + if (!is.list(x)) { + what <- keys[1L] + .plan_refuse( + "A spec needs named entries, as in {.code list({keys[1L]} = , {keys[2L]} = )}.", + if (identical(what, "lat")) "coords" else "events" + ) + } + x +} + +.plan_controls <- function(controls) { + if (is.null(controls)) { + return(list()) + } + if (S7::S7_inherits(controls, masque_control_spec)) { + return(list(default = controls)) + } + nms <- names(controls) + ok <- is.list(controls) && length(controls) >= 1L && !is.null(nms) && + all(nzchar(nms)) && !anyDuplicated(nms) && + all(vapply(controls, S7::S7_inherits, logical(1L), masque_control_spec)) + if (!ok) { + .plan_refuse( + "`controls` must be one {.fn control_spec} or a named list of them.", + "controls" + ) + } + controls +} + +# Print ---------------------------------------------------------------------- + +# local(): see the plot method in detect_plot.R. +local(S7::method(print, .plan_class) <- function(x, ...) { + cli::cli_h1("masque_plan ({x@vocabulary} vocabulary)") + tally <- table(factor( + x@roles$role, + levels = c("design", "treatment", "outcome", "covariate", "date", "id", "ignore") + )) + tally <- tally[tally > 0L] + cli::cli_bullets(c( + "*" = sprintf( + "Roles: %s", + paste(sprintf("%d %s", tally, names(tally)), collapse = ", ") + ), + "*" = sprintf("Conditional: %s", paste(x@conditional, collapse = ", ")), + "*" = sprintf("Ladder: %s; grain: %s", x@ladder, x@grain), + "*" = sprintf("Column names aliased: %s", paste(x@alias_names, collapse = ", ")), + "*" = sprintf("Coordinate pairs: %d", length(x@coords)), + "*" = sprintf("Linked-event specs: %d", length(x@events)), + "*" = sprintf("Control specs: %d", length(x@controls)) + )) + for (nm in names(x@controls)) { + s <- x@controls[[nm]] + cli::cli_text("{.strong {nm}}") + cli::cli_bullets(c( + " " = sprintf("on %s, unit %s", .spec_on_line(s), .spec_cols_line(s@unit, "row")), + " " = sprintf("strata %s", .spec_cols_line(s@strata, "none")), + " " = sprintf("target %s; replicates: %s", .spec_target_line(s), s@n) + )) + } + invisible(x) +}) + +# Resolution on the source, at mask() ---------------------------------------- + +# The plan as it ran, with every spec resolved on the source table. +.plan_resolve <- function(plan, df, roles, rec, args) { + .check_spec_names(names(plan@controls), rec) + specs <- lapply(plan@controls, .plan_resolve_spec, df = df, roles = roles, rec = rec) + resolved_roles <- as.data.frame(rec@roles) + attr(resolved_roles, "mode") <- rec@mode + .plan_class( + roles = resolved_roles, + coords = .plan_wrap(args$coords, c("lat", "lon")), + events = .plan_wrap(args$events, c("time", "status")), + conditional = args$conditional, + ladder = args$ladder, + grain = args$grain, + alias_names = args$alias_names, + controls = specs, + vocabulary = "source", + mode = rec@mode + ) +} + +.plan_resolve_spec <- function(spec, df, roles, rec) { + design <- intersect( + names(df), roles$col[roles$role == "design" & roles$action != "drop"] + ) + on <- .resolve_control_on(spec@on, df, roles, spec@target) + .check_spec_retained(c(on, spec@target), roles) + .check_spec_on_labels(on, df, roles, rec) + if (!is.null(rec@seed) && rec@seed == spec@seed) { + .control_refuse( + c( + "The spec's seed equals the seed of {.fn mask}.", + x = "The analyst reads the spec's seed in the plan, and the mask seed reveals every displacement and map.", + i = "Use two different seeds." + ), + "seed" + ) + } + unit <- if (length(spec@unit)) spec@unit else .default_control_unit(df, on, design) + strata <- if (length(spec@strata)) { + spec@strata + } else { + .default_control_strata(df, on, design, unit) + } + grain <- spec@grain + analysed <- roles$col[ + roles$role %in% c("outcome", "covariate", "date") & + roles$action %in% c("scramble", "alias") + ] + for (g in .prop_or(rec, "grain", list())) { + if (all(g$unit %in% design)) { + for (cl in setdiff(intersect(g$cols, analysed), c(names(grain), spec@target))) { + grain[[cl]] <- g$unit + } + } + } + draft <- masque_control_spec( + on = on, target = spec@target, + effect = .plan_draw_effect(spec, on, df, rec), + strata = strata, unit = unit, grain = grain, n = spec@n, seed = spec@seed + ) + .resolve_control_spec(draft, df, roles)$spec +} + +# A spec's name reaches the analyst as written, so it holds no hidden name. +.check_spec_names <- function(spec_names, rec) { + hidden <- c( + names(rec@column_name_map), + unlist(lapply(rec@level_maps, names), use.names = FALSE) + ) + tokens <- unlist(strsplit(as.character(spec_names), "[^[:alnum:]]+"), use.names = FALSE) + bad <- intersect(tokens, hidden) + if (length(bad)) { + .plan_refuse( + "A control spec's name holds {.val {bad}}, which the clone hides. Rename the spec.", + "controls" + ) + } + invisible(TRUE) +} + +# A column gone from the clone cannot be named in a plan the analyst reads. +.check_spec_retained <- function(cols, roles) { + dropped <- intersect(cols, roles$col[roles$action == "drop"]) + if (length(dropped)) { + .control_refuse( + "A control spec names {.field {dropped}}, with action {.val drop}: it is not in the clone.", + "dropped" + ) + } + invisible(TRUE) +} + +# The levels of a numeric `on` are its real values, unless a map relabels them. +.check_spec_on_labels <- function(on, df, roles, rec) { + action <- roles$action[match(on, roles$col)] + if (is.numeric(df[[on]]) && !identical(action, "keep") && + !length(rec@level_maps[[on]])) { + .control_refuse( + c( + "`on` ({.field {on}}) is re-simulated numeric: its levels in a plan would be real values.", + i = "Give it the action {.val alias} or {.val keep}." + ), + "on" + ) + } + invisible(TRUE) +} + +# A drawn effect is drawn over the sorted clone labels. Drawn over source +# levels, the vector's order and the public seed would give away the map. +.plan_draw_effect <- function(spec, on, df, rec) { + effect <- spec@effect + if (!is.list(effect)) { + return(effect) + } + levels <- .control_levels(df[[on]]) + map <- rec@level_maps[[on]] + if (!length(map)) { + return(.resolve_control_effect(effect, levels, spec@seed)) + } + clone <- as.character(.apply_level_map_forward(levels, map, col = on)) + draw <- .resolve_control_effect(effect, sort(clone, method = "radix"), spec@seed) + stats::setNames(unname(draw[clone]), levels) +} + +# Coarsest design column, then pair, within which `on` is constant and whose +# units can reach a 5 percent level. A row alone is constant by construction, so +# a quarter of the rows must share a unit for the constancy to mean anything. +.default_control_unit <- function(df, on, design) { + n <- nrow(df) + too_few <- character() + sets <- c(as.list(design), if (length(design) >= 2L) { + utils::combn(design, 2L, simplify = FALSE) + }) + for (size in 1:2) { + pick <- Filter(function(s) length(s) == size, sets) + n_units <- vapply(pick, function(s) { + id <- as.integer(.unit_id(df, s)) + sizes <- tabulate(id) + shared <- sum(sizes[sizes >= 2L]) + if (shared < n / 4 || !.constant_within(id, df[[on]])) { + return(NA_integer_) + } + if (!.level_attainable(id, df[[on]], rep(1L, n))) { + too_few <<- c(too_few, paste(s, collapse = " + ")) + return(NA_integer_) + } + max(id) + }, integer(1L)) + if (any(!is.na(n_units))) { + chosen <- pick[[which.min(n_units)]] + if (sum(!is.na(n_units)) > 1L) { + warning(warningCondition( + sprintf( + "`%s` is constant within several design columns (%s). The default unit is %s, the coarsest. Name the unit `%s` was assigned at in `unit`: a finer one hides pseudoreplication in the negative controls.", + on, + paste(vapply(pick[!is.na(n_units)], paste, character(1L), collapse = " + "), collapse = ", "), + paste(chosen, collapse = " + "), on + ), + class = "masque_control_unit_ambiguous" + )) + } + return(chosen) + } + } + if (length(too_few)) { + warning(warningCondition( + sprintf( + "`%s` is constant within %s, which has too few units for a 5 percent randomisation level. The default unit is the row; name a `unit` to choose otherwise.", + on, paste(too_few, collapse = ", ") + ), + class = "masque_control_unit_small" + )) + } + character() +} + +# Finest design partition above the unit whose every stratum holds every level. +.default_control_strata <- function(df, on, design, unit) { + unit_id <- if (length(unit)) as.integer(.unit_id(df, unit)) else seq_len(nrow(df)) + all_levels <- length(unique(df[[on]][!is.na(df[[on]])])) + usable <- function(cols) { + id <- as.integer(.unit_id(df, cols)) + if (max(id) < 2L || !.constant_within(unit_id, id)) { + return(FALSE) + } + held <- tapply(df[[on]], id, function(v) length(unique(v[!is.na(v)]))) + all(held == all_levels) + } + passing <- Filter(usable, setdiff(design, c(unit, on))) + if (!length(passing)) { + return(character()) + } + if (usable(passing)) { + return(passing) + } + sizes <- vapply(passing, function(cl) max(as.integer(.unit_id(df, cl))), integer(1L)) + passing[which.max(sizes)] +} + +# Translation between source and clone vocabulary ---------------------------- + +.plan_col_translator <- function(rec, to) { + map <- rec@column_name_map + from_to <- if (length(map)) { + if (identical(to, "clone")) { + unlist(map) + } else { + stats::setNames(names(map), unlist(map)) + } + } else { + character() + } + function(x) { + if (!length(x)) { + return(x) + } + hits <- x %in% names(from_to) + x[hits] <- unname(from_to[x[hits]]) + x + } +} + +.plan_spec_translate <- function(spec, rec, to) { + rename <- .plan_col_translator(rec, to) + on_source <- if (identical(to, "clone")) spec@on else .plan_col_translator(rec, "source")(spec@on) + effect <- spec@effect + map <- rec@level_maps[[on_source]] + if (is.numeric(effect) && length(map)) { + relabel <- if (identical(to, "clone")) { + .apply_level_map_forward + } else { + .apply_level_map_inverse + } + names(effect) <- as.character(relabel(names(effect), map, col = on_source)) + } + # Clone-label order on the way out, so the vector's order carries no map. + if (is.numeric(effect)) { + effect <- effect[order(names(effect), method = "radix")] + } + grain <- lapply(spec@grain, rename) + names(grain) <- rename(names(grain)) + masque_control_spec( + on = rename(spec@on), + target = if (is.null(spec@target)) NULL else rename(spec@target), + effect = effect, + strata = rename(spec@strata), + unit = rename(spec@unit), + grain = grain, + n = spec@n, + seed = spec@seed + ) +} + +#' The plan in clone vocabulary, for the analyst +#' +#' The plan of a [mask()] result, in the names and labels of the clone. It +#' carries the roles as [roles()] gives them, the linked events and each control +#' spec as resolved on the source. It holds no coordinate declaration, and +#' nothing else from the recipe. With [controls()] it gives the analyst the +#' pre-declared controls on the clone. +#' +#' @param m A `masque` object from [mask()]. A clone masked without a plan gives +#' a plan with no control specs. +#' +#' @returns A `masque_plan` in clone vocabulary. +#' +#' @examples +#' r <- propose_roles(iris, detect = FALSE) +#' clone_plan(mask(iris, r, mode = "local", seed = 1L)) +#' +#' @seealso [masque_plan()], [controls()], [handover()]. +#' +#' @export +clone_plan <- function(m) { + if (!S7::S7_inherits(m, masque_obj)) { + cli::cli_abort( + "`m` must be a {.cls masque} object; got {.cls {class(m)[1]}}.", + class = c("masque_bad_masque_obj_refusal", "orchestra_refusal") + ) + } + rec <- m@recipe + source_plan <- .prop_or(rec, "plan", NULL) + public <- as.data.frame(.public_roles(rec, m@synthetic)) + attr(public, "mode") <- rec@mode + rename <- .plan_col_translator(rec, "clone") + conditional <- rec@conditional + grain <- TRUE + specs <- list() + if (!is.null(source_plan)) { + conditional <- source_plan@conditional + if (is.character(conditional) && !identical(conditional, "design")) { + conditional <- rename(conditional) + } + grain <- source_plan@grain + specs <- lapply(source_plan@controls, .plan_spec_translate, rec = rec, to = "clone") + .check_clone_vocabulary(specs, names(m@synthetic)) + } + .plan_class( + roles = public, + coords = list(), + events = .card_event_specs(rec), + conditional = conditional, + ladder = .prop_or(rec, "ladder", "levels"), + grain = grain, + alias_names = FALSE, + controls = specs, + vocabulary = "clone", + mode = rec@mode + ) +} + +# A spec bound for the analyst names clone columns only. +.check_clone_vocabulary <- function(specs, clone_cols) { + named <- unlist(lapply(specs, function(s) { + c(s@on, s@target, s@strata, s@unit, names(s@grain), unlist(s@grain)) + })) + if (!all(named %in% clone_cols)) { + .plan_refuse( + "A control spec names a column outside the clone, so no clone plan is written.", + "vocabulary" + ) + } + invisible(TRUE) +} + +# controls() from a plan ------------------------------------------------------ + +# Offset that roots a drawn effect away from the spec's seed. +.CONTROL_SALT <- 15485863L + +# 256 bits from the operating system, or NULL where neither source exists. +.system_entropy <- function() { + if (file.exists("/dev/urandom")) { + con <- file("/dev/urandom", "rb", raw = TRUE) + on.exit(close(con)) + return(readBin(con, "raw", 32L)) + } + if (requireNamespace("openssl", quietly = TRUE)) { + return(openssl::rand_bytes(32L)) + } + NULL +} + +# Not from the R stream: a seeded mask() must not determine it. Time, process +# id and a temporary name can be enumerated, so they do not serve as a key. +.new_owner_key <- function() { + bytes <- .system_entropy() + if (is.null(bytes)) { + return(character()) + } + paste(as.character(bytes), collapse = "") +} + +# Arguments of controls() from a plan, on either side of the boundary. +.controls_from_plan <- function(df, plan, spec, events, seed, recipe) { + picked <- .plan_pick_spec(plan, spec) + if (is.null(recipe)) { + if (!identical(plan@vocabulary, "clone")) { + .control_refuse( + c( + "This plan is in source vocabulary, and no `recipe` was given.", + i = "The analyst uses {.fn clone_plan}; the owner passes `recipe =`, which draws from a private seed." + ), + "source_plan" + ) + } + return(list( + df = .blank_on_to_na(df, picked$spec), + roles = plan@roles, + spec = picked$spec, + events = if (is.null(events) && length(plan@events)) plan@events else events, + seed = seed, + key = character(), + notes = character() + )) + } + if (!S7::S7_inherits(recipe, masque_recipe)) { + .control_refuse("`recipe` must be a {.cls masque_recipe}; got {.cls {class(recipe)[1]}}.", "recipe") + } + source_spec <- if (identical(plan@vocabulary, "clone")) { + .plan_spec_translate(picked$spec, recipe, "source") + } else { + picked$spec + } + .check_spec_as_declared(source_spec, picked$name, recipe) + draw <- .control_source_draw(.prop_or(recipe, "owner_key", character()), source_spec@seed, seed) + list( + df = .controls_replay_source(df, recipe), + roles = as.data.frame(recipe@roles), + spec = source_spec, + events = if (is.null(events) && length(recipe@events)) { + lapply(recipe@events, function(e) list(time = e$time, status = e$status)) + } else { + events + }, + seed = draw$seed, + key = draw$key, + notes = "Source side: the draw is private. Nothing from a single replicate leaves. The permutation p-value is the result of the test on the real table, and the rate at which the positives find the shift reflects its residual variance: check both as results." + ) +} + +# A clone read back from CSV holds "" where `on` held NA. A plan whose effect +# names "" as a level has a real blank label, and it stays. +.blank_on_to_na <- function(df, spec) { + .check_control_table(df) + df <- .as_owned_dt(df) + on <- spec@on + if (is.null(on) || !on %in% names(df) || "" %in% names(spec@effect)) { + return(df) + } + x <- df[[on]] + if (is.factor(x)) { + levels(x)[levels(x) == ""] <- NA + data.table::set(df, j = on, value = x) + } else if (is.character(x)) { + data.table::set(df, which(x == ""), on, NA_character_) + } + df +} + +.plan_pick_spec <- function(plan, spec) { + specs <- plan@controls + if (!length(specs)) { + .control_refuse("This plan holds no control spec.", "spec") + } + if (is.null(spec)) { + if (length(specs) != 1L) { + nms <- names(specs) + .control_refuse( + "This plan holds {length(specs)} control specs; name one in `spec`: {.val {nms}}.", + "spec" + ) + } + return(list(name = names(specs), spec = specs[[1L]])) + } + if (!is.character(spec) || length(spec) != 1L || !spec %in% names(specs)) { + nms <- names(specs) + .control_refuse("With a plan, `spec` names one of its specs: {.val {nms}}.", "spec") + } + list(name = spec, spec = specs[[spec]]) +} + +# A spec fixed in the plan before any real result exists stays fixed. +.check_spec_as_declared <- function(spec, name, recipe) { + declared <- .prop_or(recipe, "plan", NULL) + if (is.null(declared)) { + return(invisible(TRUE)) + } + on_file <- declared@controls[[name]] + if (!is.null(on_file) && is.numeric(on_file@effect)) { + on_file@effect <- on_file@effect[order(names(on_file@effect), method = "radix")] + } + if (is.numeric(spec@effect)) { + spec@effect <- spec@effect[order(names(spec@effect), method = "radix")] + } + same <- !is.null(on_file) && all(vapply( + c("on", "target", "effect", "strata", "unit", "grain", "n", "seed"), + function(p) isTRUE(all.equal(S7::prop(spec, p), S7::prop(on_file, p))), + logical(1L) + )) + if (!same) { + .control_refuse( + c( + "Control spec {.val {name}} differs from the one fixed at {.fn mask}.", + i = "A control is declared before any real result exists, so the owner runs the spec on the recipe." + ), + "spec_changed" + ) + } + invisible(TRUE) +} + +# The table the clone was built on: cleaning and conformance replayed, the +# schema and NA mask checked. +.controls_replay_source <- function(df, recipe) { + .check_control_table(df) + df <- .apply_cleaning_forward(df, recipe@cleaning) + df <- .as_owned_dt(.apply_conform_forward(df, .prop_or(recipe, "conform", NULL))) + retained <- setdiff(recipe@roles$col, .recipe_dropped_cols(recipe)) + gone <- setdiff(retained, names(df)) + if (length(gone)) { + .control_refuse( + "The table lacks column{?s} the recipe was built on: {.field {gone}}.", + "mismatch" + ) + } + .check_recipe_integrity(df, recipe) + df +} + +# The owner's draw: an explicit integer seed, or the whole digest of the +# recipe's key. An integer can be searched against a released statistic. +.control_source_draw <- function(owner_key, spec_seed, seed) { + if (!is.null(seed)) { + .check_seed(seed) + if (as.integer(seed) == spec_seed) { + .control_refuse( + "`seed` equals the spec's seed, which the analyst holds. The source side needs a private seed.", + "seed" + ) + } + return(list(seed = as.integer(seed), key = character())) + } + if (!length(owner_key)) { + .control_refuse( + "This recipe holds no key: it was masked without a plan, or where no system random source was available. Pass a private `seed`.", + "seed" + ) + } + list( + seed = NULL, + key = digest::digest(paste(owner_key, spec_seed), algo = "sha256", serialize = FALSE) + ) +} + +# YAML ------------------------------------------------------------------------ + +#' Save and read a masking plan as YAML +#' +#' A plan on disk can be read and compared as text. `save_plan()` needs the +#' `yaml` package. A custom `on_land` function cannot be written, and is refused. +#' A source-vocabulary plan names real columns and labels, so it stays with the +#' owner. The analyst's file comes from [clone_plan()] or [handover()]. +#' +#' @param plan A [masque_plan()]. +#' @param path Path of a `.yaml` or `.yml` file. +#' +#' @returns `save_plan()` returns `path` invisibly; `read_plan()` a `masque_plan`. +#' +#' @examples +#' if (requireNamespace("yaml", quietly = TRUE)) { +#' r <- propose_roles(iris, detect = FALSE) +#' f <- tempfile(fileext = ".yaml") +#' save_plan(masque_plan(r), f) +#' read_plan(f) +#' } +#' +#' @export +save_plan <- function(plan, path) { + if (!S7::S7_inherits(plan, .plan_class)) { + .plan_refuse("`plan` must be a {.cls masque_plan}; got {.cls {class(plan)[1]}}.", "class") + } + .check_plan_path(path) + .check_yaml_available() + yaml::write_yaml(.plan_to_list(plan), path, precision = 17L) + invisible(path) +} + +#' @rdname save_plan +#' @export +read_plan <- function(path) { + .check_plan_path(path) + .check_yaml_available() + if (!file.exists(path)) { + .plan_refuse("File not found: {.file {path}}.", "file") + } + x <- tryCatch( + yaml::read_yaml(path), + error = function(e) { + .plan_refuse("{.file {path}} is not valid YAML: {conditionMessage(e)}", "file") + } + ) + if (!is.list(x) || !identical(as.integer(x$masque_plan), 1L)) { + .plan_refuse("{.file {path}} is not a saved masque plan.", "file") + } + .plan_from_list(x) +} + +.check_plan_path <- function(path) { + ok <- is.character(path) && length(path) == 1L && !is.na(path) && + grepl("\\.ya?ml$", path, ignore.case = TRUE) + if (!ok) { + .plan_refuse("`path` must be a single path ending in {.file .yaml} or {.file .yml}.", "file") + } +} + +.check_yaml_available <- function() { + if (!requireNamespace("yaml", quietly = TRUE)) { + .plan_refuse("Writing and reading a plan needs {.pkg yaml}.", "yaml") + } +} + +.plan_to_list <- function(plan) { + r <- plan@roles + coords <- lapply(plan@coords, function(s) { + s <- as.list(s) + if (any(vapply(s, is.function, logical(1L)))) { + .plan_refuse("A coordinate spec with an `on_land` function cannot be written to a file.", "file") + } + s + }) + list( + masque_plan = 1L, + vocabulary = plan@vocabulary, + mode = plan@mode, + conditional = as.list(plan@conditional), + ladder = plan@ladder, + grain = plan@grain, + alias_names = as.list(plan@alias_names), + roles = lapply(seq_len(nrow(r)), function(i) { + list(col = r$col[i], role = r$role[i], action = r$action[i], kind = r$kind[i]) + }), + coords = coords, + events = lapply(plan@events, function(e) { + list(time = as.list(e$time), status = as.list(e$status)) + }), + controls = lapply(plan@controls, .spec_to_list) + ) +} + +.spec_to_list <- function(s) { + effect <- s@effect + list( + on = s@on, + target = as.list(s@target), + effect_sd = if (is.list(effect)) effect$sd, + effect = if (is.numeric(effect)) { + lapply(seq_along(effect), function(i) { + list(level = names(effect)[i], shift = effect[[i]]) + }) + }, + strata = as.list(s@strata), + unit = as.list(s@unit), + grain = lapply(seq_along(s@grain), function(i) { + list(col = names(s@grain)[i], unit = as.list(s@grain[[i]])) + }), + n = s@n, + seed = s@seed + ) +} + +.chr <- function(x) as.character(unlist(x)) + +.plan_from_list <- function(x) { + roles <- data.frame( + col = vapply(x$roles, function(r) as.character(r$col), character(1L)), + role = vapply(x$roles, function(r) as.character(r$role), character(1L)), + action = vapply(x$roles, function(r) as.character(r$action), character(1L)), + kind = vapply(x$roles, function(r) as.character(r$kind), character(1L)), + stringsAsFactors = FALSE + ) + attr(roles, "mode") <- x$mode + .plan_class( + roles = roles, + coords = lapply(x$coords, as.list), + events = lapply(x$events, function(e) { + list(time = .chr(e$time), status = .chr(e$status)) + }), + conditional = unlist(x$conditional), + ladder = as.character(x$ladder), + grain = isTRUE(x$grain), + alias_names = unlist(x$alias_names), + controls = lapply(x$controls, .spec_from_list), + vocabulary = as.character(x$vocabulary), + mode = x$mode + ) +} + +.spec_from_list <- function(s) { + effect <- if (!is.null(s$effect_sd)) { + list(sd = as.double(s$effect_sd)) + } else if (length(s$effect)) { + stats::setNames( + vapply(s$effect, function(e) as.double(e$shift), numeric(1L)), + vapply(s$effect, function(e) as.character(e$level), character(1L)) + ) + } + grain <- lapply(s$grain, function(g) .chr(g$unit)) + names(grain) <- vapply(s$grain, function(g) as.character(g$col), character(1L)) + masque_control_spec( + on = if (is.null(s$on)) NULL else as.character(s$on), + target = if (length(s$target)) .chr(s$target), + effect = effect, + strata = .chr(s$strata), + unit = .chr(s$unit), + grain = if (length(grain)) grain else list(), + n = as.integer(s$n), + seed = as.integer(s$seed) + ) +} + +# Handover -------------------------------------------------------------------- + +#' Hand a clone to an analyst +#' +#' Writes what the analyst works from into one folder: the clone (`clone.csv`), +#' the plan in clone vocabulary (`plan.yaml`), the data card (`card.yaml`) and a +#' plain note (`README.txt`). The recipe is never written. The release gate runs +#' first, as for [write_set()]: an unresolved HIGH finding blocks the write +#' unless `allow_high = TRUE`. A local-mode clone has not been audited and is +#' meant to stay with the owner, so it is refused unless `allow_local = TRUE`. +#' +#' `clone.csv` loses column classes. Read label columns as text where a label +#' looks like a number, or `controls()` will not find its levels. +#' +#' @param m A `masque` object from [mask()]. +#' @param dir The folder to write. An existing folder that holds files is +#' refused unless `overwrite = TRUE`. +#' @param name Dataset name for the data card; `NULL` uses `"clone"`. It is +#' written as given, so it should say nothing the clone hides. +#' @param overwrite Replace the four files if they exist. +#' @param allow_high Write despite HIGH findings, with a warning. +#' @param allow_local Write a local-mode clone, for a folder the owner keeps. +#' +#' @returns The paths written, invisibly. +#' +#' @examples +#' if (requireNamespace("yaml", quietly = TRUE)) { +#' r <- propose_roles(iris, mode = "collaborate", detect = FALSE) +#' m <- mask(iris, r, mode = "collaborate", seed = 1L) +#' d <- tempfile() +#' handover(m, d) +#' list.files(d) +#' } +#' +#' @seealso [clone_plan()], [data_card()], [controls()]. +#' +#' @export +handover <- function(m, dir, name = NULL, overwrite = FALSE, allow_high = FALSE, + allow_local = FALSE) { + if (!S7::S7_inherits(m, masque_obj)) { + cli::cli_abort( + "`m` must be a {.cls masque} object; got {.cls {class(m)[1]}}.", + class = c("masque_bad_masque_obj_refusal", "orchestra_refusal") + ) + } + if (!is.character(dir) || length(dir) != 1L || is.na(dir) || !nzchar(dir)) { + cli::cli_abort( + "`dir` must be a single non-empty string.", + class = c("masque_bad_handover_dir_refusal", "orchestra_refusal") + ) + } + .check_yaml_available() + if (!identical(m@mode, "collaborate") && !isTRUE(allow_local)) { + cli::cli_abort( + c( + "This clone was masked in local mode: it has no audit and keeps real labels.", + i = "Mask with {.code mode = \"collaborate\"} to hand it over, or pass `allow_local = TRUE` for a folder you keep." + ), + class = c("masque_handover_local_refusal", "orchestra_refusal") + ) + } + high <- .gate_release(m, allow_high, "the findings are named in README.txt") + if (dir.exists(dir) && length(list.files(dir, all.files = TRUE, no.. = TRUE)) && + !isTRUE(overwrite)) { + cli::cli_abort( + c( + "{.file {dir}} already holds files.", + i = "Choose an empty folder, or pass `overwrite = TRUE`." + ), + class = c("masque_handover_exists_refusal", "orchestra_refusal") + ) + } + dir.create(dir, recursive = TRUE, showWarnings = FALSE) + + paths <- file.path(dir, c("clone.csv", "plan.yaml", "card.yaml", "README.txt")) + data.table::fwrite(m@synthetic, paths[1L]) + save_plan(clone_plan(m), paths[2L]) + data_card(m, file = paths[3L], name = name) + writeLines(.handover_note(m, high), paths[4L]) + invisible(paths) +} + +# Finding names in the clone's vocabulary: the note goes to the analyst. +.handover_high_names <- function(m, high) { + .plan_col_translator(m@recipe, "clone")(high) +} + +.handover_note <- function(m, high) { + plan <- clone_plan(m) + r <- plan@roles + kept <- r$col[r$role == "outcome" & r$action == "keep"] + c( + "Masked clone for pipeline development", + "", + sprintf( + "clone.csv holds %d rows and %d columns. An empty cell is a missing value.", + nrow(m@synthetic), ncol(m@synthetic) + ), + "plan.yaml holds the column roles and any declared controls, in the clone's names.", + "card.yaml describes the table for downstream tools.", + "", + "Re-simulated columns hold no real values. Do not read an effect or an estimate from this table.", + if (length(kept)) { + sprintf("These outcome columns hold real values: %s.", paste(kept, collapse = ", ")) + }, + if (!identical(m@mode, "collaborate")) { + "This is a local-mode clone. It keeps real labels and stays with the data owner." + }, + "Write and test your code here. The data owner runs the finished code on the real table.", + if (length(plan@controls)) { + paste0( + "In R, read_plan(\"plan.yaml\") and controls(clone, plan) rebuild the declared ", + "known-truth controls." + ) + }, + if (length(high)) { + sprintf( + "The owner released this clone after reviewing HIGH leakage findings on: %s.", + paste(.handover_high_names(m, high), collapse = ", ") + ) + } + ) +} diff --git a/R/recipe.R b/R/recipe.R index c5b7236..e382205 100644 --- a/R/recipe.R +++ b/R/recipe.R @@ -88,9 +88,21 @@ masque_recipe <- S7::new_class( allow_kept_outcome = S7::new_property( class = S7::class_logical, default = FALSE ), + # Fresh entropy behind the owner's control seeds, which no seed can guess. + # Empty without a plan. Absent before 0.17.0. + owner_key = S7::new_property( + class = S7::class_character, default = character() + ), + # The plan as it ran, specs resolved on the source; NULL without a plan. + # Absent before 0.17.0. + plan = S7::new_property( + class = S7::new_union(NULL, .plan_class), default = NULL + ), warnings = S7::new_property( class = S7::class_character, default = character() ), + # A fingerprint of the values, beside the NA mask's. Absent before 0.17.0. + content_fp = S7::new_property(class = S7::class_list, default = list()), integrity_fp = S7::class_character ) ) diff --git a/R/recipe_apply.R b/R/recipe_apply.R index 48f0730..2ec21eb 100644 --- a/R/recipe_apply.R +++ b/R/recipe_apply.R @@ -34,7 +34,8 @@ #' @param rec A `masque_recipe` object (e.g. from `recipe(m)`). #' @param check_integrity Logical. When `TRUE` (default), verifies that #' the NA mask of `original` matches the recipe's recorded -#' `integrity_fp`. Mismatches error with guidance. Pass `FALSE` to +#' `integrity_fp` and values against the recipe's fingerprint of them, +#' numbers within rounding from a file. Mismatches error with guidance. Pass `FALSE` to #' bypass when the missingness has legitimately changed since the #' recipe was built. #' diff --git a/R/recipe_io.R b/R/recipe_io.R index 573ca75..0e7a2b3 100644 --- a/R/recipe_io.R +++ b/R/recipe_io.R @@ -3,8 +3,8 @@ #' Writes the recipe to a single `.rds` file. The default is #' **runtime-minimal**: #' no simulator state (copula covariance, raw margins) is written, only the -#' translation maps, factor metadata, storage classes, integrity fingerprint, -#' and warnings. This keeps the saved artefact small and reduces the +#' translation maps, factor metadata, storage classes, integrity fingerprints, +#' the key behind the owner's controls, and warnings. This keeps the saved artefact small and reduces the #' information that would leak if the recipe file alone were shared. #' #' `include_simulator = TRUE` is accepted and has no effect: the recipe diff --git a/R/recipe_print.R b/R/recipe_print.R index c993fc4..57957c4 100644 --- a/R/recipe_print.R +++ b/R/recipe_print.R @@ -206,6 +206,13 @@ local({ str <- utils::str S7::method(str, masque_recipe) <- .str_redacted S7::method(str, masque_obj) <- .str_redacted + S7::method(str, masque_controls) <- function(object, ...) { + cat(sprintf( + " %d rows, %d replicates; details not shown. See print().\n", + nrow(object@table), object@spec@n + )) + invisible(NULL) + } }) @@ -367,10 +374,21 @@ reveal_maps <- function(rec, table = NULL) { frac <- NA_real_ } if (!length(used)) { - return("conditional requested (no conditioning column; pooled fallback)") + shifted <- .prop_or(x, "conditioning_shifted", character()) + return(if (length(shifted)) { + sprintf("conditional: pooled copula with additive shifts on %s", paste(shifted, collapse = ", ")) + } else { + "conditional requested (no conditioning column; pooled fallback)" + }) } + r <- x@roles + has_treatment <- any(requested %in% r$col[r$role == "treatment"]) line <- sprintf( - "conditional (treatment -> outcome preserved; strata: %s)", + if (has_treatment) { + "conditional (treatment -> outcome preserved; strata: %s)" + } else { + "design-conditioned (treatment effect not carried; strata: %s)" + }, paste(used, collapse = ", ") ) dropped <- setdiff(requested, used) diff --git a/R/release_gate.R b/R/release_gate.R index 53d2dee..09d74a7 100644 --- a/R/release_gate.R +++ b/R/release_gate.R @@ -92,7 +92,8 @@ # Gate a package-managed write: abort ("masque_blocked_write") on unresolved # HIGH findings, or warn ("masque_high_override") under `allow_high = TRUE`. -.gate_release <- function(m, allow_high = FALSE) { +.gate_release <- function(m, allow_high = FALSE, + record = "the override is recorded on the recipe") { high <- if (identical(m@mode, "collaborate")) { .audit_high_cols(m) } else { @@ -113,7 +114,7 @@ i = "Re-role, alias, or drop each flagged column, then mask again.", i = paste0( "Or pass `allow_high = TRUE` to write anyway after your own ", - "review - the override is recorded." + "review - {record}." ) ), class = "masque_blocked_write" diff --git a/R/roles_actions.R b/R/roles_actions.R index d5fbc9a..9f0b1c7 100644 --- a/R/roles_actions.R +++ b/R/roles_actions.R @@ -138,6 +138,51 @@ set_role <- function(roles, cols, role = NULL, action = NULL) { roles$col[keep] } +# `conditional` is one flag, the word "design", or column names. +.check_conditional_arg <- function(conditional) { + ok <- if (is.character(conditional)) { + length(conditional) >= 1L && !anyNA(conditional) && + all(nzchar(conditional)) && !anyDuplicated(conditional) + } else { + is.logical(conditional) && length(conditional) == 1L && !is.na(conditional) + } + if (ok) { + return(invisible(conditional)) + } + cli::cli_abort( + paste0( + "`conditional` must be `TRUE`, `FALSE`, {.val design} or a character ", + "vector of retained design and treatment column names." + ), + class = c("masque_bad_conditional_refusal", "orchestra_refusal") + ) +} + +# The columns a conditional clone conditions on, before the ladder. +.resolve_conditioning <- function(conditional, roles) { + if (isFALSE(conditional)) { + return(character()) + } + retained <- .conditioning_cols(roles) + if (isTRUE(conditional)) { + return(retained) + } + if (identical(conditional, "design")) { + return(intersect(retained, roles$col[roles$role == "design"])) + } + bad <- setdiff(conditional, retained) + if (length(bad)) { + cli::cli_abort( + c( + "`conditional` names {.field {bad}}.", + x = "Only retained design and treatment columns can define a partition." + ), + class = c("masque_bad_conditional_refusal", "orchestra_refusal") + ) + } + conditional +} + # Per-mode default action for a (role, kind) pair, used by propose_roles(), # set_role(), the v1 upgrade and NA-action resolution. .default_action <- function(role, kind, mode) { diff --git a/R/synthesise_numeric_conditional.R b/R/synthesise_numeric_conditional.R index 070375d..af95c00 100644 --- a/R/synthesise_numeric_conditional.R +++ b/R/synthesise_numeric_conditional.R @@ -218,13 +218,14 @@ synthesise_numeric_conditional <- function(x_num, groups, min_stratum = 5L) { }, numeric(1)) } -# A column can carry a shift when two or more levels hold two or more rows; -# single-row levels are pooled by `.collapse_singletons()`. -.shiftable <- function(v) { +# A column can carry a shift when two or more levels reach the floor; smaller +# levels are pooled by `.collapse_singletons()`. +.shiftable <- function(v, floor = 2L) { tab <- table(as.character(v)) - sum(tab >= 2L) >= 2L + sum(tab >= floor) >= 2L } + # Pairs of dropped blocking and environment columns, as "a:b" (rep R1 in one # county is not R1 in another). Coordinates form no pairs. .nested_pairs <- function(cols) { @@ -245,12 +246,16 @@ synthesise_numeric_conditional <- function(x_num, groups, min_stratum = 5L) { interaction(lapply(.cols(df, parts), as.character), drop = TRUE, sep = ":") } -# Levels with a single row share one pooled level, so the shift a row -# receives is never that row's own outcome. -.collapse_singletons <- function(v) { +# Levels below the floor share one pooled level, so the shift a row receives is +# never that row's own outcome. A pool still below the floor gets no shift. +.collapse_singletons <- function(v, floor = 2L) { v <- as.character(v) tab <- table(v) - single <- names(tab)[tab < 2L] + single <- names(tab)[tab < floor] + if (sum(tab[single]) < floor) { + v[v %in% single] <- NA_character_ + return(factor(v)) + } v[v %in% single] <- ".singleton" factor(v) } @@ -271,7 +276,7 @@ synthesise_numeric_conditional <- function(x_num, groups, min_stratum = 5L) { #' that restores the error variance the fit removed (`p` counts the stratum #' and each term's degrees of freedom, weighted by its shrinkage). #' @noRd -.design_shifts <- function(x_num, df, used, shifted) { +.design_shifts <- function(x_num, df, used, shifted, treatment = character()) { shift <- matrix( 0, nrow(x_num), ncol(x_num), dimnames = list(NULL, names(x_num)) ) @@ -286,7 +291,14 @@ synthesise_numeric_conditional <- function(x_num, groups, min_stratum = 5L) { with_stratum <- nlevels(key) >= 2L && !any(used %in% shifted) terms <- lapply(shifted, function(term) .shift_term(df, term)) names(terms) <- shifted - shifted <- shifted[vapply(terms, .shiftable, logical(1L))] + # A design level of two rows gives a shift close to its rows' own values. A + # level of three or four can too; the audit grades that, since real alpha blocks + # are this small. A treatment keeps a floor of two: its means are the point. + floors <- ifelse(shifted %in% treatment, 2L, 3L) + names(floors) <- shifted + shifted <- shifted[vapply(shifted, function(term) { + .shiftable(terms[[term]], floors[[term]]) + }, logical(1L))] n_fit <- min(vapply(x_num, function(y) sum(!is.na(y)), integer(1L))) # Over the parameter budget, pairs go first (last pair first), then # columns in the ladder's drop order. @@ -294,7 +306,10 @@ synthesise_numeric_conditional <- function(x_num, groups, min_stratum = 5L) { if (!length(shifted)) { return(none) } - rhs <- lapply(terms[shifted], .collapse_singletons) + rhs <- lapply(shifted, function(term) { + .collapse_singletons(terms[[term]], floors[[term]]) + }) + names(rhs) <- shifted if (with_stratum) { rhs <- c(list(.stratum = key), rhs) } diff --git a/R/synthesise_numeric_jitter.R b/R/synthesise_numeric_jitter.R index 4d64820..37c44b4 100644 --- a/R/synthesise_numeric_jitter.R +++ b/R/synthesise_numeric_jitter.R @@ -8,7 +8,8 @@ #' clone; a value can fall up to half a resolution step outside it. #' Whole-numbered columns (integer class, or a double whose observed values #' are all whole) get stochastic rounding, so the output stays -#' whole-numbered in its original storage class. A column with at most two +#' whole-numbered in its original storage class and can fall one full step +#' outside the range. A column with at most two #' distinct observed values is #' left as drawn: the local copula draw already returns observed values, #' and jitter followed by rounding would flip a share of them. diff --git a/R/write_set.R b/R/write_set.R index a3dfb98..b186591 100644 --- a/R/write_set.R +++ b/R/write_set.R @@ -47,7 +47,10 @@ write_set <- function(m, path, overwrite = FALSE, allow_high = FALSE) { if (!is.character(path) || length(path) != 1L || !nzchar(path)) { cli::cli_abort("`path` must be a single non-empty string.") } - .gate_release(m, allow_high) + .gate_release( + m, allow_high, + "the override is only in that warning, so keep your own record" + ) .write_set_dispatch(m@synthetic, path, overwrite) invisible(path) } diff --git a/README.md b/README.md index 04d36e8..7148d36 100644 --- a/README.md +++ b/README.md @@ -98,6 +98,30 @@ real data in scientific inference. --- +### Known-truth controls + +A pipeline developed on the clone can be checked against datasets in which +the answer is known. A control spec, declared once, names the column to +reassign, the design strata it is permuted within, the outcome an +effect is planted on and the size of that effect. From it, `controls()` +derives a null (every analysed column permuted), negative replicates (one +column permuted within strata) and positive replicates (the negative plus +the planted shift), by permutation alone. The same spec runs on the clone +for the analyst and, with a private seed, on the original for the owner, +who returns a verdict about the pipeline rather than a dataset. + +```r +spec <- control_spec(on = "gen", target = "yield", effect = list(sd = 0.5), + strata = "rep", n = 19L, seed = 2026L) +ctl <- controls(synthetic(m), roles(m), spec) +control_table(ctl, "positive", 1L) +``` + +A `masque_plan()` holds the roles and the spec together, so the spec is fixed +before any real result exists. `mask(df, plan)` resolves it on the source, +`handover()` gives the analyst the clone and the plan in clone vocabulary, and +`controls(real, plan, recipe = rec)` runs it on the owner's side. + ## Threat model `masque` is **not** a privacy-preserving or differential-privacy tool. It is a diff --git a/_pkgdown.yml b/_pkgdown.yml index bf60104..6f53478 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -55,6 +55,21 @@ reference: contents: - audit_mask +- title: Known-truth controls + desc: Null, negative and positive control datasets derived from a table by a declared spec. + contents: + - control_spec + - controls + - control_table + +- title: Plan and handover + desc: Roles and named specs in one object, and what an analyst receives. + contents: + - masque_plan + - clone_plan + - save_plan + - handover + - title: Geospatial post-processing desc: Coarsen or re-anchor synthetic coordinates while keeping them on land and plausible. contents: @@ -73,6 +88,7 @@ articles: - recipe_anatomy - confidentiality - design_preservation + - controls navbar: structure: diff --git a/inst/WORDLIST b/inst/WORDLIST index 5ffa407..7aaff98 100644 --- a/inst/WORDLIST +++ b/inst/WORDLIST @@ -1,31 +1,6 @@ -Allshouse -CRD -Doherty -Donut -Gesink -IBD -Leone -Lifecycle -NOTEs -NSW -ORCID -PII -POSIXct -POSIXlt -Pre -RCBD -README -S3 -S4 -S7 -SHA -Serre -TSV -X11 -XQuartz -Zandbergen agroclimatic al +Allshouse anonymisation anonymise anonymised @@ -37,33 +12,58 @@ categoricals centroid's changelog cli +CMD +Codecov +comparator's conductoR +confounder +covr +CRD cryptographic dataentry dev difftime +Doherty +Donut donut errored et +genotype's geocoded geolocations geomask geomasking geospatial +Gesink ggplot2 handoff +IBD interpretability jitter's joinable knitr +Leone +Lifecycle +lintr maintainer's masqued matchable +microdata multiset +NOTEs +NSW oldrel orchestrator +ORCID passthrough +PII +pkgdown +POSIXct +POSIXlt +Pre pre +RCBD +README regexes reproducibly retargeting @@ -72,13 +72,21 @@ roadmap roled roxygen runnable +S3 +S4 +S7 scriptable +sdcMicro +Serre +SHA shareability subprocess +synthpop tibble tokenises toolchains trippable +TSV unaliased unclassed underreport @@ -89,11 +97,6 @@ v0 v1 validator walkthrough -CMD -Codecov -covr -lintr -microdata -pkgdown -sdcMicro -synthpop +X11 +XQuartz +Zandbergen diff --git a/man/apply_recipe.Rd b/man/apply_recipe.Rd index e98a88c..099d4c4 100644 --- a/man/apply_recipe.Rd +++ b/man/apply_recipe.Rd @@ -13,7 +13,8 @@ apply_recipe(original, rec, check_integrity = TRUE) \item{check_integrity}{Logical. When \code{TRUE} (default), verifies that the NA mask of \code{original} matches the recipe's recorded -\code{integrity_fp}. Mismatches error with guidance. Pass \code{FALSE} to +\code{integrity_fp} and values against the recipe's fingerprint of them, +numbers within rounding from a file. Mismatches error with guidance. Pass \code{FALSE} to bypass when the missingness has legitimately changed since the recipe was built.} } diff --git a/man/audit_mask.Rd b/man/audit_mask.Rd index 06f1c00..dedc7c4 100644 --- a/man/audit_mask.Rd +++ b/man/audit_mask.Rd @@ -112,6 +112,12 @@ A column constant within a design unit that was synthesised row by row (\code{grain = FALSE}, or a collaborate grain with fewer than five units) carries a note saying the clone does not keep that grain. +The grade of a re-simulated number compares rows. A shift on a design level +of three or four rows, with within-level noise as large as the signal, can +grade low while the correlation of the level means stays near 0.85. A low +class means the rows are not copies. It does not mean the level means are +hidden. + As a reference point, a collaborate-mode clone of \code{agridat::besag.met} (yield as the outcome, design columns kept) scores \code{low} on every column, and none of its yields equals the original value. diff --git a/man/clone_plan.Rd b/man/clone_plan.Rd new file mode 100644 index 0000000..caec068 --- /dev/null +++ b/man/clone_plan.Rd @@ -0,0 +1,30 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plan.R +\name{clone_plan} +\alias{clone_plan} +\title{The plan in clone vocabulary, for the analyst} +\usage{ +clone_plan(m) +} +\arguments{ +\item{m}{A \code{masque} object from \code{\link[=mask]{mask()}}. A clone masked without a plan gives +a plan with no control specs.} +} +\value{ +A \code{masque_plan} in clone vocabulary. +} +\description{ +The plan of a \code{\link[=mask]{mask()}} result, in the names and labels of the clone. It +carries the roles as \code{\link[=roles]{roles()}} gives them, the linked events and each control +spec as resolved on the source. It holds no coordinate declaration, and +nothing else from the recipe. With \code{\link[=controls]{controls()}} it gives the analyst the +pre-declared controls on the clone. +} +\examples{ +r <- propose_roles(iris, detect = FALSE) +clone_plan(mask(iris, r, mode = "local", seed = 1L)) + +} +\seealso{ +\code{\link[=masque_plan]{masque_plan()}}, \code{\link[=controls]{controls()}}, \code{\link[=handover]{handover()}}. +} diff --git a/man/control_spec.Rd b/man/control_spec.Rd new file mode 100644 index 0000000..271232f --- /dev/null +++ b/man/control_spec.Rd @@ -0,0 +1,92 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/control_spec.R +\name{control_spec} +\alias{control_spec} +\title{Declare the controls to derive from a table} +\usage{ +control_spec( + on = NULL, + target = NULL, + effect = NULL, + strata = NULL, + unit = NULL, + grain = NULL, + n = 1L, + seed = NULL +) +} +\arguments{ +\item{on}{The name of the column the negative control reassigns, by +permuting it within \code{strata} at its \code{unit}. The outcome is not changed, +so any real effect of \code{on} stays in it as unexplained variation. Must be +a treatment or covariate column, categorical or +whole-valued. \code{NULL} (the default) means the single treatment column of +the table; a table with several or no treatment columns needs \code{on} +named.} + +\item{target}{The name of one or more numeric outcome columns the +positive control plants the effect on, or \code{NULL} for no positive +control. A logical, factor, date, event or two-valued column is +refused when the spec meets the table.} + +\item{effect}{The planted effect per level of \code{on}, in units of the +target's standard deviation in the table at hand. That standard deviation +includes the design variance and any real effect of \code{on}, so the size is +not a multiple of an analysis's residual. Give it as a named numeric +vector (\code{c(G01 = 0.5, G02 = -0.5)}; a level not named shifts by zero), +or \code{list(sd = s)} to draw one shift per level from a normal +distribution with standard deviation \code{s}, under the spec's seed, when +the spec first meets a table. Required with \code{target}. On a whole-valued +target the shift is rounded, which can change the ratios between levels; +\code{shift_realised} in \verb{@truth} holds the effect that is in the table, and a +warning (class \code{masque_control_shape_rounded}) marks a changed shape.} + +\item{strata}{Design columns within which \code{on} is permuted, or \code{NULL} +for no stratification. Every unit of \code{on} must lie within one stratum. +Prefer complete replicates to incomplete blocks; see \code{\link[=controls]{controls()}}.} + +\item{unit}{Design columns within which \code{on} is constant. \code{on} is then +permuted across those units, so name the unit \code{on} was assigned at. +\code{NULL} (the default) permutes rows, and in a plan picks the coarsest design +column on which \code{on} is constant and which has enough units for a 5 percent +level, with a warning (class \code{masque_control_unit_ambiguous}) when several +qualify.} + +\item{grain}{A named list giving, for other analysed columns constant +within a design unit, the unit columns: \code{list(rain = "trial")}. Such a +column is permuted across its units in the null control, so it stays +constant within each unit.} + +\item{n}{The number of negative (and positive) replicates.} + +\item{seed}{The seed of the control draws. Required.} +} +\value{ +A \code{masque_control_spec} object, with a print method. +} +\description{ +A control spec says what \code{\link[=controls]{controls()}} permutes and plants: the column +the negative control reassigns (\code{on}), the outcome the +positive control plants an effect on (\code{target}), the planted effect per +level of \code{on} in outcome-SD units (\code{effect}), the design strata within +which \code{on} is permuted and the unit it is permuted at, the other +unit-grained columns, the number of replicates and the seed. The spec is +declared once, before any control table exists, and is the record of +what was planted; print it to read the declaration. +} +\details{ +The null control needs nothing beyond the seed. The negative control +needs \code{on}, which defaults to the single treatment column of the table. +The positive control needs \code{target} and \code{effect} together. +} +\examples{ +spec <- control_spec( + on = "gen", target = "yield", effect = c(G01 = 0.5, G02 = -0.5), + strata = "rep", n = 19L, seed = 2026L +) +spec + +} +\seealso{ +\code{\link[=controls]{controls()}}, \code{\link[=control_table]{control_table()}}. +} diff --git a/man/control_table.Rd b/man/control_table.Rd new file mode 100644 index 0000000..6e1431b --- /dev/null +++ b/man/control_table.Rd @@ -0,0 +1,43 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/controls.R +\name{control_table} +\alias{control_table} +\title{Produce one control dataset} +\usage{ +control_table(ctl, kind = c("null", "negative", "positive"), i = 1L) +} +\arguments{ +\item{ctl}{A \code{masque_controls} object from \code{\link[=controls]{controls()}}.} + +\item{kind}{One of \code{"null"}, \code{"negative"}, \code{"positive"}.} + +\item{i}{The replicate, from 1 to \code{n} of the spec; ignored for the null.} +} +\value{ +A data.table with the columns, classes, NA mask and row order of +the input table. +} +\description{ +Regenerates a control dataset of a \code{\link[=controls]{controls()}} object from its seed: the +null, or negative or positive replicate \code{i}. Positive replicate \code{i} is +negative replicate \code{i} plus the planted shift. The result carries a +\code{masque_control} attribute naming its kind and replicate; its values +alone do not tell it from the table it came from, so keep the manifest +with it. +} +\examples{ +trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +r <- propose_roles(trial, detect = FALSE) +r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +r <- set_role(r, "gen", role = "treatment", action = "keep") +r <- set_role(r, "yield", role = "outcome", action = "scramble") +r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +spec <- control_spec(on = "gen", strata = "rep", n = 3L, seed = 1L) +ctl <- controls(trial, r, spec) +neg <- control_table(ctl, "negative", 2L) +table(neg$block, neg$gen)[1:2, 1:6] + +} +\seealso{ +\code{\link[=controls]{controls()}}, \code{\link[=control_spec]{control_spec()}}. +} diff --git a/man/controls.Rd b/man/controls.Rd new file mode 100644 index 0000000..15af779 --- /dev/null +++ b/man/controls.Rd @@ -0,0 +1,145 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/controls.R +\name{controls} +\alias{controls} +\title{Derive null, negative and positive controls from a table} +\usage{ +controls(df, roles, spec = NULL, events = NULL, seed = NULL, recipe = NULL) +} +\arguments{ +\item{df}{The table to derive controls from: a data frame or data.table.} + +\item{roles}{Its role table, as returned by \code{\link[=roles]{roles()}} for a clone or by +\code{\link[=propose_roles]{propose_roles()}} and \code{\link[=set_role]{set_role()}} for any table. Or a plan: the analyst +passes \code{\link[=clone_plan]{clone_plan()}} with the clone, the owner passes the plan and the +\code{recipe} with the real table.} + +\item{spec}{A \code{\link[=control_spec]{control_spec()}}. With a plan in \code{roles}, the name of one of +its specs, or \code{NULL} when it holds exactly one.} + +\item{events}{Linked time/status pairs among the outcome columns, in the +form \code{\link[=mask]{mask()}} takes. The null permutes each pair as one block. \code{NULL} +when there are none.} + +\item{seed}{A private integer seed for the permutations that overrides the +spec's seed, for the owner's side; \code{NULL} uses the spec's seed, or with a +\code{recipe} the recipe's key. A value equal to the spec's seed is refused +there. See "Two sides, two seeds" for what an integer can and cannot hide.} + +\item{recipe}{With a plan in \code{roles}, the recipe of the clone. \code{df} is then +the real table: its cleaning and conformance are replayed as in +\code{\link[=apply_recipe]{apply_recipe()}}, a table with a different schema, NA mask or values is +refused, +and a spec that differs from the one fixed at \code{\link[=mask]{mask()}} is refused.} +} +\value{ +A \code{masque_controls} object: the resolved spec (\verb{@spec}), the +manifest (\verb{@manifest}, one row per dataset), the truth table +(\verb{@truth}, one row per target and level of \code{on}) and the notes +(\verb{@notes}). Datasets are produced by \code{\link[=control_table]{control_table()}}. +} +\description{ +Builds the control datasets a \code{\link[=control_spec]{control_spec()}} declares, from any table +with a role table: a synthetic clone, a comparator's clone, or, on the +owner's side, the original. Nothing is re-simulated. Three kinds of +control exist, each with a known truth: +} +\details{ +\itemize{ +\item \code{null}: every analysed column (outcome, covariate and date columns +with action \code{scramble} or \code{alias}) is permuted independently over its +non-missing rows; a column declared unit-grained is permuted across its +units and stays constant within each. Design, treatment and kept +columns, the NA mask and every column's multiset of values are kept. +No association among analysed columns, or with the treatment, remains. +Use it as a smoke test: a pipeline that finds signal here builds that +signal itself. A leak already in the table, such as a covariate that +copies the outcome, is shuffled away with the rest and passes. +\item \code{negative}: the declared column \code{on} alone is permuted within the +declared design strata, at its unit, while the design structure and all +other associations are kept. The outcome is not changed, so any real +effect of \code{on} stays in it as unexplained variation. Choose strata that +every level of \code{on} can reach. Complete replicates qualify. Within incomplete blocks each block +keeps its set of levels, so part of the real effect of \code{on} survives. +With \code{n} replicates it gives a permutation null for a whole pipeline: a +false-positive rate, a permutation p-value. +\item \code{positive}: the same permutation as the negative replicate, plus the +declared shift per level of \code{on} on each target, in units of the +target's standard deviation in the table at hand. Positive minus +negative is the planted truth, exactly. +} +} +\section{Declaration before data}{ + +The spec is resolved against the table here: \code{on} defaults to the single +treatment column; strata and unit are checked for nesting; an \code{effect} +given as \code{list(sd = )} is drawn under the spec's seed, once, so the same +spec plants the same vector on every table it meets. The resolved spec +and the truth table are on the returned object and in its print. +} + +\section{Two sides, two seeds}{ + +An analyst who holds only the clone runs this on the clone. The owner +runs it on the original, offline, on a private draw (the planted vector +still comes from the spec's seed). A public seed would let anyone who knows +the design, which the clone shares, rebuild every permutation and read the +original outcome out of per-replicate statistics. + +With a plan and its \code{recipe}, the permutations come from SHA-256 of a +256-bit key the recipe holds, so there is no seed to search. A private +integer \code{seed} is the weaker route: it has at most 31 bits, and anyone given +a per-replicate statistic can try each value, because the true seed fits the +statistic exactly and a wrong one does not. + +Control tables built from an original never leave the owner. What may leave +goes through the owner's usual output checking, classed by what it +estimates. The false-positive rate says little about the real outcome. The +rate at which the positives find the shift depends on the residual variance. +The permutation p-value is the result of the analyst's test on the real +table. A vector of per-replicate means or contrasts is not an aggregate, +under either route, and every further pipeline run on the original is +another set of queries. +} + +\section{Limits}{ + +A kept column is structure and is never permuted, so leakage through +kept columns, ids or row order passes the null. No control can detect +confounding. A planted effect is clean: uniform within a level, with no +confounding and no interaction. +Nothing is clipped, so a shift can take a non-negative outcome below +zero; \code{\link[=control_table]{control_table()}} warns when it does. The null collapses the +design means of row-grained outcomes, so false-positive calibration +comes from the negative control, not the null. + +A negative control is a test of the stated size for a comparison that uses +only contrasts within the strata, so the analysis needs the strata in its +model. Permuting within replicates and fitting no replicate term puts the +replicate variance in the residual and rejects far less often than the +nominal level. A shared rate on the clone and the original shows that the two +agree, and a rate of zero on both says nothing about calibration. A warning +(class \code{masque_control_unit_small}) marks units and strata with too few +arrangements for a 5 percent level, and another (class +\code{masque_control_strata_weak}) marks strata that hold one level of \code{on}. +} + +\examples{ +trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +r <- propose_roles(trial, detect = FALSE) +r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +r <- set_role(r, "gen", role = "treatment", action = "keep") +r <- set_role(r, "yield", role = "outcome", action = "scramble") +r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +spec <- control_spec( + on = "gen", target = "yield", effect = list(sd = 0.5), + strata = "rep", n = 5L, seed = 2026L +) +ctl <- controls(trial, r, spec) +ctl +head(control_table(ctl, "positive", 1L)) + +} +\seealso{ +\code{\link[=control_spec]{control_spec()}}, \code{\link[=control_table]{control_table()}}, \code{\link[=mask]{mask()}}, \code{\link[=roles]{roles()}}. +} diff --git a/man/handover.Rd b/man/handover.Rd new file mode 100644 index 0000000..acef7e0 --- /dev/null +++ b/man/handover.Rd @@ -0,0 +1,58 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plan.R +\name{handover} +\alias{handover} +\title{Hand a clone to an analyst} +\usage{ +handover( + m, + dir, + name = NULL, + overwrite = FALSE, + allow_high = FALSE, + allow_local = FALSE +) +} +\arguments{ +\item{m}{A \code{masque} object from \code{\link[=mask]{mask()}}.} + +\item{dir}{The folder to write. An existing folder that holds files is +refused unless \code{overwrite = TRUE}.} + +\item{name}{Dataset name for the data card; \code{NULL} uses \code{"clone"}. It is +written as given, so it should say nothing the clone hides.} + +\item{overwrite}{Replace the four files if they exist.} + +\item{allow_high}{Write despite HIGH findings, with a warning.} + +\item{allow_local}{Write a local-mode clone, for a folder the owner keeps.} +} +\value{ +The paths written, invisibly. +} +\description{ +Writes what the analyst works from into one folder: the clone (\code{clone.csv}), +the plan in clone vocabulary (\code{plan.yaml}), the data card (\code{card.yaml}) and a +plain note (\code{README.txt}). The recipe is never written. The release gate runs +first, as for \code{\link[=write_set]{write_set()}}: an unresolved HIGH finding blocks the write +unless \code{allow_high = TRUE}. A local-mode clone has not been audited and is +meant to stay with the owner, so it is refused unless \code{allow_local = TRUE}. +} +\details{ +\code{clone.csv} loses column classes. Read label columns as text where a label +looks like a number, or \code{controls()} will not find its levels. +} +\examples{ +if (requireNamespace("yaml", quietly = TRUE)) { + r <- propose_roles(iris, mode = "collaborate", detect = FALSE) + m <- mask(iris, r, mode = "collaborate", seed = 1L) + d <- tempfile() + handover(m, d) + list.files(d) +} + +} +\seealso{ +\code{\link[=clone_plan]{clone_plan()}}, \code{\link[=data_card]{data_card()}}, \code{\link[=controls]{controls()}}. +} diff --git a/man/jitter_coordinates.Rd b/man/jitter_coordinates.Rd index 762b4f5..9b01055 100644 --- a/man/jitter_coordinates.Rd +++ b/man/jitter_coordinates.Rd @@ -43,7 +43,11 @@ most \code{max_km}.} displacement that lands in the sea, using \code{\link[maps:map.where]{maps::map.where()}} (requires the \code{maps} package). A function \verb{function(lon, lat)} returning a logical vector supplies your own test (for example an \code{sf}-based high-resolution -coastline). \code{FALSE} applies no constraint.} +coastline). \code{FALSE} applies no constraint. The test conditions the annulus: +near a coast the accepted directions are the ones on land, so the mean +displacement points inland and the median distance can move by kilometres, +while no point falls inside \code{min_km}. A site that cannot reach land is +missing.} \item{max_tries}{Maximum re-draws per site before giving up. Default \code{100}. A site that cannot be placed on land is set to \code{NA} on both axes, with a @@ -92,23 +96,27 @@ therefore comes back with one masked coordinate per place, exactly as the source table has one true coordinate per place. \code{by} chooses how a site is identified: \itemize{ -\item \code{by = NULL} (default) treats rows that share an identical input coordinate -pair as one site. This needs no user action and cannot alter genuinely +\item \code{by = NULL} (default) treats rows that share an input coordinate pair, to +nine decimal places, as one site. This needs no user action and cannot alter genuinely point-level data, where every input pair is already distinct. \item \code{by = c("trial")} (one or more column names) treats rows sharing those values as one site, for the case where a site's recorded coordinates differ slightly between rows. Where such a group holds more than one distinct input coordinate, the group's row-weighted centroid is used and the consolidation is reported. -\item \code{by = FALSE} restores the pre-0.10.0 behaviour of displacing every row -independently, and warns if the input carries repeated coordinate pairs. -It exists only so a table masked by an earlier version can be reproduced. +\item \code{by = FALSE} displaces every row independently, as versions before 0.10.0 +did, and warns if the input carries repeated coordinate pairs. The draw +follows the row's position, so shuffled rows move differently. } +The draw is a function of the seed, the two column names and the site's +value: its coordinates, or its \code{by} values. Row order and the other sites +in the table play no part. + Displacing each row of a repeated coordinate separately leaks the site. Donut displacement is isotropic, so the mean of many independent draws around one true site converges on that site: averaging the 360 rows of -a single trial recovers the true position to a median 0.58 km, against the +a single trial recovers the true position to a median 0.64 km, against the 5 km floor the donut was asked for. One draw per site removes the estimator and leaves the full displacement in place. } @@ -123,19 +131,31 @@ out, exactly as its rows could be before. Measured at the 5-20 km default: \tabular{lr}{ Rebuilds kept \tab Median recovery error \cr -1 \tab 14.2 km \cr -2 \tab 10.1 km \cr -4 \tab 6.4 km \cr -8 \tab 4.5 km \cr +1 \tab 14.6 km \cr +2 \tab 9.4 km \cr +4 \tab 6.3 km \cr +8 \tab 4.4 km \cr 16 \tab 3.1 km } -Eight kept rebuilds fall below the 5 km floor the donut was asked for. - -The remedy is to \strong{reuse the seed}. The same \code{seed} reproduces the -same displacement for the same sites, so any number of rebuilds yields one -point and nothing to average. Draw a fresh seed only when you intend to -supersede every earlier version, and retire the ones you replace. +Seven kept rebuilds fall below the 5 km floor the donut was asked for. + +A known region also cuts the annulus. Someone who knows a site lies in a +square a few times \code{max_km} wide or smaller can discard the part of the +annulus around the released point that falls outside it, with no second +release needed. + +The remedy is to \strong{reuse the seed}. The same \code{seed} gives a site the same +displacement in every rebuild, even when rows are shuffled, added or removed, +so any number of rebuilds yields one point and nothing to average. Three +things start a new draw for a site: a different seed, a changed site value +(a corrected coordinate, a renamed \code{by} label) and renamed coordinate +columns. A clone built before version 0.17.0 used a different draw, so a +rebuild under 0.17.0 is a new release of those sites even with the same seed. +Treat each of these as a fresh seed: retire the versions it replaces. The seed +reveals every displacement, so it stays as confidential as the recipe. A seed +has 31 bits: where a clone shows site labels and holds two pairs at one place, +it can be found by search, so alias the label or declare one pair. Choose the unit carefully. It is the finest grouping that denotes \strong{one physical place at one time}. For a multi-year trial series that is location diff --git a/man/mask.Rd b/man/mask.Rd index d382d92..703ea73 100644 --- a/man/mask.Rd +++ b/man/mask.Rd @@ -30,7 +30,10 @@ mask( record and its merges and type changes go on the recipe, so \code{\link[=apply_recipe]{apply_recipe()}} replays them; \code{clean} is not applied a second time.} -\item{roles}{A roles table from \code{\link[=propose_roles]{propose_roles()}} (possibly edited). +\item{roles}{A roles table from \code{\link[=propose_roles]{propose_roles()}} (possibly edited), or a +\code{\link[=masque_plan]{masque_plan()}}. A plan supplies \code{coords}, \code{events}, \code{conditional}, +\code{ladder}, \code{grain} and \code{alias_names} where the call leaves them out, and its +control specs are resolved on \code{df} and written back to the recipe. Tables from masque <= 0.5.0 are upgraded with a deprecation warning; see \code{\link[=roles_validate]{roles_validate()}}.} @@ -66,8 +69,9 @@ original-to-alias map is stored in the recipe and inverted by aliased synthetic round-trips. Alias the names when the schema itself is sensitive.} -\item{conditional}{Logical scalar (default \code{FALSE}). The -\emph{collaborate-grade conditional clone}. When \code{FALSE}, scrambled +\item{conditional}{\code{FALSE} (default), \code{TRUE}, \code{"design"}, or a character +vector of column names. The \emph{collaborate-grade conditional clone}. +When \code{FALSE}, scrambled numeric columns are re-simulated from one global Gaussian copula - marginals and global covariance survive, but the treatment-to-outcome relationship does not, so a causal model fitted on the clone recovers @@ -89,7 +93,19 @@ into a global fallback. The rung reached is recorded on the recipe as the pooled share as \code{fallback_frac}, and any coarsening or residual pooling raises a classed \code{masque_conditional_degraded} warning -- including the case where no treatment or design column survives at -all, in which case the clone is the pooled copula.} +all, in which case the clone is the pooled copula. + +\code{"design"} conditions on the retained design columns alone. The clone +keeps the design means, block and environment variance, while a treatment +effect is not carried: a real effect moves into the residual, as it does +when an owner permutes the treatment for a negative control, so the clone +stands in for those negatives. The residual is not the one a real fit has. +Linked \code{events} are stratified only by treatment columns of the partition, +so \code{"design"} and a design-only partition do not keep their design means. +A character vector of names gives the partition explicitly, from the +retained design and treatment columns, and runs through the same ladder; +treatment columns are still never dropped. The single word \code{"design"} +always means this option, even where a column has that name.} \item{ladder}{How the coarsening ladder orders the design columns it may drop, and what it keeps of a dropped one. \code{"hierarchy"} (the default) @@ -109,7 +125,12 @@ stratum keep the stratum columns' main effects the same way, which is how a three-replicate variety trial keeps its genotype means. A level with a single row is pooled with the other singletons before the fit, so no row's own outcome is carried. The terms carried are recorded on -the recipe as \code{conditioning_shifted}. \code{"levels"} +the recipe as \code{conditioning_shifted}. The interaction of a dropped column +with a kept treatment is not carried, so a multi-environment clone can keep +its genotype and environment means and lose the genotype-by-environment +pattern. Shrinkage of a shift uses the mean level size, so on unbalanced +columns the small levels are shrunk less than their own size would give. +\code{"levels"} is the ladder of masque 0.11.1 to 0.12.0: columns are dropped in decreasing order of distinct values and a dropped column leaves no trace in the clone.} @@ -131,7 +152,8 @@ survives masking, coarsened; the recipe records the parameters and the site grouping it was coarsened under, and \code{\link[=apply_recipe]{apply_recipe()}} retargets to the real coordinates. Several published jitters of the same point average toward it: keep the seed, which the recipe records, and do not publish more than -one jittered release of the same sites.} +one jittered release of the same sites. A seed has 31 bits: where a clone +shows site labels and holds two pairs at one place, it can be found by search.} \item{events}{Optional linked time-to-event outcomes. One spec per group of linked events, \code{list(time = c(...), status = c(...))}, with the time and diff --git a/man/mask_set.Rd b/man/mask_set.Rd index 9e0270e..b932f3f 100644 --- a/man/mask_set.Rd +++ b/man/mask_set.Rd @@ -51,7 +51,7 @@ modes must be reconciled explicitly. Otherwise default to \code{"local"}.} \code{TRUE} aliases every non-link column (linked join keys keep their names so the synthetic set stays joinable).} -\item{conditional}{Logical scalar (default \code{FALSE}). Passed through to +\item{conditional}{\code{FALSE} (default), \code{TRUE} or \code{"design"}. Passed through to each per-table \code{\link[=mask]{mask()}} call: when \code{TRUE}, every table's numeric block is re-simulated within its own treatment-by-design strata so the treatment-to-outcome relationship survives the clone. See diff --git a/man/masque.Rd b/man/masque.Rd index 95a64c1..306af40 100644 --- a/man/masque.Rd +++ b/man/masque.Rd @@ -52,7 +52,8 @@ in \code{\link[=mask]{mask()}} and \code{\link[=mask_set]{mask_set()}}; with no \item{alias_names}{Hide column names; see \code{\link[=mask]{mask()}} / \code{\link[=mask_set]{mask_set()}}.} -\item{conditional}{Logical scalar (default \code{FALSE}). The conditional +\item{conditional}{\code{FALSE} (default), \code{TRUE}, \code{"design"} or column names. +The conditional clone mode passed through to \code{\link[=mask]{mask()}} / \code{\link[=mask_set]{mask_set()}}: when \code{TRUE}, numeric columns are re-simulated within each treatment-by-design stratum so the treatment-to-outcome relationship survives the clone. diff --git a/man/masque_plan.Rd b/man/masque_plan.Rd new file mode 100644 index 0000000..736096a --- /dev/null +++ b/man/masque_plan.Rd @@ -0,0 +1,75 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plan.R +\name{masque_plan} +\alias{masque_plan} +\title{A masking plan: roles and named specs in one object} +\usage{ +masque_plan( + roles, + coords = NULL, + events = NULL, + conditional = FALSE, + ladder = c("hierarchy", "levels"), + grain = TRUE, + alias_names = FALSE, + controls = NULL +) +} +\arguments{ +\item{roles}{A roles table from \code{\link[=propose_roles]{propose_roles()}}.} + +\item{coords, events, conditional, ladder, grain, alias_names}{As in \code{\link[=mask]{mask()}}.} + +\item{controls}{\code{NULL}, one \code{\link[=control_spec]{control_spec()}}, or a named list of them. A +single spec is named \code{default}.} +} +\value{ +A \code{masque_plan} object. +} +\description{ +Holds what a \code{\link[=mask]{mask()}} call needs apart from the table: the roles, the +coordinate and event declarations, the conditioning choice and any named +\code{\link[=control_spec]{control_spec()}}s. \code{mask(df, plan)} reads them from the plan; an argument +passed to \code{\link[=mask]{mask()}} as well overrides the plan's value for that call. +} +\details{ +At \code{mask()} each control spec is resolved on the source table and written +back: its unit, its strata, the grain of its columns and its effect vector. +The clone does not depend on the specs. The resolved plan is on the recipe, +and \code{\link[=clone_plan]{clone_plan()}} gives the same plan in clone vocabulary for the analyst. + +A spec's \code{unit}, where not declared, resolves to the coarsest design +partition within which \code{on} is constant, or to the row. Its \code{strata}, where +not declared, resolve to the finest design partition above the unit in which +every stratum holds every level of \code{on}, or to none. A stratum that misses +levels of \code{on} would keep part of the real effect in a negative control, so +incomplete blocks are not chosen by default. Declare a whole-plot unit where +few plots hold more than one row: it is taken only when a quarter of the rows +share a unit. + +A drawn effect, \code{list(sd = )}, is drawn over the clone's labels and shows +nothing of the source. A declared effect is written to the clone plan as +given, so values that rise with a level reveal that level's rank. +} +\examples{ +trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +r <- propose_roles(trial, detect = FALSE) +r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +r <- set_role(r, "gen", role = "treatment", action = "alias") +r <- set_role(r, "yield", role = "outcome", action = "scramble") +r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +p <- masque_plan( + r, + conditional = "design", + controls = control_spec( + on = "gen", target = "yield", effect = list(sd = 0.5), n = 5L, seed = 7L + ) +) +p +m <- mask(trial, p, mode = "local", seed = 1L) +clone_plan(m) + +} +\seealso{ +\code{\link[=mask]{mask()}}, \code{\link[=clone_plan]{clone_plan()}}, \code{\link[=save_plan]{save_plan()}}, \code{\link[=controls]{controls()}}, \code{\link[=handover]{handover()}}. +} diff --git a/man/save_plan.Rd b/man/save_plan.Rd new file mode 100644 index 0000000..f30f392 --- /dev/null +++ b/man/save_plan.Rd @@ -0,0 +1,34 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/plan.R +\name{save_plan} +\alias{save_plan} +\alias{read_plan} +\title{Save and read a masking plan as YAML} +\usage{ +save_plan(plan, path) + +read_plan(path) +} +\arguments{ +\item{plan}{A \code{\link[=masque_plan]{masque_plan()}}.} + +\item{path}{Path of a \code{.yaml} or \code{.yml} file.} +} +\value{ +\code{save_plan()} returns \code{path} invisibly; \code{read_plan()} a \code{masque_plan}. +} +\description{ +A plan on disk can be read and compared as text. \code{save_plan()} needs the +\code{yaml} package. A custom \code{on_land} function cannot be written, and is refused. +A source-vocabulary plan names real columns and labels, so it stays with the +owner. The analyst's file comes from \code{\link[=clone_plan]{clone_plan()}} or \code{\link[=handover]{handover()}}. +} +\examples{ +if (requireNamespace("yaml", quietly = TRUE)) { + r <- propose_roles(iris, detect = FALSE) + f <- tempfile(fileext = ".yaml") + save_plan(masque_plan(r), f) + read_plan(f) +} + +} diff --git a/man/save_recipe.Rd b/man/save_recipe.Rd index be23f7d..52f5788 100644 --- a/man/save_recipe.Rd +++ b/man/save_recipe.Rd @@ -23,8 +23,8 @@ written without simulator state.} Writes the recipe to a single \code{.rds} file. The default is \strong{runtime-minimal}: no simulator state (copula covariance, raw margins) is written, only the -translation maps, factor metadata, storage classes, integrity fingerprint, -and warnings. This keeps the saved artefact small and reduces the +translation maps, factor metadata, storage classes, integrity fingerprints, +the key behind the owner's controls, and warnings. This keeps the saved artefact small and reduces the information that would leak if the recipe file alone were shared. } \details{ diff --git a/tests/testthat/helper-controls.R b/tests/testthat/helper-controls.R new file mode 100644 index 0000000..f128545 --- /dev/null +++ b/tests/testthat/helper-controls.R @@ -0,0 +1,107 @@ +# Fixtures for the control datasets: a small MET with trial-level covariates, +# a whole-plot factor, a whole-valued outcome and missing cells. +.ctl_met <- function(n_trial = 4L, n_rep = 3L, n_gen = 8L, seed = 11L) { + withr::with_seed(seed, { + trials <- sprintf("T%d", seq_len(n_trial)) + gens <- sprintf("G%02d", seq_len(n_gen)) + g <- expand.grid( + gen = gens, rep = seq_len(n_rep), trial = trials, + stringsAsFactors = FALSE, KEEP.OUT.ATTRS = FALSE + ) + g$block <- sprintf("B%d", (match(g$gen, gens) - 1L) %/% 4L + 1L) + g$irrig <- ifelse(g$rep %% 2L == 1L, "dry", "wet") + trial_eff <- stats::rnorm(n_trial, 0, 0.8) + gen_eff <- stats::rnorm(n_gen, 0, 0.4) + g$yield <- round( + 3 + trial_eff[match(g$trial, trials)] + gen_eff[match(g$gen, gens)] + + ifelse(g$irrig == "wet", 0.5, 0) + stats::rnorm(nrow(g), 0, 0.3), + 2 + ) + g$count <- as.integer(round(g$yield * 3)) + g$aet <- round(300 + 40 * trial_eff, 1)[match(g$trial, trials)] + g$plot_cov <- round(g$yield * 10 + stats::rnorm(nrow(g), 0, 1), 2) + g$soil <- rep_len(c("clay", "loam", "sand"), n_trial)[ + match(g$trial, trials) + ] + g$sow <- as.Date("2024-05-01") + 7L * match(g$trial, trials) + g$yield[c(3L, 17L, 40L)] <- NA + g$plot_cov[c(5L, 6L)] <- NA + g$aet[g$trial == trials[2L]] <- NA + g$rep <- sprintf("R%d", g$rep) + g[, c( + "trial", "rep", "block", "irrig", "gen", "yield", "count", "aet", + "plot_cov", "soil", "sow" + )] + }) +} + +.ctl_roles <- function(df) { + r <- suppressWarnings(propose_roles(df, detect = FALSE)) + r <- set_role(r, c("trial", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "irrig", role = "covariate", action = "scramble") + r <- set_role(r, c("yield", "count"), role = "outcome", action = "scramble") + r <- set_role( + r, c("aet", "plot_cov", "soil"), role = "covariate", action = "scramble" + ) + set_role(r, "sow", role = "date", action = "scramble") +} + +# The standard spec on the MET: genotype permuted within blocks, the +# trial-level covariates declared unit-grained. +.ctl_spec <- function(..., n = 2L, seed = 3L) { + control_spec( + on = "gen", + strata = c("trial", "rep", "block"), + grain = list(aet = "trial", soil = "trial", sow = "trial"), + n = n, + seed = seed, + ... + ) +} + +# Two arms, no design: for effect recovery. +.ctl_two_arm <- function(n = 1000L, seed = 5L) { + withr::with_seed(seed, data.frame( + trt = rep(c("A", "B"), each = n / 2L), + y = round(stats::rnorm(n, 10, 2), 3), + stringsAsFactors = FALSE + )) +} + +.ctl_two_arm_roles <- function(df) { + r <- suppressWarnings(propose_roles(df, detect = FALSE)) + r <- set_role(r, "trt", role = "treatment", action = "keep") + set_role(r, "y", role = "outcome", action = "scramble") +} + +# Many levels, no design: for rank recovery. +.ctl_many_levels <- function(k = 20L, m = 30L, seed = 7L) { + withr::with_seed(seed, data.frame( + lev = rep(sprintf("L%02d", seq_len(k)), each = m), + y = round(stats::rnorm(k * m, 10, 2), 3), + stringsAsFactors = FALSE + )) +} + +.ctl_many_roles <- function(df) { + r <- suppressWarnings(propose_roles(df, detect = FALSE)) + r <- set_role(r, "lev", role = "treatment", action = "keep") + set_role(r, "y", role = "outcome", action = "scramble") +} + +# The non-missing values of two vectors are the same multiset. +.ctl_same_multiset <- function(a, b) { + identical( + sort(as.character(a[!is.na(a)])), + sort(as.character(b[!is.na(b)])) + ) +} + +# A refusal: an error of the given class under the shared marker, no warning. +.ctl_expect_refusal <- function(expr, class) { + err <- testthat::expect_no_warning(tryCatch(expr, error = function(e) e)) + testthat::expect_s3_class(err, "error") + testthat::expect_true(inherits(err, class)) + testthat::expect_true(inherits(err, "orchestra_refusal")) +} diff --git a/tests/testthat/test-conditional-disclosure.R b/tests/testthat/test-conditional-disclosure.R new file mode 100644 index 0000000..d9f4beb --- /dev/null +++ b/tests/testthat/test-conditional-disclosure.R @@ -0,0 +1,208 @@ +# A conditional clone must not rebuild a row's value from a tiny design level, +# and the audit must see a clone that tracks the original row by row. + +.fx_pairs <- function(n = 400L, seed = 2L) { + withr::with_seed(seed, { + d <- data.frame( + gen = paste0("G", sample(20L, n, TRUE)), + rep = paste0("R", rep(1:4, length.out = n)), + pairid = paste0("P", rep(seq_len(n / 2L), each = 2L)) + ) + u <- rep(stats::rnorm(n / 2L, 0, 2), each = 2L) + d$y <- 10 + u + stats::rnorm(n, 0, 0.2) + d$z <- 3 + 0.5 * u + stats::rnorm(n, 0, 0.2) + d + }) +} + +.fx_pair_roles <- function(d) { + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("rep", "pairid"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + set_role(r, c("y", "z"), role = "outcome", action = "scramble") +} + +test_that("a two-row design level gives no shift that rebuilds the value", { + d <- .fx_pairs() + r <- .fx_pair_roles(d) + for (cnd in list(TRUE, "design")) { + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = cnd, + seed = 1L, quiet = TRUE)) + expect_lt(abs(stats::cor(d$y, synthetic(m)$y)), 0.5) + } +}) + +test_that("a design level of ample rows still carries its shift", { + d <- .fx_pairs(400L) + withr::with_seed(5L, { + d$env <- rep(paste0("E", 1:8), each = 50L) + d$y <- rep(stats::rnorm(8L, 0, 3), each = 50L) + stats::rnorm(400L) + }) + d$pairid <- NULL + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("rep", "env"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, c("y", "z"), role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = 1L, quiet = TRUE)) + env_sd <- function(x) stats::sd(tapply(x$y, x$env, mean)) + expect_lt(abs(env_sd(synthetic(m)) / env_sd(d) - 1), 0.2) +}) + +test_that("the audit raises HIGH when a clone tracks the original row by row", { + withr::local_seed(4L) + n <- 300L + k <- 5L + d <- data.frame( + gen = paste0("G", sample(4L, n, TRUE)), + block = paste0("B", rep(seq_len(n / k), each = k)) + ) + d$y <- rep(stats::rnorm(n / k, 0, 5), each = k) + stats::rnorm(n, 0, 0.05) + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, "block", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "y", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = 1L, quiet = TRUE)) + expect_gt(stats::cor(d$y, synthetic(m)$y), 0.95) + expect_identical(m@audit$leakage_class[m@audit$col == "y"], "high") + skip_if_not_installed("yaml") + expect_error(handover(m, file.path(withr::local_tempdir(), "p")), + class = "masque_blocked_write") + # Falsifier: the marginal clone of the same table is not flagged. + g <- suppressWarnings(mask(d, r, mode = "collaborate", seed = 1L, quiet = TRUE)) + expect_false(identical(g@audit$leakage_class[g@audit$col == "y"], "high")) +}) + +test_that("the design clone's residual matches an owner's negative control", { + # A permutation moves a real treatment effect into the residual. The design + # clone does the same, so negatives on the two sides share a denominator. + withr::local_seed(8L) + d <- expand.grid(gen = paste0("G", 1:10), rep = paste0("R", 1:6), + stringsAsFactors = FALSE) + d$y <- as.numeric(factor(d$gen)) * 1.8 + as.numeric(factor(d$rep)) + + stats::rnorm(nrow(d)) + r <- propose_roles(d, mode = "local", detect = FALSE) + r <- set_role(r, "rep", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + rms <- function(x) stats::anova(stats::lm(y ~ rep + gen, x))["Residuals", "Mean Sq"] + own <- controls(d, r, control_spec(on = "gen", strata = "rep", n = 40L, seed = 3L), + seed = 11L) + owner_rms <- vapply(1:40, function(i) rms(control_table(own, "negative", i)), numeric(1)) + clone_rms <- vapply(1:40, function(s) { + rms(synthetic(suppressWarnings(mask(d, r, seed = s, quiet = TRUE, conditional = "design")))) + }, numeric(1)) + expect_gt(mean(owner_rms), 5 * rms(d)) + expect_lt(abs(mean(clone_rms) / mean(owner_rms) - 1), 0.25) +}) + +test_that("big environment means in large cells grade medium, not high", { + withr::local_seed(6L) + d <- expand.grid(env = paste0("E", 1:10), plot = 1:40, stringsAsFactors = FALSE) + d$gen <- paste0("G", sample(10L, nrow(d), TRUE)) + d$y <- rep(stats::rnorm(10L, 0, 30), times = 40L) + stats::rnorm(nrow(d)) + d$plot <- NULL + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, "env", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "y", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = 1L, quiet = TRUE)) + expect_gt(stats::cor(d$y, synthetic(m)$y), 0.95) + expect_identical(m@audit$leakage_class[m@audit$col == "y"], "medium") +}) + +test_that("three- and four-row levels that track their rows are HIGH and blocked", { + for (k in 3:4) { + n <- 240L + withr::with_seed(k, { + d <- data.frame( + gen = paste0("G", sample(10L, n, TRUE)), + lv = paste0("L", rep(seq_len(n / k), each = k)) + ) + d$y <- rep(stats::rnorm(n / k, 0, 3), each = k) + stats::rnorm(n, 0, 0.2) + }) + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, "lv", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "y", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = 1L, quiet = TRUE)) + expect_gt(abs(stats::cor(d$y, synthetic(m)$y)), 0.9) + expect_identical(m@audit$leakage_class[m@audit$col == "y"], "high") + skip_if_not_installed("yaml") + expect_error( + handover(m, file.path(withr::local_tempdir(), "p")), + class = "masque_blocked_write" + ) + } +}) + +test_that("a tracking subset of small levels is seen among many single rows", { + withr::local_seed(14L) + sizes <- c(rep(3L, 30L), rep(1L, 310L)) + d <- data.frame(lv = rep(sprintf("L%03d", seq_along(sizes)), sizes)) + d$gen <- paste0("G", sample(10L, nrow(d), TRUE)) + d$y <- rep(stats::rnorm(length(sizes), 0, 3), sizes) + stats::rnorm(nrow(d), 0, 0.2) + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, "lv", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "y", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = 1L, quiet = TRUE)) + overall <- stats::cor(d$y, synthetic(m)$y) + expect_lt(abs(overall), 0.7) + # The overall figure alone reads low; the rows of the small levels do not. + expect_identical(m@audit$leakage_class[m@audit$col == "y"], "medium") +}) + +test_that("a lone small level is not shifted to its own value", { + withr::local_seed(12L) + n <- 300L + d <- data.frame( + gen = paste0("G", sample(5L, n + 1L, TRUE)), + lv = c(paste0("L", rep(seq_len(n / 10L), each = 10L)), "rare") + ) + d$y <- rep(stats::rnorm(n / 10L + 1L, 10, 1), c(rep(10L, n / 10L), 1L)) + + stats::rnorm(n + 1L, 0, 0.2) + d$y[n + 1L] <- 24 + d$env <- rep(c("a", "b", "c"), length.out = n + 1L) + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("lv", "env"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "y", role = "outcome", action = "scramble") + rare <- vapply(1:12, function(s) { + synthetic(suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = s, quiet = TRUE)))$y[n + 1L] + }, numeric(1)) + expect_lt(abs(mean(rare) - 10), 6) +}) + +test_that("strong sites do not block a design clone; within-cell tracking does", { + sim <- function(site_sd, within_share) { + withr::with_seed(21L, { + d <- expand.grid(site = paste0("S", 1:6), rep = paste0("R", 1:3), block = paste0("B", 1:5), + plot = 1:6, stringsAsFactors = FALSE) + d$block <- paste(d$site, d$rep, d$block, sep = "_") + d$rep <- paste(d$site, d$rep, sep = "_") + d$plot <- NULL + d$gen <- paste0("G", sample(30L, nrow(d), TRUE)) + s <- stats::setNames(stats::rnorm(6L, 0, site_sd), paste0("S", 1:6)) + b <- stats::setNames(stats::rnorm(90L, 0, within_share), unique(d$block)) + d$y <- s[d$site] + b[d$block] + stats::rnorm(nrow(d), 0, 1) + d + }) + } + run <- function(d) { + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "y", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", conditional = "design", + seed = 1L, quiet = TRUE)) + m@audit$leakage_class[m@audit$col == "y"] + } + expect_false(identical(run(sim(25, 0.3)), "high")) + expect_identical(run(sim(1, 15)), "high") +}) diff --git a/tests/testthat/test-conditional-partition.R b/tests/testthat/test-conditional-partition.R new file mode 100644 index 0000000..1cd5dc9 --- /dev/null +++ b/tests/testthat/test-conditional-partition.R @@ -0,0 +1,167 @@ +# conditional = "design" and an explicit conditioning partition (N04). + +.fx_blocks <- function(seed = 1L) { + withr::with_seed(seed, { + d <- expand.grid( + rep = paste0("R", 1:3), block = paste0("B", 1:4), plot = 1:6, + stringsAsFactors = FALSE + ) + d$block <- paste(d$rep, d$block, sep = "_") + d$gen <- paste0("G", sample(rep(1:8, length.out = nrow(d)))) + blk <- stats::setNames(stats::rnorm(12L, 0, 3), unique(d$block)) + gen <- stats::setNames(stats::rnorm(8L, 0, 3), paste0("G", 1:8)) + d$y <- 50 + blk[d$block] + gen[d$gen] + stats::rnorm(nrow(d)) + d$plot <- NULL + d + }) +} + +.fx_block_roles <- function(d) { + r <- propose_roles(d, detect = FALSE) + r <- set_role(r, c("rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + set_role(r, "y", role = "outcome", action = "scramble") +} + +.mask_q <- function(...) { + suppressWarnings(mask(..., mode = "local", quiet = TRUE)) +} + +test_that("conditional = 'design' carries design means and nulls treatment", { + d <- .fx_blocks() + r <- .fx_block_roles(d) + block_sd <- function(x) stats::sd(tapply(x$y, x$block, mean)) + gen_sd <- function(x) stats::sd(tapply(x$y, x$gen, mean)) + runs <- lapply(1:30, function(s) { + synthetic(.mask_q(d, r, seed = s, conditional = "design")) + }) + b <- vapply(runs, block_sd, numeric(1)) + g <- vapply(runs, gen_sd, numeric(1)) + expect_lt(abs(mean(b) / block_sd(d) - 1), 0.15) + # The real genotype spread is about 3; a null clone holds only sampling noise. + expect_lt(mean(g), 0.5 * gen_sd(d)) + + # Falsifier: TRUE keeps the genotype spread, FALSE loses the block spread. + cond <- synthetic(.mask_q(d, r, seed = 1L, conditional = TRUE)) + expect_gt(gen_sd(cond), 0.7 * gen_sd(d)) + marg <- lapply(1:10, function(s) { + synthetic(.mask_q(d, r, seed = s, conditional = FALSE)) + }) + # A pooled clone's block means hold only noise: sd(y) / sqrt(6 plots). + expect_lt(mean(vapply(marg, block_sd, numeric(1))), + 1.1 * stats::sd(d$y) / sqrt(6)) + expect_gt(block_sd(d), 1.5 * stats::sd(d$y) / sqrt(6)) +}) + +test_that("the recipe records the design partition without the treatment", { + d <- .fx_blocks() + m <- .mask_q(d, .fx_block_roles(d), seed = 2L, conditional = "design") + rec <- recipe(m) + expect_true(rec@conditional) + expect_setequal(rec@conditioning_cols, c("rep", "block")) + expect_false("gen" %in% rec@conditioning_used) + expect_match(.recipe_fidelity_line(rec), "treatment effect not carried") +}) + +test_that("an explicit partition is used, and TRUE is unchanged by the option", { + d <- .fx_blocks() + r <- .fx_block_roles(d) + a <- .mask_q(d, r, seed = 3L, conditional = c("rep", "block")) + b <- .mask_q(d, r, seed = 3L, conditional = "design") + expect_identical(synthetic(a), synthetic(b)) + + e <- .mask_q(d, r, seed = 3L, conditional = "rep") + expect_identical(recipe(e)@conditioning_used, "rep") + # gen x rep holds three rows a cell: the treatment stays, rep is given up. + f <- .mask_q(d, r, seed = 3L, conditional = c("gen", "rep")) + expect_setequal(recipe(f)@conditioning_cols, c("gen", "rep")) + expect_identical(recipe(f)@conditioning_used, "gen") + + t1 <- .mask_q(d, r, seed = 3L, conditional = TRUE) + t2 <- .mask_q(d, r, seed = 3L, conditional = c("rep", "block", "gen")) + expect_identical( + data.table::setcolorder(data.table::copy(synthetic(t1)), names(d)), + data.table::setcolorder(data.table::copy(synthetic(t2)), names(d)) + ) +}) + +test_that("structure is preserved and the recipe round-trips", { + d <- .fx_blocks() + d$y[c(3L, 40L)] <- NA + r <- .fx_block_roles(d) + m <- .mask_q(d, r, seed = 4L, conditional = "design") + s <- synthetic(m) + expect_identical(s$rep, d$rep) + expect_identical(s$block, d$block) + expect_identical(s$gen, d$gen) + expect_identical(is.na(s$y), is.na(d$y)) + expect_identical(apply_recipe(d, recipe(m))$gen, s$gen) +}) + +test_that("the ladder coarsens a thin design partition and warns", { + d <- .fx_blocks() + d$row <- seq_len(nrow(d)) + r <- propose_roles(d, detect = FALSE) + r <- set_role(r, c("rep", "block", "row"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + expect_warning( + m <- mask(d, r, mode = "local", seed = 5L, quiet = TRUE, + conditional = "design"), + class = "masque_conditional_degraded" + ) + expect_false("row" %in% recipe(m)@conditioning_used) +}) + +test_that("with no design column left the design option warns and pools", { + d <- .fx_blocks() + r <- propose_roles(d, detect = FALSE) + r <- set_role(r, c("rep", "block"), role = "covariate", action = "scramble") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + expect_warning( + m <- mask(d, r, mode = "local", seed = 6L, quiet = TRUE, + conditional = "design"), + class = "masque_conditional_degraded" + ) + expect_length(recipe(m)@conditioning_used, 0L) +}) + +test_that("a bad partition is refused by class", { + d <- .fx_blocks() + r <- .fx_block_roles(d) + cls <- "masque_bad_conditional_refusal" + expect_error(.mask_q(d, r, conditional = "nope"), class = cls) + expect_error(.mask_q(d, r, conditional = c("rep", "y")), class = cls) + expect_error(.mask_q(d, r, conditional = character()), class = cls) + expect_error(.mask_q(d, r, conditional = NA_character_), class = cls) + expect_error(.mask_q(d, r, conditional = c("rep", "rep")), class = cls) + expect_error(.mask_q(d, r, conditional = 1), class = cls) + r2 <- set_role(r, "rep", role = "design", action = "drop") + expect_error(.mask_q(d, r2, conditional = "rep"), class = cls) +}) + +test_that("the design option leaves linked events unstratified by treatment", { + n <- 200L + d <- withr::with_seed(8L, data.frame( + arm = rep(c("a", "b"), length.out = n), + centre = rep(c("c1", "c2", "c3", "c4"), each = n / 4L), + t1 = stats::rexp(n, 0.2) + )) + d$t2 <- d$t1 + withr::with_seed(9L, stats::rexp(n, 0.1)) + d$s1 <- withr::with_seed(10L, stats::rbinom(n, 1L, ifelse(d$arm == "a", 0.9, 0.1))) + d$s2 <- 1L + r <- propose_roles(d, detect = FALSE) + r <- set_role(r, "centre", role = "design", action = "keep") + r <- set_role(r, "arm", role = "treatment", action = "keep") + r <- set_role(r, c("t1", "t2", "s1", "s2"), role = "outcome", action = "scramble") + ev <- list(list(time = c("t1", "t2"), status = c("s1", "s2"))) + gap <- function(m) { + s <- synthetic(m) + abs(mean(s$s1[s$arm == "a"]) - mean(s$s1[s$arm == "b"])) + } + des <- .mask_q(d, r, seed = 1L, conditional = "design", events = ev) + tru <- .mask_q(d, r, seed = 1L, conditional = TRUE, events = ev) + expect_lt(gap(des), 0.25) + expect_gt(gap(tru), 0.6) +}) diff --git a/tests/testthat/test-controls-default-unit.R b/tests/testthat/test-controls-default-unit.R new file mode 100644 index 0000000..83f0e3c --- /dev/null +++ b/tests/testthat/test-controls-default-unit.R @@ -0,0 +1,149 @@ +# The plan's default unit, and the warnings on a control that cannot read as a +# test size. + +confounded <- function(tech = TRUE) { + set.seed(2) + d <- data.frame( + field = rep(paste0("f", 1:3), each = 30L), + tech = rep(paste0("t", 1:9), each = 10L), + plot = 1:90 + ) + d$gen <- paste0("G", match(d$field, paste0("f", 1:3))) + d$y <- c(0, 5, 1)[match(d$field, paste0("f", 1:3))] + stats::rnorm(90L) + design <- if (tech) c("field", "tech") else "field" + if (!tech) d$tech <- NULL + r <- suppressWarnings(propose_roles(d, detect = FALSE)) + r <- set_role(r, design, role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + r <- set_role(r, "plot", role = "id", action = "keep") + list(d = d, r = r) +} + +test_that("a confounder too coarse for a 5 percent level is not the default", { + f <- confounded() + spec <- control_spec(on = "gen", target = "y", effect = list(sd = 0.5), n = 20L, seed = 3L) + m <- mask(f$d, masque_plan(f$r, controls = spec), seed = 4L, quiet = TRUE) + own <- controls(f$d, clone_plan(m), recipe = recipe(m)) + expect_identical(own@spec@unit, "tech") + p <- vapply(1:20, function(i) { + stats::anova(stats::lm(y ~ gen, control_table(own, "negative", i)))["gen", "Pr(>F)"] + }, numeric(1L)) + expect_gt(length(unique(signif(p, 6L))), 1L) + # Column order does not choose. + r2 <- f$r[rev(seq_len(nrow(f$r))), ] + d2 <- f$d[, rev(names(f$d))] + expect_identical( + recipe(mask(d2, masque_plan(r2, controls = spec), seed = 4L, quiet = TRUE))@plan@controls$default@unit, + "tech" + ) +}) + +test_that("between two legal columns the coarser is chosen, with a warning", { + d <- data.frame( + field = rep(paste0("f", 1:8), each = 8L), + tech = rep(paste0("t", 1:16), each = 4L), + gen = rep(paste0("G", 1:4), each = 16L) + ) + expect_warning( + u1 <- masque:::.default_control_unit(d, "gen", c("field", "tech")), + class = "masque_control_unit_ambiguous" + ) + expect_identical(u1, "field") + expect_identical( + suppressWarnings(masque:::.default_control_unit(d, "gen", c("tech", "field"))), "field" + ) +}) + +test_that("a column with too few units is passed over, with a warning", { + f <- confounded(tech = FALSE) + spec <- control_spec(on = "gen", n = 20L, seed = 3L) + expect_warning( + m <- mask(f$d, masque_plan(f$r, controls = spec), seed = 4L, quiet = TRUE), + class = "masque_control_unit_small" + ) + expect_length(recipe(m)@plan@controls$default@unit, 0L) +}) + +test_that("a named unit with too few arrangements warns and still runs", { + f <- confounded(tech = FALSE) + spec <- control_spec(on = "gen", unit = "field", n = 2L, seed = 3L) + expect_warning(ctl <- controls(f$d, f$r, spec), class = "masque_control_unit_small") + expect_identical(ctl@spec@unit, "field") +}) + +test_that("arrangements are counted within strata and across equal labels", { + id <- 1:3 + expect_false(masque:::.level_attainable(id, c("a", "b", "c"), rep(1L, 3L))) + expect_false(masque:::.level_attainable(1:6, c("a", "b", "c", "d", "e", "f"), rep(1L, 6L))) + expect_true(masque:::.level_attainable(1:72, rep(paste0("G", 1:24), 3L), rep(1:3, each = 24L))) + # Two equal labels: choose(6, 3) = 20 arrangements are 10 partitions. + expect_false(masque:::.level_attainable(1:6, rep(c("a", "b"), each = 3L), rep(1L, 6L))) + expect_false(masque:::.level_attainable(1:6, c("a", "a", "b", "b", "b", "b"), rep(1:2, each = 3L))) + expect_true(masque:::.level_attainable(1:7, c(rep("a", 3L), rep("b", 4L)), rep(1L, 7L))) +}) + +test_that("label symmetry counts across strata, and a unit's NA row does not drop it", { + # Paired, five blocks: 32 arrangements, 16 partitions. + pair <- rep(c("a", "b"), 5L) + expect_false(masque:::.level_attainable(1:10, pair, rep(1:5, each = 2L))) + # A randomised block of three treatments in two blocks: 36 arrangements, 6 partitions. + expect_false(masque:::.level_attainable(1:6, rep(c("a", "b", "c"), 2L), rep(1:2, each = 3L))) + # Eight units whose first rows hold NA are still eight units. + unit <- rep(1:8, each = 2L) + on <- rep(c("a", "a", "b", "b", "c", "c", "d", "d"), each = 2L) + on[seq(1L, 15L, by = 2L)] <- NA + expect_true(masque:::.level_attainable(unit, on, rep(1L, 16L))) +}) + +test_that("strata holding one level warn from a tenth of the strata", { + d <- data.frame( + site = rep(paste0("S", 1:5), each = 8L), + trt = c(rep("a", 8L), rep("b", 8L), rep(c("a", "b"), 12L)), + y = seq_len(40L) + 0.5 + ) + r <- suppressWarnings(propose_roles(d, detect = FALSE)) + r <- set_role(r, "site", role = "design", action = "keep") + r <- set_role(r, "trt", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + expect_warning( + ctl <- controls(d, r, control_spec(on = "trt", strata = "site", seed = 1L)), + class = "masque_control_strata_weak" + ) + expect_identical(ctl@manifest$strata_constant_share[1L], 0.4) +}) + +test_that("the stored absence of strata goes back through the constructor", { + f <- confounded() + m <- mask(f$d, masque_plan(f$r, controls = control_spec(on = "gen", n = 2L, seed = 3L)), + seed = 4L, quiet = TRUE) + s <- recipe(m)@plan@controls$default + expect_length(s@strata, 0L) + again <- control_spec(on = "gen", strata = s@strata, unit = s@unit, n = 2L, seed = 3L) + expect_length(again@strata, 0L) +}) + +test_that("rounding that changes the declared shape warns", { + set.seed(4) + d <- data.frame(gen = rep(c("a", "b", "c"), 20L), y = as.integer(rpois(60L, 10))) + r <- suppressWarnings(propose_roles(d, detect = FALSE)) + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + expect_warning( + ctl <- controls(d, r, control_spec(on = "gen", target = "y", effect = c(a = 0.4, b = -0.2), seed = 1L)), + class = "masque_control_shape_rounded" + ) + expect_identical(ctl@truth$shift_realised[ctl@truth$level == "a"], -ctl@truth$shift_realised[ctl@truth$level == "b"]) + many <- data.frame(gen = rep(paste0("G", 1:24), 3L), y = as.integer(rpois(72L, 10))) + rm <- suppressWarnings(propose_roles(many, detect = FALSE)) + rm <- set_role(rm, "gen", role = "treatment", action = "keep") + rm <- set_role(rm, "y", role = "outcome", action = "scramble") + expect_warning( + controls(many, rm, control_spec(on = "gen", target = "y", effect = stats::setNames(rep(c(0.4, -0.2), 12L), paste0("G", 1:24)), seed = 1L)), + class = "masque_control_shape_rounded" + ) + expect_no_warning( + controls(d, r, control_spec(on = "gen", target = "y", effect = c(a = 2, b = -1), seed = 1L)), + class = "masque_control_shape_rounded" + ) +}) diff --git a/tests/testthat/test-controls-events.R b/tests/testthat/test-controls-events.R new file mode 100644 index 0000000..0838dbc --- /dev/null +++ b/tests/testthat/test-controls-events.R @@ -0,0 +1,33 @@ +# A declared linked-event pair moves as one row block in the null, so the +# time order within a row and the status patterns survive. + +test_that("a declared event pair moves as one block in the null", { + df <- .events_fixture(n = 300L) + r <- .events_roles(df) + spec <- control_spec(on = "arm", seed = 1L) + ctl <- controls(df, r, spec, events = .events_spec) + nul <- control_table(ctl, "null") + key_in <- paste(df$time_rec, df$status_rec, df$time_death, df$status_death) + key_out <- paste(nul$time_rec, nul$status_rec, nul$time_death, nul$status_death) + expect_identical(sort(key_in), sort(key_out)) + expect_false(identical(key_in, key_out)) + expect_true(all(nul$time_rec <= nul$time_death)) + expect_identical(nul$id, df$id) + expect_identical(nul$arm, df$arm) +}) + +test_that("without the declaration the pair breaks, and an event column is no target", { + df <- .events_fixture(n = 300L) + r <- .events_roles(df) + ctl <- controls(df, r, control_spec(on = "arm", seed = 1L)) + nul <- control_table(ctl, "null") + key_in <- paste(df$time_rec, df$status_rec, df$time_death, df$status_death) + key_out <- paste(nul$time_rec, nul$status_rec, nul$time_death, nul$status_death) + expect_false(identical(sort(key_in), sort(key_out))) + .ctl_expect_refusal( + controls(df, r, control_spec( + on = "arm", target = "time_rec", effect = c(B = 1), seed = 1L + ), events = .events_spec), + "masque_control_target_kind_refusal" + ) +}) diff --git a/tests/testthat/test-controls-negative.R b/tests/testthat/test-controls-negative.R new file mode 100644 index 0000000..cfbcd80 --- /dev/null +++ b/tests/testthat/test-controls-negative.R @@ -0,0 +1,142 @@ +# The negative control: `on` permuted within strata at its unit, nothing +# else touched. + +test_that("the negative permutes on within strata and keeps everything else", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec()) + neg <- control_table(ctl, "negative", 1L) + for (cl in setdiff(names(df), "gen")) { + expect_identical(neg[[cl]], df[[cl]], info = cl) + } + stratum <- paste(df$trial, df$rep, df$block) + for (s in unique(stratum)) { + expect_identical(sort(neg$gen[stratum == s]), sort(df$gen[stratum == s])) + } + expect_false(identical(neg$gen, df$gen)) + expect_identical(ctl@manifest$n_strata[1L], 24L) + expect_identical(ctl@manifest$strata_constant_share[1L], 0) + expect_identical(attr(neg, "masque_control")$replicate, 1L) +}) + +test_that("replicates differ and the treatment-by-stratum incidence is kept", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec(n = 3L)) + n1 <- control_table(ctl, "negative", 1L) + n2 <- control_table(ctl, "negative", 2L) + expect_false(identical(n1$gen, n2$gen)) + expect_identical(table(n1$trial, n1$gen), table(df$trial, df$gen)) +}) + +test_that("a whole-plot on is permuted at its unit within coarser strata", { + df <- .ctl_met() + r <- .ctl_roles(df) + spec <- control_spec( + on = "irrig", unit = c("trial", "rep"), strata = "trial", n = 1L, seed = 9L + ) + ctl <- controls(df, r, spec) + neg <- control_table(ctl, "negative", 1L) + unit <- masque:::.unit_id(df, c("trial", "rep")) + expect_true(masque:::.constant_within(unit, neg$irrig)) + for (t in unique(df$trial)) { + u_in <- unique(df[df$trial == t, c("rep", "irrig")])$irrig + u_out <- unique(as.data.frame(neg)[neg$trial == t, c("rep", "irrig")])$irrig + expect_identical(sort(u_out), sort(u_in)) + } + expect_identical(ctl@manifest$unit[1L], "trial + rep") +}) + +test_that("a degenerate restricted permutation is refused", { + df <- .ctl_met() + r <- .ctl_roles(df) + spec <- control_spec( + on = "irrig", strata = c("trial", "rep", "block"), seed = 1L + ) + .ctl_expect_refusal(controls(df, r, spec), "masque_control_degenerate_refusal") +}) + +test_that("on constant in most strata warns", { + df <- data.frame( + site = rep(c("S1", "S2", "S3", "S4"), each = 4L), + trt = c(rep("a", 4L), rep("b", 4L), rep("a", 4L), c("a", "a", "b", "b")), + y = seq_len(16L) + 0.5, + stringsAsFactors = FALSE + ) + r <- suppressWarnings(propose_roles(df, detect = FALSE)) + r <- set_role(r, "site", role = "design", action = "keep") + r <- set_role(r, "trt", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + spec <- control_spec(on = "trt", strata = "site", seed = 1L) + expect_warning( + expect_warning(ctl <- controls(df, r, spec), class = "masque_control_strata_weak"), + class = "masque_control_unit_small" + ) + expect_identical(ctl@manifest$strata_constant_share[1L], 0.75) +}) + +test_that("unit and strata must nest, be design columns, and hold on constant", { + df <- .ctl_met() + r <- .ctl_roles(df) + .ctl_expect_refusal( + controls(df, r, control_spec( + on = "irrig", unit = c("trial", "rep"), strata = "block", seed = 1L + )), + "masque_control_strata_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "gen", unit = "block", seed = 1L)), + "masque_control_unit_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "gen", strata = "gen", seed = 1L)), + "masque_control_strata_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "gen", unit = "soil", seed = 1L)), + "masque_control_unit_refusal" + ) +}) + +test_that("on defaults to the single treatment and is checked by role and kind", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, control_spec(strata = "trial", seed = 1L)) + expect_identical(ctl@spec@on, "gen") + + r_two <- set_role(r, "irrig", role = "treatment", action = "keep") + .ctl_expect_refusal( + controls(df, r_two, control_spec(strata = "trial", seed = 1L)), + "masque_control_on_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "trial", seed = 1L)), + "masque_control_on_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "yield", seed = 1L)), + "masque_control_on_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "plot_cov", seed = 1L)), + "masque_control_on_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec(on = "nope", seed = 1L)), + "masque_control_on_refusal" + ) +}) + +test_that("a whole-valued numeric treatment can be on", { + df <- .ctl_two_arm(n = 40L) + df$dose <- rep(c(50L, 100L), 20L) + r <- .ctl_two_arm_roles(df) + r <- set_role(r, "trt", role = "covariate", action = "scramble") + r <- set_role(r, "dose", role = "treatment", action = "keep") + ctl <- controls(df, r, control_spec( + target = "y", effect = c(`100` = 0.5), seed = 1L + )) + expect_identical(ctl@spec@on, "dose") + expect_identical(ctl@truth$level, c("50", "100")) + expect_identical(ctl@truth$effect_sd, c(0, 0.5)) +}) diff --git a/tests/testthat/test-controls-null.R b/tests/testthat/test-controls-null.R new file mode 100644 index 0000000..43cbe80 --- /dev/null +++ b/tests/testthat/test-controls-null.R @@ -0,0 +1,111 @@ +# The null control: every analysed column permuted at its grain, nothing else +# touched, every association removed. + +test_that("the null keeps structure, NA mask, classes and multisets", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec()) + nul <- control_table(ctl, "null") + + expect_identical(names(nul), names(df)) + expect_identical(nrow(nul), nrow(df)) + for (cl in c("trial", "rep", "block", "gen")) { + expect_identical(nul[[cl]], df[[cl]]) + } + expect_identical( + unname(is.na(as.data.frame(nul))), unname(is.na(df)) + ) + expect_identical( + vapply(nul, function(x) class(x)[1L], character(1L)), + vapply(df, function(x) class(x)[1L], character(1L)) + ) + for (cl in c("yield", "count", "plot_cov", "irrig", "aet", "soil", "sow")) { + expect_true(.ctl_same_multiset(nul[[cl]], df[[cl]]), info = cl) + } + expect_identical(attr(nul, "masque_control")$kind, "null") + expect_true(is.na(attr(nul, "masque_control")$replicate)) +}) + +test_that("a declared unit-grained column stays constant within its unit", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec()) + nul <- control_table(ctl, "null") + trial_id <- masque:::.unit_id(df, "trial") + for (cl in c("aet", "soil", "sow")) { + expect_true(masque:::.constant_within(trial_id, nul[[cl]]), info = cl) + } + # The all-missing trial stays missing; the unit values are the same set. + expect_true(all(is.na(nul$aet[nul$trial == "T2"]))) + expect_identical( + sort(unique(nul$aet[!is.na(nul$aet)])), + sort(unique(df$aet[!is.na(df$aet)])) + ) + # The unit values moved: at least one trial holds another trial's value. + expect_false(identical(nul$aet, df$aet) && identical(nul$soil, df$soil)) +}) + +test_that("a column not constant within its declared unit is row-permuted with a note", { + df <- .ctl_met() + r <- .ctl_roles(df) + spec <- control_spec( + on = "gen", strata = c("trial", "rep", "block"), + grain = list(plot_cov = "trial"), seed = 3L + ) + ctl <- controls(df, r, spec) + expect_length(ctl@notes, 1L) + expect_match(ctl@notes, "plot_cov is not constant within trial", fixed = TRUE) + nul <- control_table(ctl, "null") + expect_true(.ctl_same_multiset(nul$plot_cov, df$plot_cov)) +}) + +test_that("the null removes the association among analysed columns", { + df <- .ctl_met(n_trial = 5L, n_gen = 20L) + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec()) + nul <- control_table(ctl, "null") + before <- stats::cor(df$yield, df$plot_cov, use = "complete.obs") + after <- stats::cor(nul$yield, nul$plot_cov, use = "complete.obs") + expect_gt(before, 0.95) + expect_lt(abs(after), 0.3) +}) + +test_that("controls() and control_table() leave the caller's RNG untouched", { + df <- .ctl_met() + r <- .ctl_roles(df) + set.seed(123) + before <- .Random.seed + ctl <- controls(df, r, .ctl_spec()) + nul <- control_table(ctl, "null") + neg <- control_table(ctl, "negative", 1L) + expect_identical(.Random.seed, before) +}) + +test_that("the same seed gives the same null on data.frame and data.table input", { + df <- .ctl_met() + r <- .ctl_roles(df) + a <- control_table(controls(df, r, .ctl_spec()), "null") + b <- control_table(controls(data.table::as.data.table(df), r, .ctl_spec()), "null") + expect_identical(a, b) + c3 <- control_table(controls(df, r, .ctl_spec(seed = 4L)), "null") + expect_false(identical(a$yield, c3$yield)) +}) + +test_that("a kept outcome is reported and left alone; no analysed column is refused", { + df <- .ctl_met() + r <- .ctl_roles(df) + r_keep <- set_role(r, "yield", role = "outcome", action = "keep") + expect_warning( + ctl <- controls(df, r_keep, .ctl_spec()), + class = "masque_control_kept_outcome" + ) + expect_identical(control_table(ctl, "null")$yield, df$yield) + + r_all <- r + r_all$action <- "keep" + err <- tryCatch( + suppressWarnings(controls(df, r_all, .ctl_spec())), + error = function(e) e + ) + expect_true(inherits(err, "masque_control_analysed_refusal")) +}) diff --git a/tests/testthat/test-controls-owner-key.R b/tests/testthat/test-controls-owner-key.R new file mode 100644 index 0000000..7a24dc6 --- /dev/null +++ b/tests/testthat/test-controls-owner-key.R @@ -0,0 +1,105 @@ +# The owner's draw: a 256-bit key, used whole, and no integer to search. + +owner_fixture <- function() { + set.seed(5) + d <- expand.grid(gen = paste0("g", 1:6), rep = paste0("r", 1:6), stringsAsFactors = FALSE) + d$y <- stats::rnorm(36L) + r <- suppressWarnings(propose_roles(d, detect = FALSE)) + r <- set_role(r, "rep", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + list(d = d, r = r, spec = control_spec(on = "gen", strata = "rep", n = 3L, seed = 99L)) +} + +test_that("the owner key is 256 bits from the system source", { + key <- masque:::.new_owner_key() + expect_match(key, "^[0-9a-f]{64}$") + expect_false(identical(key, masque:::.new_owner_key())) + set.seed(1) + a <- masque:::.new_owner_key() + set.seed(1) + expect_false(identical(a, masque:::.new_owner_key())) +}) + +test_that("the key is the system's bytes and nothing else", { + testthat::local_mocked_bindings(.system_entropy = function() as.raw(1:32), .package = "masque") + expect_identical(masque:::.new_owner_key(), paste(as.character(as.raw(1:32)), collapse = "")) +}) + +test_that("a recipe draw is the whole digest and holds no integer seed", { + draw <- masque:::.control_source_draw(strrep("ab", 32L), 99L, NULL) + expect_null(draw$seed) + expect_match(draw$key, "^[0-9a-f]{64}$") + f <- owner_fixture() + m <- mask(f$d, masque_plan(f$r, controls = f$spec), seed = 1L, quiet = TRUE) + own <- controls(f$d, clone_plan(m), recipe = recipe(m)) + expect_true(all(is.na(own@seeds))) + expect_true(all(is.na(own@manifest$seed))) + expect_false(any(grepl(substr(own@key, 1L, 12L), utils::capture.output(print(own)), fixed = TRUE))) + expect_length(utils::capture.output(utils::str(own)), 1L) + # A key differing in its last digit gives other permutations. + other <- own + last <- substr(own@key, 64L, 64L) + other@key <- paste0(substr(own@key, 1L, 63L), if (last == "0") "1" else "0") + expect_false(identical( + control_table(own, "negative", 1L)$gen, control_table(other, "negative", 1L)$gen + )) +}) + +test_that("a keyed table ignores the R stream and leaves it alone", { + f <- owner_fixture() + m <- mask(f$d, masque_plan(f$r, controls = f$spec), seed = 1L, quiet = TRUE) + own <- controls(f$d, clone_plan(m), recipe = recipe(m)) + set.seed(1) + a <- control_table(own, "negative", 2L) + before <- .Random.seed + set.seed(2) + b <- control_table(own, "negative", 2L) + expect_identical(a$gen, b$gen) + set.seed(1) + invisible(control_table(own, "negative", 2L)) + expect_identical(.Random.seed, before) + expect_identical(sort(a$gen), sort(f$d$gen)) + expect_true(all(tapply(a$gen, a$rep, function(v) length(unique(v))) == 6L)) +}) + +test_that("keyed permutations are uniform", { + draws <- vapply(seq_len(6000L), function(i) { + paste(masque:::.keyed_perm(paste("k", i), 3L), collapse = "") + }, character(1L)) + counts <- table(draws) + expect_length(counts, 6L) + expect_gt(stats::chisq.test(counts)$p.value, 1e-4) + expect_identical(sort(masque:::.keyed_perm("k", 50L)), 1:50) + expect_identical(masque:::.keyed_perm("k", 1L), 1L) +}) + +test_that("with no system source the mask warns and controls ask for a seed", { + f <- owner_fixture() + testthat::local_mocked_bindings(.system_entropy = function() NULL, .package = "masque") + expect_identical(masque:::.new_owner_key(), character()) + expect_warning( + m <- mask(f$d, masque_plan(f$r, controls = f$spec), seed = 1L, quiet = TRUE), + class = "masque_owner_key_unavailable" + ) + expect_error( + controls(f$d, clone_plan(m), recipe = recipe(m)), class = "masque_control_seed_refusal" + ) + own <- controls(f$d, clone_plan(m), recipe = recipe(m), seed = 7L) + expect_false(anyNA(own@seeds)) +}) + +test_that("an explicit integer seed keeps its stream", { + f <- owner_fixture() + ctl <- controls(f$d, f$r, f$spec, seed = 7L) + seeds <- as.integer(masque:::.sub_seeds(7L, 4L)) + expect_identical(ctl@seeds, seeds) + out <- withr::with_seed(seeds[2L], { + perm <- integer(36L) + for (g in split(seq_len(36L), masque:::.unit_id(f$d, "rep"))) { + perm[g] <- g[sample.int(length(g))] + } + f$d$gen[perm] + }) + expect_identical(control_table(ctl, "negative", 1L)$gen, out) +}) diff --git a/tests/testthat/test-controls-positive.R b/tests/testthat/test-controls-positive.R new file mode 100644 index 0000000..9c8415e --- /dev/null +++ b/tests/testthat/test-controls-positive.R @@ -0,0 +1,138 @@ +# The positive control: the negative plus the declared per-level shift on +# each target. Positive minus negative is the planted truth exactly. + +test_that("positive minus negative equals the realised shift, row by row", { + df <- .ctl_met() + r <- .ctl_roles(df) + spec <- .ctl_spec(target = c("yield", "count"), effect = c(G01 = 0.5, G02 = -0.5)) + ctl <- controls(df, r, spec) + neg <- control_table(ctl, "negative", 2L) + pos <- control_table(ctl, "positive", 2L) + for (cl in setdiff(names(df), c("yield", "count"))) { + expect_identical(pos[[cl]], neg[[cl]], info = cl) + } + truth <- ctl@truth + for (tg in c("yield", "count")) { + shift <- truth$shift_realised[truth$target == tg][ + match(pos$gen, truth$level[truth$target == tg]) + ] + diff <- pos[[tg]] - neg[[tg]] + obs <- !is.na(neg[[tg]]) + expect_identical(is.na(pos[[tg]]), is.na(neg[[tg]])) + expect_equal(diff[obs], shift[obs], tolerance = 1e-12) + } + expect_true(is.integer(pos$count)) + expect_true(all(truth$whole_valued[truth$target == "count"])) + expect_identical(nrow(truth), 16L) + expect_identical( + truth$effect_sd[truth$target == "yield"], + c(0.5, -0.5, 0, 0, 0, 0, 0, 0) + ) + expect_equal( + truth$shift_raw[truth$target == "yield" & truth$level == "G01"], + 0.5 * stats::sd(df$yield, na.rm = TRUE) + ) +}) + +test_that("a shift that rounds to zero on a whole-valued target is refused", { + df <- .ctl_met() + r <- .ctl_roles(df) + spec <- .ctl_spec(target = "count", effect = c(G01 = 0.05)) + .ctl_expect_refusal(controls(df, r, spec), "masque_control_shift_vanishes_refusal") +}) + +test_that("a shift below zero on a non-negative target is reported", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec(target = "yield", effect = c(G01 = -30))) + expect_warning( + pos <- control_table(ctl, "positive", 1L), + class = "masque_control_sign_crossed" + ) + expect_true(any(pos$yield < 0, na.rm = TRUE)) +}) + +test_that("list(sd = ) draws the same vector whatever the run seed", { + df <- .ctl_met() + r <- .ctl_roles(df) + spec <- .ctl_spec(target = "yield", effect = list(sd = 0.5)) + a <- controls(df, r, spec) + b <- controls(df, r, spec, seed = 99L) + expect_identical(a@spec@effect, b@spec@effect) + expect_identical(names(a@spec@effect), sprintf("G%02d", 1:8)) + expect_false(identical( + control_table(a, "negative", 1L)$gen, control_table(b, "negative", 1L)$gen + )) + expect_identical(control_table(a, "null"), control_table(a, "null")) +}) + +test_that("a two-arm planted effect is recovered without bias", { + df <- .ctl_two_arm() + r <- .ctl_two_arm_roles(df) + n_rep <- 50L + ctl <- controls(df, r, control_spec( + on = "trt", target = "y", effect = c(B = 1), n = n_rep, seed = 21L + )) + shift <- ctl@truth$shift_realised[ctl@truth$level == "B"] + est <- vapply(seq_len(n_rep), function(i) { + d <- control_table(ctl, "positive", i) + unname(stats::coef(stats::lm(y ~ trt, data = d))["trtB"]) + }, numeric(1L)) + est_null <- vapply(seq_len(n_rep), function(i) { + d <- control_table(ctl, "negative", i) + unname(stats::coef(stats::lm(y ~ trt, data = d))["trtB"]) + }, numeric(1L)) + se <- stats::sd(est) / sqrt(n_rep) + expect_lt(abs(mean(est) - shift), 3 * se) + expect_lt(abs(mean(est_null)), 3 * stats::sd(est_null) / sqrt(n_rep)) +}) + +test_that("a many-level planted vector is recovered in rank", { + df <- .ctl_many_levels() + r <- .ctl_many_roles(df) + n_rep <- 50L + ctl <- controls(df, r, control_spec( + on = "lev", target = "y", effect = list(sd = 1), n = n_rep, seed = 22L + )) + truth <- ctl@truth$effect_sd + rho <- vapply(seq_len(n_rep), function(i) { + d <- control_table(ctl, "positive", i) + means <- tapply(d$y, d$lev, mean) + stats::cor(truth, means[ctl@truth$level], method = "spearman") + }, numeric(1L)) + expect_gte(sum(rho >= 0.9), 45L) +}) + +test_that("targets are checked by kind, grain, events and against on", { + df <- .ctl_met() + r <- .ctl_roles(df) + .ctl_expect_refusal( + controls(df, r, .ctl_spec(target = "soil", effect = c(G01 = 1))), + "masque_control_target_kind_refusal" + ) + .ctl_expect_refusal( + controls(df, r, .ctl_spec(target = "irrig", effect = c(G01 = 1))), + "masque_control_target_kind_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec( + on = "gen", target = "aet", effect = c(G01 = 1), grain = list(aet = "trial"), + seed = 1L + )), + "masque_control_target_grain_refusal" + ) + .ctl_expect_refusal( + controls(df, r, control_spec( + on = "irrig", target = "irrig", effect = c(wet = 1), seed = 1L + )), + "masque_control_on_refusal" + ) + .ctl_expect_refusal( + controls(df, r, .ctl_spec(target = "nope", effect = c(G01 = 1))), + "masque_control_target_refusal" + ) + .ctl_expect_refusal( + controls(df, r, .ctl_spec(target = "yield", effect = c(G99 = 1))), + "masque_control_effect_refusal" + ) +}) diff --git a/tests/testthat/test-controls-spec.R b/tests/testthat/test-controls-spec.R new file mode 100644 index 0000000..360b6c5 --- /dev/null +++ b/tests/testthat/test-controls-spec.R @@ -0,0 +1,83 @@ +# control_spec(): the declaration is validated before it meets a table. + +test_that("the seed is required", { + .ctl_expect_refusal(control_spec(on = "gen"), "masque_control_seed_refusal") + .ctl_expect_refusal( + control_spec(on = "gen", seed = NULL), "masque_control_seed_refusal" + ) + .ctl_expect_refusal( + control_spec(on = "gen", seed = 1.5), "masque_bad_seed_refusal" + ) +}) + +test_that("target and effect come together", { + .ctl_expect_refusal( + control_spec(target = "y", seed = 1L), "masque_control_effect_refusal" + ) + .ctl_expect_refusal( + control_spec(effect = c(A = 1), seed = 1L), "masque_control_effect_refusal" + ) +}) + +test_that("effect forms are a named vector or list(sd = )", { + .ctl_expect_refusal( + control_spec(target = "y", effect = 0.5, seed = 1L), + "masque_control_effect_refusal" + ) + .ctl_expect_refusal( + control_spec(target = "y", effect = c(A = 1, A = 2), seed = 1L), + "masque_control_effect_refusal" + ) + .ctl_expect_refusal( + control_spec(target = "y", effect = c(A = Inf), seed = 1L), + "masque_control_effect_refusal" + ) + .ctl_expect_refusal( + control_spec(target = "y", effect = list(sd = -1), seed = 1L), + "masque_control_effect_refusal" + ) + .ctl_expect_refusal( + control_spec(target = "y", effect = list(scale = 1), seed = 1L), + "masque_control_effect_refusal" + ) + s1 <- control_spec(target = "y", effect = c(A = 1L), seed = 1L) + expect_identical(s1@effect, c(A = 1)) + s2 <- control_spec(target = "y", effect = list(sd = 2L), seed = 1L) + expect_identical(s2@effect, list(sd = 2)) +}) + +test_that("n, on, strata and grain are checked", { + .ctl_expect_refusal(control_spec(n = 0L, seed = 1L), "masque_control_n_refusal") + .ctl_expect_refusal(control_spec(n = 1.5, seed = 1L), "masque_control_n_refusal") + .ctl_expect_refusal( + control_spec(on = c("a", "b"), seed = 1L), "masque_control_on_refusal" + ) + .ctl_expect_refusal( + control_spec(strata = c("a", "a"), seed = 1L), "masque_control_strata_refusal" + ) + .ctl_expect_refusal( + control_spec(grain = list("trial"), seed = 1L), "masque_control_grain_refusal" + ) + .ctl_expect_refusal( + control_spec(grain = list(aet = 1L), seed = 1L), "masque_control_grain_refusal" + ) + s <- control_spec(n = 19, seed = 2026, unit = "trial") + expect_identical(s@n, 19L) + expect_identical(s@seed, 2026L) + expect_identical(s@unit, "trial") + expect_identical(s@strata, character()) +}) + +test_that("the spec prints its declaration", { + s <- control_spec( + on = "gen", target = "yield", effect = c(G01 = 0.5), + strata = c("rep", "block"), grain = list(aet = "trial"), n = 19L, + seed = 1L + ) + out <- capture_full(print(s)) + expect_match(out, "Replicates: 19", fixed = TRUE) + expect_match(out, "G01 +0.5", fixed = TRUE) + expect_match(out, "aet within trial", fixed = TRUE) + s2 <- control_spec(target = "y", effect = list(sd = 0.3), seed = 1L) + expect_match(capture_full(print(s2)), "drawn from N(0, 0.3^2)", fixed = TRUE) +}) diff --git a/tests/testthat/test-controls-table.R b/tests/testthat/test-controls-table.R new file mode 100644 index 0000000..006901b --- /dev/null +++ b/tests/testthat/test-controls-table.R @@ -0,0 +1,73 @@ +# control_table(): regeneration from seed, argument checks, the manifest and +# the print. + +test_that("a control table is regenerated identically from its seed", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec(target = "yield", effect = c(G01 = 1))) + expect_identical(control_table(ctl, "null"), control_table(ctl, "null")) + expect_identical( + control_table(ctl, "negative", 2L), control_table(ctl, "negative", 2L) + ) + expect_identical( + control_table(ctl, "positive", 2L), control_table(ctl, "positive", 2L) + ) +}) + +test_that("kind and replicate are checked", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec()) + .ctl_expect_refusal(control_table(ctl, "negative", 3L), "masque_control_kind_refusal") + .ctl_expect_refusal(control_table(ctl, "negative", 0L), "masque_control_kind_refusal") + .ctl_expect_refusal(control_table(ctl, "positive", 1L), "masque_control_kind_refusal") + .ctl_expect_refusal(control_table(df, "null"), "masque_control_kind_refusal") + expect_error(control_table(ctl, "blank")) + .ctl_expect_refusal(controls(df, r, list(on = "gen")), "masque_control_spec_refusal") + .ctl_expect_refusal(controls(df[1L, ], r, .ctl_spec()), "masque_control_table_refusal") + .ctl_expect_refusal(controls(df, r, .ctl_spec(), seed = 1.5), "masque_bad_seed_refusal") +}) + +test_that("the manifest lists every dataset with its own seed", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec(n = 3L, target = "yield", effect = c(G01 = 1))) + m <- ctl@manifest + expect_identical(nrow(m), 7L) + expect_identical(m$kind, c("null", rep("negative", 3L), rep("positive", 3L))) + expect_identical(m$replicate, c(NA, 1L, 2L, 3L, 1L, 2L, 3L)) + expect_identical(m$seed[2:4], m$seed[5:7]) + expect_identical(anyDuplicated(m$seed[1:4]), 0L) + expect_identical(m$permuted[2L], "gen") + expect_match(m$permuted[1L], "yield", fixed = TRUE) + expect_identical(m$strata[1L], "trial + rep + block") + expect_identical(m$target[1L], "yield") + ctl0 <- controls(df, r, .ctl_spec()) + expect_identical(nrow(ctl0@manifest), 3L) + expect_true(is.na(ctl0@manifest$target[1L])) +}) + +test_that("the controls object prints its declaration and truth", { + df <- .ctl_met() + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec(target = "yield", effect = c(G01 = 1))) + out <- capture_full(print(ctl)) + expect_match(out, "Datasets: 1 null, 2 negative, 2 positive", fixed = TRUE) + expect_match(out, "Planted truth", fixed = TRUE) + expect_match(out, "24 strata", fixed = TRUE) + ctl2 <- controls(df, r, control_spec( + on = "gen", strata = "trial", grain = list(plot_cov = "trial"), seed = 1L + )) + expect_match(capture_full(print(ctl2)), "Notes", fixed = TRUE) +}) + +test_that("the input table is not modified", { + df <- data.table::as.data.table(.ctl_met()) + before <- data.table::copy(df) + r <- .ctl_roles(df) + ctl <- controls(df, r, .ctl_spec(target = "yield", effect = c(G01 = 1))) + invisible(control_table(ctl, "null")) + invisible(control_table(ctl, "positive", 1L)) + expect_identical(df, before) + expect_identical(ctl@table, before) +}) diff --git a/tests/testthat/test-coord-site-invariant.R b/tests/testthat/test-coord-site-invariant.R index ac2876c..5502890 100644 --- a/tests/testthat/test-coord-site-invariant.R +++ b/tests/testthat/test-coord-site-invariant.R @@ -48,7 +48,7 @@ test_that("the donut contract still holds, now per site", { test_that("a site's true position is not recoverable by averaging its rows", { # The privacy regression. Per-row jitter left the centroid of a 360-row site - # a median 0.58 km from the truth, against a 5 km floor. + # a median 0.64 km from the truth, against a 5 km floor. n <- 360L df <- data.frame(lat = rep(-34.9, n), lon = rep(138.6, n)) err <- vapply(1:10, function(s) { diff --git a/tests/testthat/test-coord-site-keyed.R b/tests/testthat/test-coord-site-keyed.R new file mode 100644 index 0000000..3ccf8b8 --- /dev/null +++ b/tests/testthat/test-coord-site-keyed.R @@ -0,0 +1,179 @@ +# A site's displacement is a function of the seed and the site's value, never +# of where its rows sit in the table. + +.fx_sites <- function(n_sites = 60L, each = 4L) { + lat <- -37 + (seq_len(n_sites) * 0.0773) %% 5 + lon <- 139 + (seq_len(n_sites) * 0.1319) %% 10 + data.frame( + site = rep(sprintf("S%03d", seq_len(n_sites)), each = each), + lat = rep(lat, each = each), + lon = rep(lon, each = each), + stringsAsFactors = FALSE + ) +} + +.fx_moved <- function(df, out, sites = unique(df$site)) { + i <- match(sites, df$site) + cbind(dlat = out$lat[i] - df$lat[i], dlon = out$lon[i] - df$lon[i]) +} + +test_that("shuffled rows keep every site's displacement", { + df <- .fx_sites() + a <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 42L) + shuffled <- df[withr::with_seed(9L, sample.int(nrow(df))), ] + b <- jitter_coordinates(shuffled, "lat", "lon", on_land = FALSE, seed = 42L) + expect_identical(.fx_moved(df, a), .fx_moved(shuffled, b, unique(df$site))) +}) + +test_that("an added or removed site leaves the other sites alone", { + df <- .fx_sites() + a <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 42L) + + extra <- data.frame(site = "NEW", lat = -30, lon = 150) + more <- rbind(extra, df) + b <- jitter_coordinates(more, "lat", "lon", on_land = FALSE, seed = 42L) + expect_identical(.fx_moved(df, a), .fx_moved(more, b, unique(df$site))) + + fewer <- df[df$site != "S001", ] + d <- jitter_coordinates(fewer, "lat", "lon", on_land = FALSE, seed = 42L) + kept <- unique(fewer$site) + expect_identical(.fx_moved(df, a, kept), .fx_moved(fewer, d, kept)) +}) + +test_that("a named site unit is keyed on its value, not its position", { + df <- .fx_sites() + a <- jitter_coordinates( + df, "lat", "lon", by = "site", on_land = FALSE, seed = 42L + ) + shuffled <- rbind(data.frame(site = "NEW", lat = -30, lon = 150), + df[rev(seq_len(nrow(df))), ]) + b <- jitter_coordinates( + shuffled, "lat", "lon", by = "site", on_land = FALSE, seed = 42L + ) + expect_identical(.fx_moved(df, a), .fx_moved(shuffled, b, unique(df$site))) +}) + +test_that("mask() keeps the displacement across a shuffled or extended table", { + df <- .fx_sites() + df$y <- withr::with_seed(1L, stats::rnorm(nrow(df))) + roles <- propose_roles(df, detect = FALSE) + roles$role[roles$col == "site"] <- "design" + roles$action[roles$col == "site"] <- "keep" + roles$role[roles$col == "y"] <- "outcome" + roles$action[roles$col == "y"] <- "scramble" + roles$role[roles$col %in% c("lat", "lon")] <- "covariate" + roles$action[roles$col %in% c("lat", "lon")] <- "keep" + spec <- list(list(lat = "lat", lon = "lon", on_land = FALSE)) + run <- function(d) { + synthetic(suppressWarnings(mask(d, roles, seed = 42L, coords = spec))) + } + a <- run(df) + sh <- df[withr::with_seed(3L, sample.int(nrow(df))), ] + b <- run(sh) + more <- rbind(df, transform(df[1L, ], site = "NEW", lat = -30, lon = 150)) + d <- run(more) + sites <- unique(df$site) + expect_identical(.fx_moved(df, a), .fx_moved(sh, b, sites)) + expect_identical(.fx_moved(df, a), .fx_moved(more, d, sites)) +}) + +test_that("kept rebuilds under one seed add nothing to average", { + df <- .fx_sites() + a <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 42L) + rebuilds <- lapply(1:8, function(s) { + d <- df[withr::with_seed(s, sample.int(nrow(df))), ] + o <- jitter_coordinates(d, "lat", "lon", on_land = FALSE, seed = 42L) + .fx_moved(d, o, unique(df$site)) + }) + for (r in rebuilds) expect_identical(r, .fx_moved(df, a)) +}) + +test_that("the group mean over many seeds sits on the site, with the floor", { + # The same site under many fresh seeds; its rows share one displacement. + df <- data.frame(lat = rep(-34.9, 8L), lon = rep(138.6, 8L)) + seeds <- 1:4000 + moved <- t(vapply(seeds, function(s) { + o <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = s) + if (nrow(unique(o)) != 1L) stop("a site split apart") + c(o$lat[1L] - df$lat[1L], o$lon[1L] - df$lon[1L]) + }, numeric(2))) + north <- moved[, 1L] * 111.32 + east <- moved[, 2L] * 111.32 * cos(-34.9 * pi / 180) + r <- sqrt(north^2 + east^2) + # The per-axis SD of the 5-20 km donut is sqrt((R^2 + r^2) / 4) = 10.31 km. + mcse <- 10.31 / sqrt(length(seeds)) + expect_lt(abs(mean(north)), 3.5 * mcse) + expect_lt(abs(mean(east)), 3.5 * mcse) + expect_gte(min(r), 5 - 0.01) + expect_lte(max(r), 20 + 0.01) + # Area-uniform radius and uniform bearing, by Kolmogorov-Smirnov. + u_r <- (r^2 - 25) / (400 - 25) + expect_gt(stats::ks.test(pmin(pmax(u_r, 0), 1), "punif")$p.value, 0.001) + u_b <- (atan2(east, north) + pi) / (2 * pi) + expect_gt(stats::ks.test(u_b, "punif")$p.value, 0.001) +}) + +test_that("sites are displaced independently within a table", { + df <- .fx_sites(2000L, 1L) + df <- unique(df) + out <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 7L) + m <- .fx_moved(df, out) + expect_lt(abs(stats::cor(m[-1L, "dlat"], m[-nrow(m), "dlat"])), 0.1) + expect_lt(abs(stats::cor(m[, "dlat"], m[, "dlon"])), 0.1) +}) + +test_that("the gaussian scheme is keyed the same way and has the right SD", { + df <- .fx_sites(3000L, 1L) + run <- function(d) { + jitter_coordinates(d, "lat", "lon", method = "gaussian", sd_km = 10, + on_land = FALSE, seed = 5L) + } + a <- run(df) + sh <- df[rev(seq_len(nrow(df))), ] + expect_identical(.fx_moved(df, a), .fx_moved(sh, run(sh), unique(df$site))) + expect_lt(abs(stats::sd(.fx_moved(df, a)[, "dlat"]) * 111.32 - 10), 0.8) +}) + +test_that("a land test re-draws from the site's own sequence", { + df <- .fx_sites() + # Reject everything west of 145 E, whatever the row order. + land <- function(lon, lat) lon > 145 + a <- suppressWarnings(jitter_coordinates( + df, "lat", "lon", on_land = land, seed = 42L, max_tries = 40L + )) + sh <- df[rev(seq_len(nrow(df))), ] + b <- suppressWarnings(jitter_coordinates( + sh, "lat", "lon", on_land = land, seed = 42L, max_tries = 40L + )) + expect_identical(.fx_moved(df, a), .fx_moved(sh, b, unique(df$site))) +}) + +test_that("two coordinate pairs at one place draw differently", { + df <- data.frame(a_lat = -34.9, a_lon = 138.6, b_lat = -34.9, b_lon = 138.6) + o <- jitter_coordinates(df, "a_lat", "a_lon", on_land = FALSE, seed = 1L) + o <- jitter_coordinates(o, "b_lat", "b_lon", on_land = FALSE, seed = 1L) + expect_false(identical(o$a_lat, o$b_lat)) +}) + +test_that("draws do not depend on the caller's RNG kind or state", { + df <- .fx_sites() + a <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 42L) + withr::local_seed(3L, .rng_kind = "L'Ecuyer-CMRG") + before <- .Random.seed + b <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 42L) + expect_identical(a, b) + expect_identical(before, .Random.seed) +}) + +test_that("a CSV round trip of computed coordinates keeps the draw", { + n <- 50L + set.seed(31L) + df <- data.frame(lat = stats::runif(n, -38, -33), lon = stats::runif(n, 139, 149)) + a <- jitter_coordinates(df, "lat", "lon", on_land = FALSE, seed = 4L) + f <- withr::local_tempfile(fileext = ".csv") + utils::write.csv(df, f, row.names = FALSE) + back <- utils::read.csv(f) + b <- jitter_coordinates(back, "lat", "lon", on_land = FALSE, seed = 4L) + expect_lt(max(abs((a$lat - df$lat) - (b$lat - back$lat))), 2e-5) + expect_lt(max(abs((a$lon - df$lon) - (b$lon - back$lon))), 2e-5) +}) diff --git a/tests/testthat/test-integrity-values.R b/tests/testthat/test-integrity-values.R new file mode 100644 index 0000000..8056a62 --- /dev/null +++ b/tests/testthat/test-integrity-values.R @@ -0,0 +1,133 @@ +# The recipe binds the table's values as well as its NA mask. + +integrity_fixture <- function() { + set.seed(8) + d <- data.frame( + gen = rep(paste0("G", 1:6), 6L), + rep = rep(paste0("R", 1:6), each = 6L), + y = stats::rnorm(36L, 5, 2), + stringsAsFactors = FALSE + ) + r <- suppressWarnings(propose_roles(d, detect = FALSE)) + r <- set_role(r, "rep", role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + spec <- control_spec(on = "gen", strata = "rep", n = 2L, seed = 5L) + m <- mask(d, masque_plan(r, controls = spec), seed = 1L, quiet = TRUE) + list(d = d, m = m, plan = clone_plan(m), rec = recipe(m)) +} + +test_that("a table with other values and the same holes is refused", { + f <- integrity_fixture() + swapped <- f$d + swapped$y <- rev(swapped$y) + expect_error(controls(swapped, f$plan, recipe = f$rec), class = "masque_integrity_refusal") + expect_error(apply_recipe(swapped, f$rec), class = "masque_integrity_refusal") + labels <- f$d + labels$gen[1:2] <- labels$gen[2:1] + expect_error(controls(labels, f$plan, recipe = f$rec), class = "masque_integrity_refusal") + one <- f$d + one$y[20L] <- one$y[20L] + 1e-3 + expect_error(controls(one, f$plan, recipe = f$rec), class = "masque_integrity_refusal") + expect_no_error(apply_recipe(swapped, f$rec, check_integrity = FALSE)) +}) + +test_that("the table itself passes, through every route it can come back by", { + f <- integrity_fixture() + expect_no_error(controls(f$d, f$plan, recipe = f$rec)) + expect_no_error(controls(data.table::as.data.table(f$d), f$plan, recipe = f$rec)) + as_factor <- f$d + as_factor$gen <- factor(as_factor$gen) + expect_no_error(controls(as_factor, f$plan, recipe = f$rec)) + path <- withr::local_tempfile(fileext = ".csv") + utils::write.csv(f$d, path, row.names = FALSE) + expect_no_error(controls(utils::read.csv(path), f$plan, recipe = f$rec)) + data.table::fwrite(f$d, path) + expect_no_error(controls(data.table::fread(path), f$plan, recipe = f$rec)) +}) + +test_that("non-finite values and whole numbers read back as doubles match", { + d <- data.frame(a = c(1L, 2L, NA, 4L), b = c(1.5, Inf, -Inf, 2), s = as.Date("2026-01-01") + 0:3) + fp <- masque:::.content_fingerprint(d) + back <- d + back$a <- as.double(back$a) + back$s <- as.character(back$s) + expect_true(masque:::.content_matches(fp, masque:::.content_fingerprint(back))) + moved <- d + moved$b <- d$b[c(1L, 3L, 2L, 4L)] + expect_false(masque:::.content_matches(fp, masque:::.content_fingerprint(moved))) +}) + +test_that("a column's type on the way back from a file does not change the match", { + d <- data.frame( + empty = rep(NA_real_, 12L), lab = factor(1:12), whole = 3e9 + 1:12, + flag = rep(c(TRUE, FALSE), 6L), txt = as.character(1:12) + ) + fp <- masque:::.content_fingerprint(d) + back <- data.frame( + empty = rep(NA, 12L), lab = 1:12, whole = as.character(3e9 + 1:12), + flag = rep(c(1L, 0L), 6L), txt = 1:12 + ) + expect_true(masque:::.content_matches(fp, masque:::.content_fingerprint(back))) + back$lab[1L] <- 99L + expect_false(masque:::.content_matches(fp, masque:::.content_fingerprint(back))) +}) + +test_that("neither a large value nor a large table hides a change", { + set.seed(9) + n <- 1000L + y <- c(1e10, stats::rnorm(n - 1L)) + fp <- masque:::.content_fingerprint(data.frame(y = y)) + y2 <- c(y[1L], rev(y[-1L])) + expect_false(masque:::.content_matches(fp, masque:::.content_fingerprint(data.frame(y = y2)))) + path <- withr::local_tempfile(fileext = ".csv") + data.table::fwrite(data.frame(y = y), path) + expect_true(masque:::.content_matches(fp, masque:::.content_fingerprint(data.table::fread(path)))) + big <- stats::rnorm(1e5, 5000, 1000) + fp2 <- masque:::.content_fingerprint(data.frame(y = big)) + big[17L] <- big[17L] + 50 + expect_false(masque:::.content_matches(fp2, masque:::.content_fingerprint(data.frame(y = big)))) +}) + +test_that("each sum has its own tolerance", { + entry <- function(value = 100, log_value = 10) { + list(sums = list(y = list( + value = value, scale = 1e6, log_value = log_value, log_scale = 1e3, other = "x" + )), text = "t") + } + expect_true(masque:::.content_matches(entry(), entry(100 + 1e-9, 10 + 1e-12))) + expect_false(masque:::.content_matches(entry(), entry(value = 100 + 1e-7))) + expect_false(masque:::.content_matches(entry(), entry(log_value = 10 + 1e-10))) +}) + +test_that("a recipe without the value fingerprint still applies", { + f <- integrity_fixture() + rec <- f$rec + rec@content_fp <- list() + swapped <- f$d + swapped$y <- rev(swapped$y) + expect_no_error(apply_recipe(swapped, rec)) +}) + +test_that("the fingerprint stays with the owner", { + skip_if_not_installed("yaml") + f <- integrity_fixture() + r <- set_role(roles(f$m), "gen", role = "treatment", action = "alias") + m <- mask(f$d, masque_plan(r, controls = control_spec(on = "gen", strata = "rep", n = 2L, seed = 5L)), + mode = "collaborate", seed = 1L, quiet = TRUE) + f$rec <- recipe(m) + expect_gt(length(f$rec@content_fp$sums), 0L) + out <- file.path(withr::local_tempdir(), "pack") + handover(m, out) + shipped <- unlist(lapply(list.files(out, full.names = TRUE), readLines)) + value <- format(f$rec@content_fp$sums$y$value, digits = 8L) + expect_false(any(grepl(value, shipped, fixed = TRUE))) + expect_false(any(grepl(f$rec@content_fp$text, shipped, fixed = TRUE))) +}) + +test_that("the value fingerprint does not depend on how text is held in memory", { + path <- tempfile(fileext = ".csv") + writeLines(c("site,yield", "a,1", "b,2", "c,3"), path) + read <- data.table::fread(path) + expect_identical(.content_fingerprint(read), .content_fingerprint(data.table::copy(read))) +}) diff --git a/tests/testthat/test-mask-conditional-ladder.R b/tests/testthat/test-mask-conditional-ladder.R index 938a22d..3aad604 100644 --- a/tests/testthat/test-mask-conditional-ladder.R +++ b/tests/testthat/test-mask-conditional-ladder.R @@ -293,7 +293,8 @@ test_that("mask(ladder = 'hierarchy') keeps the environment effect of a MET", { expect_identical(rec@ladder, "hierarchy") expect_identical(rec@conditioning_used, c("trt", "site")) expect_identical(rec@conditioning_dropped, c("row", "rep")) - expect_setequal(rec@conditioning_shifted, c("rep", "row")) + # `row` holds two rows a level here, below the floor for a design shift. + expect_identical(rec@conditioning_shifted, "rep") f_clone <- fit(as.data.frame(synthetic(m))) # Floor 0.4: the clone re-draws each cell's six-to-eight residuals, and # site:rep in the original is half replicate signal and half noise df. diff --git a/tests/testthat/test-plan.R b/tests/testthat/test-plan.R new file mode 100644 index 0000000..ce0abd0 --- /dev/null +++ b/tests/testthat/test-plan.R @@ -0,0 +1,696 @@ +# masque_plan(): mask(df, plan), resolution, translation, both control sides. + +.fx_plan_table <- function() { + withr::with_seed(11L, { + d <- expand.grid( + site = c("Adelong", "Booleroo"), rep = c("R1", "R2", "R3"), + block = paste0("B", 1:4), plot = 1:5, + stringsAsFactors = FALSE + ) + n <- nrow(d) + d$block <- paste(d$site, d$rep, d$block, sep = "_") + d$gen <- NA_character_ + for (k in unique(paste(d$site, d$rep))) { + i <- which(paste(d$site, d$rep) == k) + d$gen[i] <- sample(rep(paste0("Variety", 1:10), 2L)) + } + d$yield <- 10 + stats::rnorm(n) + d$yield[c(5L, 90L)] <- NA + d$plot <- NULL + d + }) +} + +.fx_plan_roles <- function(d) { + r <- propose_roles(d, mode = "local", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + set_role(r, "yield", role = "outcome", action = "scramble") +} + +.fx_spec <- function(...) { + control_spec(on = "gen", target = "yield", effect = list(sd = 0.5), n = 4L, + seed = 77L, ...) +} + +test_that("mask(df, plan) equals mask() with the same arguments", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + p <- masque_plan(r, conditional = "design", grain = FALSE) + a <- suppressWarnings(mask(d, p, seed = 3L, quiet = TRUE)) + b <- suppressWarnings(mask(d, r, seed = 3L, quiet = TRUE, + conditional = "design", grain = FALSE)) + expect_identical(synthetic(a), synthetic(b)) +}) + +test_that("an argument passed to mask() overrides the plan", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + p <- masque_plan(r, conditional = TRUE) + a <- suppressWarnings(mask(d, p, seed = 3L, quiet = TRUE, conditional = FALSE)) + b <- suppressWarnings(mask(d, r, seed = 3L, quiet = TRUE)) + expect_identical(synthetic(a), synthetic(b)) + expect_false(recipe(a)@plan@conditional) +}) + +test_that("the clone is identical with and without a control spec", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + a <- mask(d, masque_plan(r), seed = 5L, quiet = TRUE) + b <- mask(d, masque_plan(r, controls = .fx_spec()), seed = 5L, quiet = TRUE) + expect_identical(synthetic(a), synthetic(b)) +}) + +test_that("mask() resolves unit, strata, effect on the source and writes back", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + s <- recipe(m)@plan@controls$default + # Every site and rep holds every variety; a block does not. + expect_setequal(s@strata, c("site", "rep")) + expect_length(s@unit, 0L) + expect_true(is.numeric(s@effect)) + expect_setequal(names(s@effect), paste0("Variety", 1:10)) + again <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + expect_identical(recipe(again)@plan@controls$default@effect, s@effect) +}) + +test_that("a whole-plot unit and declared strata resolve", { + d <- .fx_plan_table() + d$irrig <- ifelse(as.integer(factor(d$block)) %% 2L == 0L, "wet", "dry") + r <- .fx_plan_roles(d) + r <- set_role(r, "gen", role = "covariate", action = "scramble") + r <- set_role(r, "irrig", role = "treatment", action = "keep") + p <- masque_plan(r, controls = control_spec(on = "irrig", n = 2L, seed = 1L)) + m <- suppressWarnings(mask(d, p, seed = 5L, quiet = TRUE)) + s <- recipe(m)@plan@controls$default + expect_identical(s@unit, "block") + expect_setequal(s@strata, c("site", "rep")) + + p2 <- masque_plan(r, controls = control_spec( + on = "irrig", strata = "site", unit = "block", n = 2L, seed = 1L + )) + m2 <- suppressWarnings(mask(d, p2, seed = 5L, quiet = TRUE)) + expect_identical(recipe(m2)@plan@controls$default@strata, "site") +}) + +test_that("a spec the source cannot honour is refused at mask()", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + bad <- masque_plan(r, controls = control_spec(on = "yield", n = 2L, seed = 1L)) + expect_error(mask(d, bad, seed = 1L, quiet = TRUE), class = "orchestra_refusal") + bad2 <- masque_plan(r, controls = control_spec( + on = "gen", strata = "nope", n = 2L, seed = 1L + )) + expect_error(mask(d, bad2, seed = 1L, quiet = TRUE), class = "orchestra_refusal") +}) + +test_that("the clone plan is in clone vocabulary, with nothing of the source", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + m <- suppressWarnings(mask( + d, masque_plan(r, controls = .fx_spec(), alias_names = TRUE), + seed = 5L, quiet = TRUE + )) + cp <- clone_plan(m) + expect_identical(cp@vocabulary, "clone") + s <- cp@controls$default + expect_true(all(c(s@on, s@target, s@strata) %in% names(synthetic(m)))) + expect_false(any(c("gen", "yield", "site", "rep") %in% c(s@on, s@target, s@strata))) + clone_on <- as.character(synthetic(m)[[s@on]]) + expect_setequal(names(s@effect), unique(clone_on)) + expect_false(any(grepl("Variety", names(s@effect)))) + expect_length(cp@coords, 0L) + # Falsifier: the source plan does hold the source vocabulary. + expect_true(any(grepl("Variety", names(recipe(m)@plan@controls$default@effect)))) +}) + +test_that("the effect means the same variety on both sides", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + src <- recipe(m)@plan@controls$default@effect + cln <- clone_plan(m)@controls$default@effect + map <- recipe(m)@level_maps$gen + expect_equal(unname(cln[unname(map[names(src)])]), unname(src)) +}) + +test_that("the analyst's controls run on the clone from the clone plan", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + ctl <- controls(synthetic(m), clone_plan(m)) + neg <- control_table(ctl, "negative", 2L) + s <- synthetic(m) + expect_identical(neg$site, s$site) + expect_identical(is.na(neg$yield), is.na(s$yield)) + tab <- function(x) table(x$site, x$rep, x$gen) + expect_identical(tab(neg), tab(s)) + expect_false(identical(neg$gen, s$gen)) + pos <- control_table(ctl, "positive", 2L) + shift <- pos$yield - neg$yield + eff <- ctl@spec@effect + expect_equal( + tapply(shift, pos$gen, mean, na.rm = TRUE)[names(eff)], + eff * stats::sd(s$yield, na.rm = TRUE), + ignore_attr = TRUE + ) +}) + +test_that("the owner's controls take a private seed and the real table", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + cp <- clone_plan(m) + rec <- recipe(m) + own <- controls(d, cp, recipe = rec) + ana <- controls(synthetic(m), cp) + # The seeds differ, and neither is the other's permutation. + expect_false(identical(own@seeds, ana@seeds)) + expect_false(identical(own@seeds, as.integer(.sub_seeds(77L, 5L)))) + # The owner's table is the real one, in real labels. + n1 <- control_table(own, "negative", 1L) + expect_setequal(unique(n1$gen), paste0("Variety", 1:10)) + expect_identical(sort(n1$yield), sort(d$yield)) + expect_true(any(grepl("private", own@notes))) + # Source plan and clone plan give the same owner's object. + own2 <- controls(d, rec@plan, recipe = rec) + expect_identical(control_table(own2, "negative", 1L), control_table(own, "negative", 1L)) + expect_identical(own2@spec@effect, own@spec@effect) +}) + +test_that("the source side refuses what it should", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + cp <- clone_plan(m) + rec <- recipe(m) + + # A source plan without a recipe would permute real rows under a public seed. + expect_error(controls(d, rec@plan), class = "masque_control_source_plan_refusal") + # The spec's seed is the analyst's. + expect_error(controls(d, cp, recipe = rec, seed = 77L), + class = "masque_control_seed_refusal") + # A different NA mask is a different table. + other <- d + other$yield[1L] <- NA + expect_error(controls(other, cp, recipe = rec), class = "masque_integrity_refusal") + # A missing column. + gone <- d + gone$rep <- NULL + expect_error(controls(gone, cp, recipe = rec), class = "masque_control_mismatch_refusal") + # A spec changed after the clone was made. + changed <- cp + s <- changed@controls$default + s@n <- 9L + changed@controls$default <- s + expect_error(controls(d, changed, recipe = rec), + class = "masque_control_spec_changed_refusal") + # A recipe with a plain roles table. + expect_error( + controls(d, .fx_plan_roles(d), .fx_spec(), recipe = rec), + class = "masque_control_recipe_refusal" + ) + # A plan-free recipe has no key: the owner must bring a private seed. + m0 <- suppressWarnings(mask(d, .fx_plan_roles(d), seed = 1L, quiet = TRUE)) + plan <- recipe(m)@plan + expect_error( + controls(d, plan, recipe = recipe(m0)), + class = "masque_control_seed_refusal" + ) + own <- controls(d, plan, recipe = recipe(m0), seed = 123L) + expect_true(S7::S7_inherits(own, masque_controls)) +}) + +test_that("cleaning is replayed on the owner's table", { + d <- .fx_plan_table() + names(d)[names(d) == "yield"] <- "yield t/ha" + expect_warning( + cl <- clean_table(d, quiet = TRUE)$data, + class = "masque_name_repaired" + ) + yield_col <- setdiff(names(cl), c("site", "rep", "block", "gen")) + r <- propose_roles(cl, mode = "local", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, yield_col, role = "outcome", action = "scramble") + spec <- control_spec(on = "gen", target = yield_col, effect = list(sd = 0.5), + n = 2L, seed = 77L) + m <- suppressWarnings(mask(d, masque_plan(r, controls = spec), seed = 5L, quiet = TRUE)) + own <- suppressWarnings(controls(d, clone_plan(m), recipe = recipe(m))) + expect_true(yield_col %in% names(control_table(own, "negative", 1L))) +}) + +test_that("several named specs are picked by name", { + d <- .fx_plan_table() + specs <- list( + a = .fx_spec(), + b = control_spec(on = "gen", n = 2L, seed = 8L) + ) + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = specs), seed = 5L, quiet = TRUE) + cp <- clone_plan(m) + expect_error(controls(synthetic(m), cp), class = "masque_control_spec_refusal") + expect_error(controls(synthetic(m), cp, "c"), class = "masque_control_spec_refusal") + expect_identical(controls(synthetic(m), cp, "b")@spec@n, 2L) +}) + +test_that("a plan refuses bad input and the wrong verb", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + expect_error(masque_plan(1), class = "masque_plan_roles_refusal") + expect_error(masque_plan(r[, c("col", "role")]), class = "masque_plan_roles_refusal") + expect_error(masque_plan(r, conditional = 2), class = "masque_bad_conditional_refusal") + expect_error(masque_plan(r, grain = NA), class = "masque_plan_grain_refusal") + expect_error(masque_plan(r, controls = list(1)), class = "masque_plan_controls_refusal") + expect_error(masque(d, roles = masque_plan(r)), class = "masque_plan_unsupported_refusal") + m <- mask(d, masque_plan(r), seed = 1L, quiet = TRUE) + expect_error(mask(d, clone_plan(m)), class = "masque_plan_vocabulary_refusal") + expect_error(clone_plan(1), class = "masque_bad_masque_obj_refusal") +}) + +test_that("a plan with no spec still gives a clone plan and refuses controls", { + d <- .fx_plan_table() + m <- suppressWarnings(mask(d, .fx_plan_roles(d), seed = 1L, quiet = TRUE)) + cp <- clone_plan(m) + expect_length(cp@controls, 0L) + expect_error(controls(synthetic(m), cp), class = "masque_control_spec_refusal") +}) + +test_that("printing a plan shows no seed and no level", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = 5L, quiet = TRUE) + out <- paste(cli::cli_fmt(print(clone_plan(m))), collapse = "\n") + expect_match(out, "clone vocabulary") + expect_match(out, "default") + expect_false(grepl("Variety|77", out)) +}) + +test_that("a plan round-trips through YAML, with awkward labels", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + awkward <- c("NA", "yes", "1", "0.5", "a: b", "true", "~", "null", "#x", "é") + labs <- stats::setNames(c(awkward), paste0("Variety", 1:10)) + d$gen <- unname(labs[d$gen]) + r <- .fx_plan_roles(d) + r <- set_role(r, "gen", role = "treatment", action = "keep") + m <- mask(d, masque_plan(r, conditional = "design", controls = .fx_spec()), + seed = 5L, quiet = TRUE) + src <- recipe(m)@plan + f <- withr::local_tempfile(fileext = ".yaml") + save_plan(src, f) + back <- read_plan(f) + expect_identical(back@controls$default@effect, src@controls$default@effect) + expect_identical(back@controls$default@strata, src@controls$default@strata) + expect_identical(back@conditional, src@conditional) + expect_identical(back@roles$col, src@roles$col) + expect_identical(back@roles$action, src@roles$action) + expect_identical(back@vocabulary, "source") + expect_identical(back@mode, "local") + a <- suppressWarnings(mask(d, back, seed = 5L, quiet = TRUE)) + expect_identical(synthetic(a), synthetic(m)) + + # The owner's spec check holds after a round trip of the clone plan. + g <- withr::local_tempfile(fileext = ".yaml") + save_plan(clone_plan(m), g) + own <- controls(d, read_plan(g), recipe = recipe(m)) + expect_true(S7::S7_inherits(own, masque_controls)) +}) + +test_that("reading and writing refuse what they cannot do", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + f <- withr::local_tempfile(fileext = ".yaml") + expect_error(save_plan(r, f), class = "masque_plan_class_refusal") + expect_error(save_plan(masque_plan(r), "a.csv"), class = "masque_plan_file_refusal") + expect_error(read_plan("nope.yaml"), class = "masque_plan_file_refusal") + writeLines("a: 1", f) + expect_error(read_plan(f), class = "masque_plan_file_refusal") + writeLines("a: [", f) + expect_error(read_plan(f), class = "masque_plan_file_refusal") + p <- masque_plan(r, coords = list(lat = "a", lon = "b", on_land = function(lon, lat) TRUE)) + expect_error(save_plan(p, f), class = "masque_plan_file_refusal") +}) + +test_that("handover writes four files and no source vocabulary", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + r <- set_role(.fx_plan_roles(d), c("site", "block"), action = "alias") + m <- suppressWarnings(mask( + d, masque_plan(r, controls = .fx_spec(), alias_names = TRUE), + seed = 5L, quiet = TRUE + )) + dir <- withr::local_tempdir() + out <- file.path(dir, "pack") + paths <- handover(m, out, allow_local = TRUE) + expect_setequal(basename(paths), c("clone.csv", "plan.yaml", "card.yaml", "README.txt")) + expect_setequal(list.files(out), basename(paths)) + text <- unlist(lapply(paths, readLines)) + for (w in c("Variety", "Adelong", "Booleroo", "yield")) { + expect_false(any(grepl(w, text, fixed = TRUE)), info = w) + } + # Falsifier: the same scan finds the plan's words in the source plan's file. + own_file <- withr::local_tempfile(fileext = ".yaml") + save_plan(recipe(m)@plan, own_file) + own_text <- readLines(own_file) + for (w in c("Variety", "yield")) { + expect_true(any(grepl(w, own_text, fixed = TRUE)), info = w) + } + expect_error(handover(m, out, allow_local = TRUE), class = "masque_handover_exists_refusal") + handover(m, out, overwrite = TRUE, allow_local = TRUE) + + # The file round trip the note describes. + clone <- data.table::fread(file.path(out, "clone.csv")) + ctl <- controls(clone, read_plan(file.path(out, "plan.yaml"))) + expect_true(S7::S7_inherits(ctl, masque_controls)) +}) + +test_that("handover runs the release gate", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + d$id_copy <- seq_len(nrow(d)) + r <- .fx_plan_roles(d[, setdiff(names(d), "id_copy")]) + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, c("yield", "id_copy"), role = "outcome", action = "keep") + m <- suppressWarnings(mask(d, r, mode = "collaborate", seed = 1L, quiet = TRUE, + allow_kept_outcome = TRUE)) + out <- file.path(withr::local_tempdir(), "pack") + expect_error(handover(m, out), class = "masque_blocked_write") + expect_false(dir.exists(out)) + expect_warning(handover(m, out, allow_high = TRUE), class = "masque_high_override") + note <- readLines(file.path(out, "README.txt")) + expect_true(any(grepl("HIGH", note) & grepl(.audit_high_cols(m)[1L], note, fixed = TRUE))) + expect_error(handover(m, tempfile()), "named in README.txt") +}) + +test_that("the override note uses the clone's column names", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "keep") + r <- set_role(r, "yield", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, r, mode = "collaborate", seed = 1L, quiet = TRUE, alias_names = TRUE)) + expect_true("gen" %in% .audit_high_cols(m)) + out <- file.path(withr::local_tempdir(), "pack") + suppressWarnings(handover(m, out, allow_high = TRUE)) + note <- readLines(file.path(out, "README.txt")) + line <- note[grepl("HIGH", note)] + expect_length(line, 1L) + expect_false(grepl("\\bgen\\b", line)) + expect_true(grepl(recipe(m)@column_name_map[["gen"]], line, fixed = TRUE)) +}) + +# Faults found in review before the 0.17.0 tag -------------------------------- + +.fx_mapped <- function(action, seed = 12345L, spec_seed = 99L) { + d <- .fx_plan_table() + r <- set_role(.fx_plan_roles(d), "gen", role = "treatment", action = action) + spec <- control_spec(on = "gen", target = "yield", effect = list(sd = 0.3), + n = 3L, seed = spec_seed) + suppressWarnings(mask(d, masque_plan(r, controls = spec), seed = seed, quiet = TRUE)) +} + +test_that("the clone plan's effect does not reveal the level map", { + for (act in c("alias", "scramble")) { + leaks <- vapply(1:12, function(s) { + m <- .fx_mapped(act, seed = 1000L + s) + e <- clone_plan(m)@controls$default@effect + map <- recipe(m)@level_maps$gen + # Names in clone order; nothing tied to source order remains. + expect_identical(names(e), sort(names(e), method = "radix")) + # The attack: order of the vector read as the source-sorted map. + identical(unname(map[sort(names(map))]), names(e)) + }, logical(1L)) + expect_false(any(leaks), info = act) + + # The attack: recompute the draw from the public seed. It reaches clone + # labels only, and the same clone label gets the same draw in every build. + m <- .fx_mapped(act, seed = 7L) + e <- clone_plan(m)@controls$default@effect + root <- (99 + .CONTROL_SALT) %% .Machine$integer.max + draw <- withr::with_seed(root, stats::rnorm(length(e), 0, 0.3)) + expect_equal(unname(e), draw) + } +}) + +test_that("two maps give the same clone-side effect vector values by label", { + a <- clone_plan(.fx_mapped("alias", seed = 1L))@controls$default@effect + b <- clone_plan(.fx_mapped("alias", seed = 2L))@controls$default@effect + expect_identical(a, b) +}) + +test_that("the same variety still takes the same shift on both sides", { + m <- .fx_mapped("alias") + src <- recipe(m)@plan@controls$default@effect + cln <- clone_plan(m)@controls$default@effect + map <- recipe(m)@level_maps$gen + expect_equal(unname(cln[unname(map[names(src)])]), unname(src)) +}) + +test_that("a dropped on or target is refused and reaches no file", { + d <- .fx_plan_table() + d$protein <- withr::with_seed(2L, stats::rnorm(nrow(d))) + r <- .fx_plan_roles(d) + r <- set_role(r, "protein", role = "outcome", action = "scramble") + r1 <- set_role(r, "gen", role = "treatment", action = "drop") + expect_error( + mask(d, masque_plan(r1, controls = control_spec(on = "gen", n = 2L, seed = 1L)), + seed = 1L, quiet = TRUE), + class = "masque_control_dropped_refusal" + ) + r2 <- set_role(r, "protein", role = "outcome", action = "drop") + expect_error( + mask(d, masque_plan(r2, controls = control_spec( + on = "gen", target = "protein", effect = list(sd = 0.2), n = 2L, seed = 1L + )), seed = 1L, quiet = TRUE), + class = "masque_control_dropped_refusal" + ) +}) + +test_that("clone_plan fails closed on a name outside the clone", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), seed = 5L, quiet = TRUE) + rec <- m@recipe + s <- rec@plan@controls$default + s@strata <- "a_real_name" + plan <- rec@plan + plan@controls$default <- s + rec@plan <- plan + m@recipe <- rec + expect_error(clone_plan(m), class = "masque_plan_vocabulary_refusal") +}) + +test_that("a spec seed equal to the mask seed is refused", { + d <- .fx_plan_table() + p <- masque_plan(.fx_plan_roles(d), controls = .fx_spec()) + expect_error(mask(d, p, seed = 77L, quiet = TRUE), class = "masque_control_seed_refusal") +}) + +test_that("mask_set() refuses a plan rather than ignoring it", { + d <- .fx_plan_table() + p <- masque_plan(.fx_plan_roles(d), alias_names = TRUE) + expect_error( + mask_set(list(a = d), roles = list(a = p), seed = 1L, quiet = TRUE), + class = "masque_plan_unsupported_refusal" + ) +}) + +test_that("handover refuses a local clone unless asked", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + m <- suppressWarnings(mask(d, .fx_plan_roles(d), seed = 1L, quiet = TRUE)) + out <- file.path(withr::local_tempdir(), "pack") + expect_error(handover(m, out), class = "masque_handover_local_refusal") + expect_false(dir.exists(out)) + handover(m, out, allow_local = TRUE) + expect_true(file.exists(file.path(out, "clone.csv"))) + expect_true(any(grepl("owner", readLines(file.path(out, "README.txt"))))) +}) + +test_that("a scrambled whole-valued on is refused: its levels are real values", { + d <- .fx_plan_table() + d$age <- rep(c(31L, 47L, 53L, 68L, 72L), length.out = nrow(d)) + r <- set_role(.fx_plan_roles(d), "gen", role = "treatment", action = "keep") + r <- set_role(r, "age", role = "covariate", action = "scramble") + expect_error( + mask(d, masque_plan(r, controls = control_spec(on = "age", n = 2L, seed = 1L)), + seed = 1L, quiet = TRUE), + class = "masque_control_on_refusal" + ) +}) + +test_that("a numeric or integer treatment with an alias runs on both sides", { + d <- .fx_plan_table() + for (vals in list(as.numeric(as.integer(factor(d$gen))) / 2, as.integer(factor(d$gen)))) { + d$dose <- vals + x <- d[, setdiff(names(d), "gen")] + r <- propose_roles(x, mode = "local", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "yield", role = "outcome", action = "scramble") + r <- set_role(r, "dose", role = "treatment", action = "alias") + spec <- control_spec(on = "dose", target = "yield", effect = list(sd = 0.3), + n = 2L, seed = 9L) + m <- suppressWarnings(mask(x, masque_plan(r, controls = spec), seed = 2L, quiet = TRUE)) + ana <- controls(synthetic(m), clone_plan(m)) + own <- controls(x, clone_plan(m), recipe = recipe(m)) + expect_true(S7::S7_inherits(ana, masque_controls)) + expect_true(S7::S7_inherits(own, masque_controls)) + map <- recipe(m)@level_maps$dose + expect_equal( + unname(ana@spec@effect[unname(map[names(own@spec@effect)])]), + unname(own@spec@effect) + ) + } +}) + +test_that("blank cells in a clone CSV stay missing under a clone plan", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + d$gen[c(2L, 50L, 90L)] <- NA + m <- suppressWarnings(mask(d, masque_plan(.fx_plan_roles(d), controls = control_spec( + on = "gen", target = "yield", effect = list(sd = 0.3), strata = "site", n = 3L, seed = 9L + )), seed = 2L, quiet = TRUE)) + out <- file.path(withr::local_tempdir(), "pack") + handover(m, out, allow_local = TRUE) + clone <- data.table::fread(file.path(out, "clone.csv")) + expect_identical(sum(clone$gen == "", na.rm = TRUE), 3L) + ctl <- suppressWarnings(controls(clone, read_plan(file.path(out, "plan.yaml")))) + neg <- control_table(ctl, "negative", 1L) + expect_identical(which(is.na(neg$gen)), c(2L, 50L, 90L)) + expect_false("" %in% ctl@truth$level) +}) + +test_that("the owner's manifest holds no private seed", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), seed = 5L, quiet = TRUE) + own <- controls(d, clone_plan(m), recipe = recipe(m)) + expect_true(all(is.na(own@manifest$seed))) + ana <- controls(synthetic(m), clone_plan(m)) + expect_false(anyNA(ana@manifest$seed)) + r <- .fx_plan_roles(d) + priv <- controls(d, r, .fx_spec(), seed = 123L) + expect_true(all(is.na(priv@manifest$seed))) +}) + +test_that("an owner's table with row names from a filter is not refused", { + d <- .fx_plan_table() + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), seed = 5L, quiet = TRUE) + filt <- d + rownames(filt) <- paste0("row", seq_len(nrow(filt))) + expect_true(S7::S7_inherits( + controls(filt, clone_plan(m), recipe = recipe(m)), masque_controls + )) +}) + +test_that("a partly doubled whole plot still resolves as the unit", { + withr::local_seed(3L) + sizes <- rep(c(1L, 1L, 2L), length.out = 60L) + d <- data.frame(wp = rep(sprintf("P%02d", seq_along(sizes)), sizes)) + d$site <- rep(c("A", "B"), length.out = nrow(d)) + d$site <- rep(rep(c("A", "B"), length.out = 60L), sizes) + d$irr <- rep(sample(rep(c("wet", "dry"), 30L)), sizes) + d$y <- stats::rnorm(nrow(d)) + r <- propose_roles(d, mode = "local", detect = FALSE) + r <- set_role(r, c("wp", "site"), role = "design", action = "keep") + r <- set_role(r, "irr", role = "treatment", action = "keep") + r <- set_role(r, "y", role = "outcome", action = "scramble") + m <- suppressWarnings(mask(d, masque_plan(r, controls = control_spec(on = "irr", n = 2L, seed = 1L)), + seed = 5L, quiet = TRUE)) + expect_identical(recipe(m)@plan@controls$default@unit, "wp") +}) + +test_that("a plan's bad coordinates or alias_names are refused at the plan", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + expect_error(masque_plan(r, coords = c("a", "b")), class = "masque_plan_coords_refusal") + expect_error(masque_plan(r, alias_names = NA), class = "masque_plan_alias_names_refusal") +}) + +test_that("the note names a kept outcome instead of calling it re-simulated", { + skip_if_not_installed("yaml") + d <- .fx_plan_table() + r <- propose_roles(d, mode = "collaborate", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "gen", role = "treatment", action = "alias") + r <- set_role(r, "yield", role = "outcome", action = "keep") + m <- suppressWarnings(mask(d, r, mode = "collaborate", seed = 1L, quiet = TRUE, + allow_kept_outcome = TRUE)) + out <- file.path(withr::local_tempdir(), "pack") + suppressWarnings(handover(m, out, allow_high = TRUE)) + note <- readLines(file.path(out, "README.txt")) + expect_true(any(grepl("real values", note))) +}) + +test_that("no value the analyst holds rebuilds the owner's seeds", { + d <- .fx_plan_table() + for (s in c(0L, 1L, 42L)) { + m <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), + seed = s, quiet = TRUE) + own <- controls(d, clone_plan(m), recipe = recipe(m)) + expect_true(all(is.na(own@seeds))) + expect_match(own@key, "^[0-9a-f]{64}$") + } + # One seed, two recipes: the owner's key differs, the clone does not. + a <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), seed = 1L, quiet = TRUE) + b <- mask(d, masque_plan(.fx_plan_roles(d), controls = .fx_spec()), seed = 1L, quiet = TRUE) + expect_identical(synthetic(a), synthetic(b)) + expect_false(identical(recipe(a)@owner_key, recipe(b)@owner_key)) + # The same recipe regenerates the owner's controls. + o1 <- controls(d, clone_plan(a), recipe = recipe(a)) + o2 <- controls(d, clone_plan(a), recipe = recipe(a)) + expect_identical(control_table(o1, "negative", 1L), control_table(o2, "negative", 1L)) + # The key shows in no display route. + key <- recipe(a)@owner_key + shown <- c(utils::capture.output(print(recipe(a))), utils::capture.output(utils::str(recipe(a)))) + expect_false(any(grepl(key, shown, fixed = TRUE))) +}) + +test_that("a spec name holding a hidden name or label is refused", { + d <- .fx_plan_table() + r <- .fx_plan_roles(d) + bad <- list(Variety1_vs_yield = .fx_spec()) + expect_error( + mask(d, masque_plan(r, controls = bad, alias_names = TRUE), seed = 1L, quiet = TRUE), + class = "masque_plan_controls_refusal" + ) + ok <- list(main_test = .fx_spec()) + m <- mask(d, masque_plan(r, controls = ok, alias_names = TRUE), seed = 1L, quiet = TRUE) + expect_named(clone_plan(m)@controls, "main_test") +}) + +test_that("a non-terminating numeric alias works on both sides", { + d <- .fx_plan_table() + d$dose <- rep(c(1 / 3, 2 / 3, 1), length.out = nrow(d)) + x <- d[, setdiff(names(d), "gen")] + r <- propose_roles(x, mode = "local", detect = FALSE) + r <- set_role(r, c("site", "rep", "block"), role = "design", action = "keep") + r <- set_role(r, "yield", role = "outcome", action = "scramble") + r <- set_role(r, "dose", role = "treatment", action = "alias") + spec <- control_spec(on = "dose", target = "yield", effect = list(sd = 0.5), + n = 2L, seed = 9L) + m <- suppressWarnings(mask(x, masque_plan(r, controls = spec), seed = 2L, quiet = TRUE)) + own <- controls(x, clone_plan(m), recipe = recipe(m)) + neg <- control_table(own, "negative", 1L) + pos <- control_table(own, "positive", 1L) + d_y <- pos$yield - neg$yield + shift <- own@truth$shift_realised[match(.level_key(pos$dose), own@truth$level)] + expect_equal(d_y[!is.na(d_y)], shift[!is.na(d_y)]) + expect_true(all(abs(shift) > 0)) +}) + +test_that("a blank on means missing unless the plan names a blank level", { + d <- .fx_plan_table() + d$gen[3L] <- "" + r <- .fx_plan_roles(d) + r <- set_role(r, "gen", role = "treatment", action = "keep") + spec <- control_spec(on = "gen", target = "yield", effect = list(sd = 0.2), strata = "site", n = 2L, seed = 9L) + m <- suppressWarnings(mask(d, masque_plan(r, controls = spec), seed = 2L, quiet = TRUE)) + ctl <- suppressWarnings(controls(synthetic(m), clone_plan(m))) + expect_true("" %in% ctl@truth$level) + expect_false(anyNA(control_table(ctl, "negative", 1L)$gen)) +}) diff --git a/tests/testthat/test-recipe-io.R b/tests/testthat/test-recipe-io.R index 151198e..49a37ff 100644 --- a/tests/testthat/test-recipe-io.R +++ b/tests/testthat/test-recipe-io.R @@ -27,27 +27,30 @@ test_that("save_recipe + read_recipe round-trip is identity", { expect_identical(rec@integrity_fp, rec2@integrity_fp) }) -test_that("save_recipe file is much smaller than the raw data (1000x20 fixture)", { - set.seed(0) - df <- data.frame( - Rep = rep(1:4, 250), - block = rep(1:5, each = 200), - Genotype = factor(rep(LETTERS[1:10], 100)), - yield = rnorm(1000) - ) - for (i in 1:16) df[[paste0("cov_", i)]] <- rnorm(1000) - r <- propose_roles(df) - r$role[r$col == "yield"] <- "outcome" - m <- mask(df, r, mode = "collaborate", seed = 1) - - tmp <- tempfile(fileext = ".rds") - save_recipe(recipe(m), tmp) - sz <- file.info(tmp)$size - # Compared with the data under the same serialiser: absolute sizes differ - # across R versions (R-devel is several times larger). - data_rds <- tempfile(fileext = ".rds") - saveRDS(df, data_rds) - expect_lt(sz, file.info(data_rds)$size) +test_that("save_recipe file does not grow with the rows of the data", { + sizes <- function(n) { + set.seed(0) + df <- data.frame( + Rep = rep(1:4, length.out = n), + block = rep(1:5, length.out = n), + Genotype = factor(rep(LETTERS[1:10], length.out = n)), + yield = rnorm(n) + ) + for (i in 1:16) df[[paste0("cov_", i)]] <- rnorm(n) + r <- propose_roles(df) + r$role[r$col == "yield"] <- "outcome" + m <- mask(df, r, mode = "collaborate", seed = 1) + recipe_rds <- tempfile(fileext = ".rds") + save_recipe(recipe(m), recipe_rds) + data_rds <- tempfile(fileext = ".rds") + saveRDS(df, data_rds) + file.info(c(recipe_rds, data_rds))$size + } + small <- sizes(1000L) + large <- sizes(10000L) + # The class definition sets the file's base size, which varies with the R + # version. A recipe holding rows would grow with the data. + expect_lt(large[1L] - small[1L], 0.01 * (large[2L] - small[2L])) }) test_that("save_recipe errors on invalid inputs", { diff --git a/tests/testthat/test-safety-gate.R b/tests/testthat/test-safety-gate.R index f10b5f8..6b87881 100644 --- a/tests/testthat/test-safety-gate.R +++ b/tests/testthat/test-safety-gate.R @@ -226,3 +226,15 @@ test_that("a joint HIGH finding blocks write_set() and names the check", { ) expect_false(dir.exists(dir)) }) + +test_that("write_set's refusal says the override is not on the recipe", { + set.seed(6) + a <- data.frame(id = 1:20, y = stats::rnorm(20L)) + r <- suppressWarnings(propose_roles(a, detect = FALSE)) + r <- set_role(r, "y", role = "outcome", action = "keep") + r <- set_role(r, "id", role = "id", action = "keep") + m <- suppressWarnings(mask_set(list(t1 = a), list(t1 = r), mode = "collaborate", seed = 1L, + allow_kept_outcome = TRUE, quiet = TRUE)) + skip_if_not(length(.audit_high_cols(m)) > 0L) + expect_error(write_set(m, file.path(tempdir(), "no-write")), "keep your own record") +}) diff --git a/tests/testthat/test-synthesise_numeric_jitter.R b/tests/testthat/test-synthesise_numeric_jitter.R index cc7494e..3a742d7 100644 --- a/tests/testthat/test-synthesise_numeric_jitter.R +++ b/tests/testthat/test-synthesise_numeric_jitter.R @@ -39,6 +39,14 @@ test_that("synthesise_numeric_collaborate stays within half a step of range", { expect_true(all(out >= min(x_obs) - half) && all(out <= max(x_obs) + half)) }) +test_that("a whole-numbered column can reach one full step past the range", { + set.seed(1) + x_obs <- rep(0:20, 10L) + out <- masque:::synthesise_numeric_collaborate(x_obs, rep(0, length(x_obs))) + expect_identical(min(out), -1L) + expect_true(all(out >= -1L)) +}) + test_that("synthesise_numeric_collaborate preserves integer class", { set.seed(1) x_obs <- sample(1L:20L, 200, replace = TRUE) @@ -75,8 +83,10 @@ test_that("Collaborate mode lowers numeric exact-match in mask()", { r <- propose_roles(df) r$role[r$col == "yield"] <- "outcome" - # Rep stays a kept design column (graded low), so no HIGH warning. - m <- mask(df, r, mode = "collaborate", seed = 1) + # Rep stays a kept design column (graded low), so no HIGH warning. The seed + # differs from the one that made the data: an equal seed draws the same + # normals, and the clone then tracks the table. + m <- mask(df, r, mode = "collaborate", seed = 2) audit <- audit_mask(m, print = FALSE) # Continuous outcome: exact_match_pct < 1 diff --git a/vignettes/confidentiality.Rmd b/vignettes/confidentiality.Rmd index 6410289..e5c8275 100644 --- a/vignettes/confidentiality.Rmd +++ b/vignettes/confidentiality.Rmd @@ -72,7 +72,7 @@ per-column work: | Date / time columns | row-permuted with class preserved | row-permuted with class preserved | | Identifiers (`id`) | kept | dropped | | Free text (`text`: a character column in which more than half the values are distinct) | kept | dropped | -| Numeric synthesis | Gaussian copula with empirical-quantile marginals (may emit observed values) | the same, plus within-resolution jitter, with whole numbers stochastically rounded. A value can fall up to half a measurement step outside the observed range | +| Numeric synthesis | Gaussian copula with empirical-quantile marginals (may emit observed values) | the same, plus within-resolution jitter, with whole numbers stochastically rounded. A value can fall up to half a measurement step outside the observed range, and a whole number up to one full step, so a column whose minimum is 0 can show -1. A conditional clone can go further, because it adds level effects after the draw | | Columns constant within a design unit | one value per unit, drawn from the real unit values | one value per unit, jittered, or row by row when there are fewer than five units | | NA mask | preserved cell-by-cell | preserved cell-by-cell | | `audit_mask()` | on demand | automatic at `mask()` time | @@ -753,7 +753,7 @@ cat("maps is not installed on this build, so the site-grouping demo is", Grouping is a confidentiality control, not only a fidelity one. Donut displacement is isotropic, so the mean of many independent draws around one true site converges on that site. Displacing a 360-row trial row by -row leaves its centroid a median 0.58 km from the truth, against the 5 km +row leaves its centroid a median 0.64 km from the truth, against the 5 km floor the donut was asked for. One draw per site removes that estimator and leaves the full displacement in place. The unit has to be chosen carefully: it is the finest grouping denoting one physical place at one diff --git a/vignettes/controls.Rmd b/vignettes/controls.Rmd new file mode 100644 index 0000000..93c1f03 --- /dev/null +++ b/vignettes/controls.Rmd @@ -0,0 +1,340 @@ +--- +title: "Known-truth controls" +output: rmarkdown::html_vignette +vignette: > + %\VignetteIndexEntry{Known-truth controls} + %\VignetteEngine{knitr::rmarkdown} + %\VignetteEncoding{UTF-8} +--- + + + +```{r setup, include = FALSE} +knitr::opts_chunk$set( + collapse = TRUE, + comment = "#>" +) +old_options <- options(width = 100) +set.seed(1) +``` + +```{r library} +library(masque) +``` + +## Why + +`masque` gives an analyst the structure of a confidential table without its +values. Controls are tables derived from a table in which part of the answer is +fixed by construction, so a pipeline can be run where the right reading is +known. + +* The **null** control permutes every analysed column, so no analysed column + is associated with another or with the treatment. It is a smoke test. A + pipeline that finds strong signal on it has built that signal itself, from the + outcome or from a column the shuffle left alone. A leak that sits in the + table, such as a covariate that copies the outcome, is shuffled away with + everything else, so a quiet null does not clear a pipeline. +* The **negative** control permutes one declared column, by default the + treatment, within the design strata. Its allocation is random within them + and everything else is as it was, the outcome included. It is a sham + assignment: the permuted column is unrelated to the outcome, and any real + effect it had stays in the outcome as extra noise. Over many replicates the + share of negatives on which a pipeline rejects is its false-positive rate on + this table under this allocation. On a randomised experiment, the rank of the + real result among them is a randomisation p-value. +* The **positive** control adds a declared shift per level to the negative. + The share of positives on which a pipeline finds it is its sensitivity to a + clean shift of that size on this table. + +In epidemiology a negative control is a variable chosen to share the bias of +the question. These controls are null by permutation and share no bias, so none +of them can show confounding. + +Nothing is re-simulated. Each control is a permutation of the table it was +built from, with a shift added for the positive. The design columns and the +missing cells are those of the input, and every column keeps its set of values, +apart from the shifted target of a positive. + +## A clone and a declaration + +The example is the alpha-design trial shipped with the package: 24 +genotypes in 72 plots, three replicates, six blocks of four plots within +each replicate. + +```{r trial} +trial <- read.csv(system.file("extdata", "john_alpha.csv", package = "masque")) +r <- propose_roles(trial, detect = FALSE) +r <- set_role(r, c("rep", "block"), role = "design", action = "keep") +r <- set_role(r, "gen", role = "treatment", action = "keep") +r <- set_role(r, "yield", role = "outcome", action = "scramble") +r <- set_role(r, c("plot", "row", "col"), role = "id", action = "keep") +m <- mask(trial, r, seed = 1L) +clone <- synthetic(m) +``` + +The control spec is declared once, before any control exists. It names the +column to permute, the strata to permute it within, the outcome to plant an +effect on, the effect per genotype in units of the outcome's standard +deviation, the number of replicates and the seed. Here the planted +genotype effects are drawn from a normal distribution with standard +deviation 0.5, under the spec's seed, so the same vector is planted on +every table the spec meets. + +```{r spec} +spec <- control_spec( + on = "gen", + target = "yield", + effect = list(sd = 0.5), + strata = "rep", + n = 19L, + seed = 2026L +) +spec +``` + +## The analyst's side: controls from the clone + +The analyst holds the clone and its role table, and nothing else. The +controls are derived from the clone, so they disclose nothing the clone +does not. + +```{r controls-clone} +ctl <- controls(clone, roles(m), spec) +ctl +``` + +The print shows the resolved declaration and the planted truth: for each +genotype, the shift in standard-deviation units and the shift in yield +units that the positive control adds to that genotype's plots. The `@truth` +table holds the same, and `@manifest` lists every dataset with its seed. + +A small pipeline stands in for the analyst's: an analysis of variance of +yield on replicate, block within replicate and genotype, returning the +p-value of the genotype term. + +```{r pipeline} +genotype_p <- function(d) { + blocks <- interaction(d$rep, d$block, drop = TRUE) + fit <- stats::lm(yield ~ blocks + gen, data = d) + stats::anova(fit)["gen", "Pr(>F)"] +} +``` + +On the clone itself the genotype term is not expected to show, because the +default clone re-simulates yield without a genotype effect. On the null it +must not show. On the negatives it shows at the pipeline's false-positive +rate. On the positives it shows at the rate the pipeline finds the planted +shift. + +```{r run} +p_clone <- genotype_p(clone) +p_null <- genotype_p(control_table(ctl, "null")) +p_neg <- vapply(seq_len(spec@n), function(i) { + genotype_p(control_table(ctl, "negative", i)) +}, numeric(1)) +p_pos <- vapply(seq_len(spec@n), function(i) { + genotype_p(control_table(ctl, "positive", i)) +}, numeric(1)) +data.frame( + dataset = c("clone", "null", "negative (19)", "positive (19)"), + p_or_share_below_0.05 = c( + round(p_clone, 3), round(p_null, 3), + mean(p_neg < 0.05), mean(p_pos < 0.05) + ) +) +``` + +The negatives give the pipeline's false-positive rate, and the positives the +rate at which it finds the planted shift. Both describe this pipeline on this +table under these permutations, learned without a single real yield. Another +table with the same design can give other rates. + +The construction can be checked as well. The genotype means on a positive +replicate, less those on its negative replicate, are the planted shifts +exactly, because the two share one permutation. This checks the controls, not +the pipeline: raw genotype means recover the shifts by construction, so only an +adjusted estimate can show a fault in the pipeline. + +```{r recovery} +neg <- control_table(ctl, "negative", 1L) +pos <- control_table(ctl, "positive", 1L) +recovered <- tapply(pos$yield, pos$gen, mean) - tapply(neg$yield, neg$gen, mean) +planted <- ctl@truth$shift_realised[match(names(recovered), ctl@truth$level)] +all.equal(as.numeric(recovered), planted) +``` + +The planted size is in units of the yield's standard deviation in the table +at hand. That standard deviation holds the block variance and any real +genotype effect, so a size of 0.5 is not half the noise the pipeline sees. +Read it against the residual of the pipeline's own model: + +```{r residual-units} +residual_sd <- function(d) { + blocks <- interaction(d$rep, d$block, drop = TRUE) + summary(stats::lm(yield ~ blocks + gen, data = d))$sigma +} +0.5 * ctl@truth$sd_y[1] / residual_sd(clone) +``` + +## The owner's side: the same spec on the original + +The owner runs the pipeline the analyst handed back on the original table, +and on controls derived from the original with the same spec. One thing +changes: the permutations are drawn privately. Here that is an integer given +as `seed`, which overrides the spec's seed; the plan route below uses a 256-bit +key instead. The planted vector still comes from the spec's seed, so the truth +is the same on both sides. A public seed would let anyone who knows the design, +which the clone shares, rebuild every permutation and read the original outcome +out of per-replicate results. An integer has at most 31 bits, so it hides the +permutations only while nothing per-replicate is released. + +```{r controls-original} +ctl_owner <- controls(trial, r, spec, seed = 40371L) +p_real <- genotype_p(trial) +p_neg_owner <- vapply(seq_len(spec@n), function(i) { + genotype_p(control_table(ctl_owner, "negative", i)) +}, numeric(1)) +p_pos_owner <- vapply(seq_len(spec@n), function(i) { + genotype_p(control_table(ctl_owner, "positive", i)) +}, numeric(1)) +``` + +Each reading is classed by what it estimates before it goes through the owner's +usual output checking. The false-positive rate over the negatives describes the +pipeline under sham assignment. For a test whose null distribution does not +depend on the yields, it says little about them. The rate +at which the positives find the shift depends on the residual variance, so it +tells something about the scale of the real noise. The permutation p-value, the +share of negatives whose p-value is at or below the real one, is the result of +the analyst's test on the real table, and it is checked as a result. + +```{r verdict} +data.frame( + verdict = c( + "false-positive rate (negatives)", "planted shift found (positives)", + "permutation p-value of the real result" + ), + value = c( + mean(p_neg_owner < 0.05), mean(p_pos_owner < 0.05), + (sum(p_neg_owner <= p_real) + 1) / (spec@n + 1) + ) +) +``` + +The owner's readings describe the pipeline on the real table, and they are the +ones that answer the analyst. The clone's readings are a rehearsal. Here the +two sides read close to each other, within what 19 replicates can resolve. +They need not. The default clone re-simulates yield without the block +structure, so its residual variance differs from the original's. A gap +between the readings is a statement about the clone. Agreement is not a check +on the pipeline, and a rate of zero on both sides agrees while saying nothing. +The same planted 0.5 is +`r round(0.5 * ctl_owner@truth$sd_y[1] / residual_sd(trial), 2)` residual +standard deviations of the pipeline's model on the original and +`r round(0.5 * ctl@truth$sd_y[1] / residual_sd(clone), 2)` on the clone. + +The control tables built from the original hold the real values and never +leave. Per-replicate results do not leave either: each replicate is one +more query against the original, and with the permutations known they are +linear equations in the real outcome. The replicate count in the spec bounds +the queries for one pipeline, and every further pipeline or spec run on the +original is another set of them. + +## Carrying the spec in a plan + +So far the spec has been a separate object. A plan fixes it in advance, with the +roles, and takes it through the boundary. At `mask()` the spec is resolved on +the source and recorded on the recipe. Where a unit or strata are not declared, +the plan picks them: here the replicate, since each replicate holds every +genotype and a block does not. The clone is unchanged by the spec. + +```{r plan-mask} +r_plan <- set_role(r, "gen", role = "treatment", action = "alias") +plan <- masque_plan( + r_plan, + conditional = "design", + controls = control_spec( + on = "gen", target = "yield", effect = list(sd = 0.5), n = 19L, seed = 2026L + ) +) +m_plan <- suppressWarnings(mask(trial, plan, mode = "local", seed = 1L)) +recipe(m_plan)@plan@controls$default@strata +``` + +`conditional = "design"` keeps the design means and block variance and carries +no genotype effect. A real effect moves into the residual, as it does when the +owner permutes genotype for a negative control, so the clone stands in for those +negatives. A fit on this clone does not show the residual of a fit on the +original. + +The analyst gets the plan in the clone's own names and labels. `handover()` +writes it to a folder with a collaborate-mode clone, the data card and a note, +and leaves the recipe out. From there the analyst rebuilds the declared controls. + +```{r plan-analyst} +analyst_plan <- clone_plan(m_plan) +ctl_analyst <- controls(synthetic(m_plan), analyst_plan) +ctl_analyst@spec@strata +``` + +The owner passes the same plan with the recipe and the real table. Cleaning is +replayed, a table that does not match the recipe is refused, a spec changed since +`mask()` is refused, and the permutations come from the digest of a key on the +recipe, with no integer seed to search, so the analyst cannot rebuild them. + +```{r plan-owner} +ctl_owner_plan <- controls(trial, analyst_plan, recipe = recipe(m_plan)) +identical(ctl_owner_plan@seeds, ctl_analyst@seeds) +``` + +## What a control cannot tell you + +* A kept column is structure and is never permuted. Leakage through kept + columns, ids or row order passes the null. +* The strata should let every level of the column reach every stratum, as + complete replicates do. Permuting within incomplete blocks leaves each + block with its own set of genotypes, so part of the real genotype signal + survives on the negatives, and the false-positive rate reads high or low + depending on the layout. +* A sequential test depends on the order of its terms. A treatment entered + before the blocks carries their variance, and its false-positive rate on + this design is well above nominal. The negatives show this, as they should. +* The negative control reassigns one column under exchangeable allocation + within the strata. The outcome is unchanged, so a large real effect of that + column makes the residual on a negative larger than on the original. It + cannot detect confounding, and on observational data it is a permutation + test with no randomisation guarantee behind it. +* A planted effect is clean: the same shift for every plot of a genotype, + no confounding, no interaction. The positives measure sensitivity to a + clean shift of the declared size, not power against the real effect, and a + pipeline that is biased under confounding still finds it. +* A shift is added in units of the outcome's standard deviation in the table + at hand. That standard deviation includes the design variance and any real + effect, so the same spec plants the same nominal size on the clone and on + the original while the shift relative to the residual may differ. The + `@truth` table records the standard deviation used and, as + `sd_within_strata`, the standard deviation left after the stratum means + are removed. That figure still holds the effect of `on` and any variance + within the strata, so it is larger than the residual of a model that fits + them. +* Nothing is clipped. A negative shift can take a non-negative outcome + below zero; `control_table()` warns when it does. On a count the shift is + added, not a change of rate. +* A negative control estimates the size of a test that uses within-stratum + contrasts, so fit the strata. On this design, with replicate strata, `yield ~ + rep + gen` rejects near 5 percent and `yield ~ gen` rejects almost never, + because the replicate variance stays in its residual. A shared rate on both + sides is agreement between them, and zero on both says nothing about + calibration. +* The null collapses the design means of row-level outcomes, so a mixed + model sees a different variance structure. Calibrate false positives on + the negative control, not on the null. +* A pipeline adjusted until its controls read well has been fitted to this + null and this planted shift. It can still be wrong for every other + question. + +```{r teardown, include = FALSE} +options(old_options) +```