Repository navigation
masque 0.16.0: CRAN resubmission - #1
Merged
Merged
Conversation
Guided masque() on a workbook or folder proposes roles on the cleaned headers. mask_set() accepts roles keyed by original or legalised names and passes each table's cleaning record to mask() through the internal .cleaning argument, so set recipes round-trip renamed headers. Duplicated input column names are refused with masque_duplicate_names_refusal.
A linked join key is aliased by the shared map in every table, so the clone keeps it. .recipe_dropped_cols() now skips any column with a level map, and mask_set() records linked keys as aliased on each table's recipe.
…n a declared coordinate is refused F5: .normalise_coords() treats a top-level list carrying lat and lon as one pair, as .normalise_events() does. N3: a declared coordinate whose action was chosen explicitly (or whose provenance is unknown) is refused with masque_coords_action_refusal; a proposed default still yields to the declaration and the recipe notes it.
The design shifts added back after the within-stratum copula were rounded only for integer storage. Whole-valued doubles are now rounded the same way, keeping their double class. The range is not clipped. The red test, test-fix0160-conditional-whole.R, landed in 86b145f by mistake.
… class The recipe bundle printed through S7's default str(), listing every level map and the seed; str() showed the seed of a single recipe; format() errored on the bundle, the masque object and the set. The bundle now prints a header, link names and each table's redacted recipe, and all four classes share a one-line format(), a property-free str() and a knit_print() registered when knitr loads.
…imulated A one-row clone returned the original value and a three-row clone the original permuted. Collaborate mode now refuses with masque_too_few_rows_refusal when nrow < 5 and any column is re-simulated; local mode warns with masque_too_few_rows. A short table of kept, aliased or dropped columns is unaffected. Two set fixtures in the guided and safety-gate tests grow their 2-row side tables to 5 rows.
save_recipe(), read_recipe(), reveal_maps() and audit_mask() refused the set objects that masque() tells users to pass them. All four now dispatch on the bundle: reveal_maps(rec, table = ) reveals one table or each, audit_mask() returns and prints one audit per table and takes a named list of originals. A wrong file names the class found and the version that reads bundles. audit_mask(original =) now applies the recipe's cleaning and reads aliased column names back, which it reported as dropped.
Summed gaps could run past the observed maximum (28.8 to 44.0 against 23.7). Each row's final time, with the times tied to it by zero gaps, is now capped at the longest final time of the rows its gaps were drawn from; earlier times are left alone, and a row whose earlier time already passes the cap is scaled as a whole so no positive gap becomes a tie. The event report and recipe line record the capped and rescaled counts.
…ipe path and a fractional seed recipe() and synthetic() passed the second message fragment into cli_abort()'s class, so the class was the template string and the message stopped at 'got'. save_recipe() wrote RDS under any name, including .csv; it now refuses a path not ending in .rds. A seed of 1.5 behaved as 1 and a seed of Inf failed inside set.seed(); every seed now passes one check (whole, finite, integer range) with masque_bad_seed_refusal, reached from mask(), mask_set() and masque().
The audit pasted every row of the NA matrix into a string, the slowest step on a large table. data.table grouping on the logical columns gives the same share at a fraction of the cost; the namespace now declares itself data.table-aware so its [ uses data.table semantics.
…lves (F6) After environment scope resolves, environment columns leave block_cols, whole_plot_col and spatial_cols, and block_cols becomes the within- environment block basis when every environment agrees on it, or empty when they disagree. A pooled block hint that lost its place is dropped. evidence$block_revision records the change and print() explains it. env = FALSE is unchanged.
mask() records the effective merge map, the applied type changes, the modes and max_levels from a conformance object (new recipe property conform, default NULL) and uses the object's cleaning record. apply_recipe() replays merges then types after the cleaning; a recipe saved before the property existed applies unchanged. Merge chains are resolved to the label left in the data; a source with two candidate targets is not merged and raises masque_conform_ambiguous_merge. Categories are found independently of types, so types = 'off' still merges. mask_set() and masque() accept a conformance object per table.
Labels that differ only inside a run of digits (G01/G02, Plot12/Plot13) are different codes: clean_table() no longer lists them and conform_table(merge_labels = 'auto') no longer merges them (it merged genotype codes into each other before). The hygiene report prints 20 pairs and the count; a recipe keeps the count and 20 pairs as a plain data frame, so a saved recipe reads back identical().
A treatment hint from the design rules is auto-applied only for a treatment-pattern name, a user treatment, or a column crossed with a design column (design-pattern name, column roled design, or detected environment). On the scoped path, a column with one value per level of a design column and a column with an outcome-pattern name are no longer design or treatment candidates, a treatment-pattern name counts whatever its cardinality, and a split-plot names both factors as the treatment. DESIGN_PATTERN gains site_name and trial_site in snake, spaced and dotted forms (also environment candidates); numeric columns named yield, grain_yield, response or outcome are proposed as outcome. env = FALSE is unchanged.
…TRUE) A covariate constant within each trial came back varying within trial in every mode, and a per-trial categorical came back with every level inside every trial. .detect_grain() finds, for each re-simulated column, the coarsest design column or pair of design columns within which it holds at most one distinct non-missing value (replicated units only). mask() synthesises those columns on the unit-level table, numerics jointly through the copula and categoricals permuted across units, and broadcasts each unit's value to its rows. Collaborate mode falls back to rows below five units with masque_grain_small_units. grain = FALSE gives the row-level clone, byte for byte. The recipe gains a grain property and one line per grain. A table with no grained column clones exactly as before.
…d it One sentence in mask(coords =) and jitter_coordinates(seed =): keep the seed and publish one jittered release of a set of sites. Also shortens three internal comments.
…n (F22) The masque_environment_disclosure warning and the roles_validate() refusal of alias on a numeric design column both say: convert the column to a factor in the data and re-run propose_roles() (alias), or drop it.
…gn unit A scrambled date-role column (a per-trial sowing date) is a grain candidate. Only retained design columns form units, so a dropped trial column is not broadcast back into the clone through its covariates.
On base, a table whose only factor was Rep had Rep promoted to a kept treatment by the CRD rule, and the audit graded that passthrough HIGH. Rep now stays a design column (graded low): the jitter tests no longer expect a HIGH warning and the blocked summary counts 1 HIGH.
…fo aside masque() without mode lets mask() and mask_set() inherit the mode of the supplied roles, so a collaborate plan is no longer downgraded to local. masque() gains detect, sheets, info and ... (forwarded to mask() or mask_set()); mask_set() gains sheets, info and refuses unused arguments with masque_unused_arg_refusal. Tables named in info are set aside before masking, never written, and only their names are kept on the recipe bundle; a table that looks like documentation raises a masque_info_hint message. read_set() refuses sheets on anything but a workbook. The guided summary counts every planned column, as the audit does, and names the dropped ones; the non-interactive notice no longer advises ask = TRUE when it was passed.
…mn and jointly The audit graded single columns on raw match rates: it could not see a joint or categorical leak, and a 0/1 outcome sat at medium by construction. Each categorical, logical, date and two-valued column now reports its same-row share in the original label space against the exact permutation chance (sum of p_k^2, within the conditioning strata for a scrambled numeric of a conditional clone) and is graded on the excess. Two-valued columns leave the exact-match rules. Table-level checks in attr(audit, 'joint') add a same-row joint match and a whole-row match over the content columns and a rare-cross count over visible categorical labels, each against permutations drawn under a fixed internal seed, plus an ungraded unit-size fingerprint note. In collaborate mode the joint verdicts raise masque_high_leakage, count in the tally and block package-managed writers; local mode reports them.
…eep warning - mask_set() draws link maps under seed, not the caller's RNG state - jitter_coordinates() and synthesise_geospatial() refuse a bad seed with masque_bad_seed_refusal - .report_conformance() shows labels with braces as text instead of evaluating them - the all-keep warning carries class masque_all_keep - .normalise_coords() reads spec entries by exact name
… print lines, set grain, side defects)
…outcome HIGH, grain audit) # Conflicts: # R/mask_set.R
…iased site The mask_set() example had 4-row tables, which the five-row floor now refuses in collaborate mode, and its comment claimed shared site aliases while site, a design column, was kept as is.
Columns graded on excess over chance (grained, whole-valued covariates and, now, whole-valued outcomes) grade HIGH at 50 points and z 4 over the chance baseline, counted in grain units. A continuous grained column whose unit values are real unit values (copied or moved between units) grades HIGH.
…bmission notes NEWS 0.16.0 replaces the unreleased 0.15.1 section. Vignettes gain grain, time-to-event, numeric treatment, kept outcome and small-table sections, alias year before masking the met_set, count table-level audit findings, and rename the plan variable that shadowed roles(). Roxygen documents the alias label schemes and what reveal_maps() prints.
…ink-map sub-seed)
x@block_cols[1L] is NA on an empty vector, so the length guard never fired and table() errored instead.
…g classes, link stream and tile guard
… on x86_64 The 15-digit text form differs between arm64 and x86_64 in the last digit, so the reproducibility and grain = FALSE digests failed on every Linux and Windows CI job. Local-mode digests at 10 digits are unchanged from 12ead24.
A proposed drop is part of the reviewed plan; coords no longer overrides it.
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment Thanks for integrating Codecov - We've got you covered ☂️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
masque 0.16.0
Behaviour changes
mask()synthesises a column that is constant within a design unit (atrial-level rainfall, a site soil type) once per unit, so the clone keeps
one value per trial. This changes the default clone of such tables.
grain = FALSEgives the row-by-row clone of 0.15.0. In collaborate modea design column with fewer than five units falls back to row-by-row
synthesis with a
masque_grain_small_unitswarning.which copied the real minimum and maximum into the clone. A value can
now sit up to half a measurement step outside the range.
collaborate mode aliases it by default (
trt_NNN).unmask()andapply_recipe()restore the exact numeric values.keepaudits HIGH and blocks thepackage writers.
allow_kept_outcome = TRUEallows it after review andis recorded on the recipe.
is refused (
masque_too_few_rows_refusal). Local mode warns.propose_roles()promotes a column to treatment only when it is crossedwith the design, so a clinical
diagnosisor a trial-level covariatestays a covariate.
yieldand similar names are proposed as outcomes,and
Site Nameortrial_siteas design.mask()andmask_set()the new arguments come beforequiet(andbefore
allow_unmasked_coordsinmask()), so pass those by name.as before.
mask_set()link aliases now followseedwhatever the caller's randomstate, from their own stream derived from
seed, so a set made with aseed in 0.15.0 gets different link aliases when it is made again.
an ordinary clone audits low instead of medium.
New features
data_card()writes a data card for a clone, or one per table of a set:size, design columns and per-column counts, each marked as exact in the
clone or re-simulated. It never lists a level label.
roles()returns the role table of a clone under the clone's columnnames.
mask()gainsgrain,events(linked time-to-event outcomes kept inorder, with exact counts of each status pattern and no time past the
longest observed follow-up) and
allow_kept_outcome.mask_set()gainsgrain,allow_kept_outcome,sheetsandinfo(documentation sheets set aside, not masked).masque()takesdetect,sheetsandinfo, and passes otherarguments to
mask()ormask_set(). An argument the verb does not takeis refused.
audit_mask()grades each column on its excess over a chance baselineand adds table-level checks (whole-row matches, rare combinations) in
attr(audit, "joint"). A column synthesised per unit, or a whole-numberedcovariate (age in years), is graded on that excess instead of on raw
exact matches, which it reaches by construction.
audit_mask()grades such a column HIGH when it matches its own row ordesign unit far above chance (50 points and 4 standard deviations), and a
continuous per-unit column HIGH when its unit values are the real ones,
even moved between units. With 5 to 8 units and few distinct values a
copy cannot be told from chance, and the audit grades it low.
masque_class, includingmasque_roles_upgraded,masque_roles_mode_mismatch,masque_plot_skippedandmasque_geo_anchor_missing.save_recipe(),read_recipe(),reveal_maps(table = )andaudit_mask()work on amask_set()result.conform_table()decisions, andapply_recipe()replays them on the original.
Bug fixes
conform_table(merge_labels = "auto")merged distinct genotype codes(
G002andG011intoG001). Codes that differ only in a run ofdigits are never merged now. A label whose merges end on two different
labels is left alone, with a warning.
conform_table()ran a label containing braces as R code when printingits report. Labels now print as text.
masque()given a roles table made for collaborate mode masked in localmode and returned real labels. It now uses the plan's mode.
print(),format(),str()and knitr output of a recipe, a recipebundle, a
masqueor amasque_setno longer show the seed or the labelmaps.
masque()writes the plan and its menus asmessages, so
suppressMessages()silences them.masque()on a folder or workbook with column names that neededrepair stopped with an error. Set recipes now keep each table's cleaning,
and
apply_recipe()keeps a linked key that the roles markdrop.coordswhose action you set todroporscrambleis refused (masque_coords_action_refusal). Before, thedeclaration silently kept it. A single pair may be passed bare, and its
names must be
latandlonexactly.collaborate mode, and a two-valued column is no longer jittered.
near-duplicate report prints at most 20 pairs.
save_recipe()accepts only a.rdspath. A seed must be one wholenumber, in
jitter_coordinates()andsynthesise_geospatial()too.audits.
apply_recipe()maps a numeric column that has an alias map instead ofpassing real values through.
non-interactive notice no longer suggests an argument already passed.
options()they set.an error, when there is no block column.
masque 0.15.0
Breaking changes
Every table masque returns is now a
data.table:synthetic(),propose_roles(),set_role(),role_options(),audit_mask(),apply_recipe(),unmask(),read_set(), thedataand report tablesof
clean_table()andconform_table(), and therecommended_rolesandper_envtables ofdetect_design(). tibble is no longer a dependency.Code that selects columns with
x[c("a", "b")]should usex[, c("a", "b")], or convert withas.data.frame()first.synthetic(),audit_mask(),set_role(),apply_recipe(),clean_table()andconform_table()return their own copy, so editingthe result by reference with
:=never changes the input data or themasque object.
Documentation
The package has a logo.
The README states no hard-coded version, carries status badges, and has
a Related packages section on synthpop and sdcMicro.
Getting started edits the role table with data.table syntax.
Continuous integration
test-coverageworkflow (covr, uploaded to Codecov) andlintworkflow (lintr with the package's
.lintr; it reports and does notfail).
masque 0.14.1
Documentation
Confidentiality states what a recipe holds on its own (the real labels,
the original column names and the seed) and adds a row for conditional
clones: they reproduce the outcome mean of each treatment, block and
environment level, so a level with only a few plots shows close to its
real mean.
The README, API stability and the other vignettes are reworded more
plainly, and the pkgdown site lists the articles in reading order.
masque 0.14.0
Security
scrambled treatment listed its levels in the scrambled order, so
levels()of the clone showed the whole map. In 0.12.0 and 0.13.0, ajoin key aliased by
mask_set()listed its aliases in the originallevel order, which undid their random assignment. If you shared such a
clone, make it again with 0.14.0 and share the new one in its place. The
same seed gives the same values. Only the level order changes.
Bug fixes
apply_recipe()returns an aliased factor with the clone's levels.It kept the original level order, so a model fitted on the translated
original had a different reference level from the same model fitted on
the clone, and coefficients compared by position did not line up. On
agridat::besag.metin collaborate mode the genotype coefficients of thetwo fits correlated at 0.05 by position and 0.92 by name. Predictions
were not affected.
Minor improvements and fixes
Messages say
columnorcolumns,siteorsites, where they saidcolumn(s).The vignettes and help pages are reworded more plainly. Recipe anatomy
and the round-trip said its coordinate example was jittered as one site;
each plot there is its own site.
masque 0.13.0
Breaking changes
The conditioning ladder follows the design hierarchy. The ladder
that coarsens the stratum of a conditional clone used to drop design
columns by level count, so on a multi-environment trial it dropped the
environment before the replicate inside it and the clone lost the
environment effect: on
agridat::besag.metthe county F statistic wentfrom 987 on the original to about 1 on the clone, and the genotype
ranking went with it.
mask(),mask_set()andmasque()take aladderargument. The new default,"hierarchy", drops plot coordinatesfirst, then blocking columns, then environment columns, ordering within a
tier by the level-adjusted share of variance a column explains, and
carries the main effect of every dropped column, and of each pair of
dropped blocking or environment columns, into the clone as an additive
shift estimated beside the stratum that was kept. Rows the stratum floor
pools into the fallback keep the stratum columns' main effects the same
way, so a three-replicate variety trial keeps its genotype means. A level
with a single row is pooled with the other singletons before the fit, so
no row's own outcome is ever carried; a term that is noise is shrunk to
nothing and the residual spread is rescaled for the degrees of freedom
the fit removed. Over twenty seeds the county F on
besag.metis 952against 987 and the nested county:rep F 28 against 33; on
agridat::yates.oatsthe block F is 11.3 against 13.2 where it was 0.8.The ladder of 0.11.1 to 0.12.0 is available as
ladder = "levels", anda recipe records which one made it.
The recipe gains
ladder,conditioning_dropped(in drop order) andconditioning_shifted; themasque_conditional_degradedwarning and therecipe print name the shifts. The new vignette What a conditional clone
keeps of the design runs eight
agridatdesigns through one stated passrule under both ladders.
Bug fixes
print()on the records fromclean_table()andconform_table()dispatches from an installed package. The S7 method assignments for
print(),format()andplot()left a copy of each generic in thenamespace, so the two S3 print methods registered on that copy and an
installed masque printed the raw list,
attr(,"class")and all. Theassignments now run inside
local(). A test installs the package into atemporary library in a subprocess and prints both records.
plot_design_summary(engine = "ggplot2")returns the plot. Itreturned the summary invisibly after printing, so
ggplot2::ggsave()failed and the Getting started vignette's description was wrong. The
ggplot2 engine now returns the
ggplotobject for every design class;the split-plot and CRD panels, which had only a base-graphics branch,
gained one. The base engine still draws and returns the summary
invisibly.
detect_design()reads a blocked multi-environment trial as theincomplete-block design it is. The lattice rule required every block to
hold the same number of treatments and every treatment the same number of
blocks, so one filler plot in
agridat::besag.metfailed both tests andeach county was reported as CRD with
repandblocknever mentioned.Regularity is now the share of blocks at the modal size and of treatments
at the modal replication, floored at 0.9 each;
besag.metreads as analpha-lattice on
rep:blockin every county.conform_table()names the reason a character column is left as text: afew values that do not parse as numbers, labels that are all distinct, or
a vocabulary wider than
max_levels. The last was reported for all three.mask()andmask_set()check the shape ofrolesbefore they read itsmode provenance, so something that is not a roles table is refused for
that reason, with the typed
masque_bad_roles_refusalclass, and nowarning about a missing
modeattribute precedes the refusal.Features
mask()refuses a missingroleswithmasque_missing_roles_refusalinstead of R's bare missing-argument error, accepts the record from
conform_table()and masks itsdata, and takesquiet = TRUEtosilence only the
masque_mode_downgradewarning for a plan that isdeliberately shared across both modes.
synthesise_geospatial()refuses a wrong-shapedanchor_centroids(adata frame, an unnamed list, an element that is not a
c(lat, lon)pair, or a list naming none of the anchor levels) with
masque_bad_anchor_centroids_refusal, instead of warning once per leveland returning every coordinate
NA.Testing
agridatexemplars (CRD, RCBD,Latin square, split-plot, alpha lattice, augmented, blocked MET, repeated
measures): allocation byte-identical, NA mask identical, blocking and
environment F at half the original or better where the original detects
the term, treatment level means correlated at 0.7 or better and within a
factor of two in spread. Three seeds on CRAN, twenty off it.
masque 0.12.0
conform_table()takes a cleaned table the last step to an analysable one, andreports every assumption it makes as a plain sentence. It handles the two jobs
clean_table()deliberately refuses: deciding that"NSW"and"Nsw"are onecategory, and deciding that a character column holding six labels is a factor.
Both are off by default (
merge_labels = "report",types = "report"), sothe first call names what it would do and changes nothing.
Storage is decided before categories, which is the ordering that matters: a
column of numbers held as text is not a set of categories, and merging
"4.2"into
"4.4"because they differ by one character is how an automatic cleanerdestroys data. Only columns that remain categorical are considered for merging,
and a one-character edit between labels shorter than four characters is ignored.
A pair differing only by capitalisation is one category whatever happens, so it
is merged onto the more frequent spelling (on a tie, the one that appeared
first). A pair differing by one edit might be two real categories, so it is
merged only when one spelling is clearly more common, and refused otherwise.
All three vignettes were brought to the orchestra vignette quality bar:
each now opens with the question in the voice of the person asking it
(a custodian or an analyst), follows a fixed Why/What/Do/Read/Limits
shape, carries a
ggplot2figure that shows what its tables cannot(the conditional-clone treatment-effect recovery, the multi-environment
coverage plot, and the original-versus-clone marginal overlay), and
demonstrates a fail-closed refusal for real in each case (an
incompatible role/action pairing, a blocked HIGH-leakage write, and an
unmapped factor level).
Every orchestra-facing decline in
mask(),mask_set(),unmask()andaudit_mask()that was a bare, unclassedcli::cli_abort()now carries aclass ending in
_refusalplus the sharedorchestra_refusalmarker, soa caller (or a conductoR node) can catch the decline programmatically
instead of string-matching the message.
masque_unmasked_coordsandmasque_conditional_degradedwere already classed and are unchanged.mask_set()draws the alias map for linked (join-key) columns from theseeded permutation like every other alias map; it was lexicographic and
seed-invariant, so a public vocabulary could invert it (M-01).
Bug fixes
The collaborate-mode alias map is no longer the sort order. Aliases
(
trt_001,<col>_L001, ...) were assigned to levels in lexicographicorder and were invariant to
seed, so a holder of the synthetic whocould guess the vocabulary -- a public variety roster, an N-rate ladder,
a site list -- inverted the map by sorting a candidate list, with no
recipe. The assignment is now drawn from a uniform random permutation
taken from the seeded stream
mask()already uses. The map is recordedon the recipe exactly as before, so
apply_recipe()/unmask()stillinvert it, and a fixed
seedstill reproduces it. Two notes forcallers: the seed is now confidential material (published beside the
synthetic it re-opens the inversion), and level frequencies are
unchanged by design and remain matchable. The
confidentialityvignette states both.
This changes the integer codes of an aliased factor, which are now a
relabelling, not a copy of the original codes. Code that compared
as.integer()of an aliased column against the source should compare thepartition instead (recode both by order of first appearance); the
incidence structure itself is unchanged.
conditional = TRUEno longer collapses silently to the pooledcopula. The conditioning stratum was treatment crossed with every
retained design column, so on a replicated factorial (six N rates by
three varieties by four replicates) every cell held one row, all of them
fell below the minimum stratum size, and the whole numeric block was
pooled -- the marginal clone under a conditional label, with the recipe
still asserting
conditional = TRUE.mask()now walks a coarseningladder: design columns are dropped, the finest first, until the cells
reach the minimum; treatment columns are never dropped. On a replicated
2 x 3 factorial with a planted effect this restores the treatment
sum-of-squares fraction from 0.004 to inside the stratified-bootstrap
envelope of the source's 0.859.
Degradation of a conditional clone now raises a classed
masque_conditional_degradedwarning -- when the ladder gives up acolumn, when rows remain in the pooled fallback, and when there is
nothing to condition on at all -- so a caller can catch it instead of
matching on message text.
Features
conditioning_used(the rung of the ladder actuallyreached) and
fallback_frac(the share of rows pooled at that rung)alongside the existing
conditioning_cols(what was requested).print()on a recipe names the coarsening and the pooled share.Testing
min_stratumat 2, 5 and 20 on a layout whose cells straddle allthree, and
synthesise_numeric_local(regularise = FALSE)on arank-deficient block -- both previously unexercised at any value.
Documentation
getting_startedvignette's environment-overview figure now has afig.cap/fig.altand two sentences reading the panel (what even barheights and a single connected component mean, and what a broken MET
would look like there).
getting_startedmode-comparisontable ("numeric values that may match observed" -> "... the observed
values").
recipe_anatomydocuments the recipe'scoordsandallow_unmasked_coordsproperties (added in 0.10.0/0.11.0) and showsrecipe(m)@coordson a worked coordinate example.README.mdno longer hard-codes a stale version number; points toNEWS.mdinstead and flags the 0.10.0/0.11.0 coordinate-handlingchanges for readers coming from an older tag.
apply_recipe()'s roxygen no longer describes the pre-0.6.0ignorerole; step 2 now states the current, mode-independent
action == "drop"rule.@examplestoread_recipe()andunmask(), the package'slast two exports without a runnable example.
±ininternal and roxygen comments to the package's ASCII
--/+/-convention (no rendered-text or behaviour change).
masque 0.11.0
Breaking changes
A coordinate in a masked table is masked unless you say otherwise.
mask()now stops when a column whose name says coordinate (gps,lat,lon,easting,northing,utm,wgs84, ...) would be writtenthrough with
action = "keep". A position locates the site and often theoperator with it, so the burden is on the caller to say what should happen
to it, and when nothing is said the default is to refuse.
Three ways to state otherwise, all unchanged in spelling:
coords, which coarsens it by the on-land geomask;drop, the defaultpropose_roles()already proposes for a coordinate-named column, or
scramble);allow_unmasked_coords = TRUE, having decided thecoordinate is not sensitive. It is recorded on the recipe.
The refusal is the classed condition
masque_unmasked_coords.A column detected as a coordinate only by the shape of its values -- a
numeric pair inside plausible latitude and longitude ranges carrying at
least four decimal places, which is metre-scale precision no temperature or
yield column has -- raises the classed warning
masque_coords_suspectedand does not stop. Confidence is lower there, and a false positive should
not block a legitimate mask. Ordinary numeric covariates do not trip it:
whole numbers, low-precision values and unpaired columns are all excluded.
Bug fixes
audit_mask()reported a coordinate declared tocoordsas HIGH leakagewith the note "kept as-is - visible to collaborators". The column had in
fact been coarsened in place by the geomask, displaced by the requested
radius and shared across its site. It now audits as medium in
collaborate mode, with the note "coordinate coarsened in place by the
geomask". The stale HIGH also tripped the package-managed write gate, so
every caller who used
coordshad to override it to write.The audit
mask()runs for itself in collaborate mode was built on atemporary recipe that omitted the coordinate record, so it reached the same
wrong conclusion as above even after the record existed.
masque 0.10.0
Breaking changes
A coordinate belongs to a site, not to a row.
jitter_coordinates()andmask(coords = )now draw one displacement per site and broadcast it toevery row of that site, instead of displacing every row independently. A
table holding many rows per physical place -- plots within a trial, samples
within a paddock, observations within a farm -- now comes back with one
masked coordinate per place, as its source table has one true coordinate per
place.
Tables masked by 0.9.2 or earlier carry the old per-row structure. Their
coordinate columns assert that observations kilometres apart share a site,
and should be treated as invalid for anything spatial until the table is
rebuilt under 0.10.0.
The change closes a confidentiality defect as well as a fidelity one. Donut
displacement is isotropic, so the mean of many independent draws around one
true site converges on that site: averaging the 360 rows of a single trial
recovered the true position to a median 0.58 km, against the 5 km floor the
donut was asked for. One draw per site removes that estimator and leaves the
full displacement in place.
jitter_coordinates()gainsby, which chooses how a site is identified.by = NULL(the default) groups rows sharing an identical input coordinate.A character vector names site columns, for the case where a site's recorded
coordinates differ slightly between rows; such a group is consolidated to its
row-weighted centroid and the consolidation is reported by a classed
masque_geo_consolidatedwarning, withmasque_geo_wide_groupraised when anamed group spans more than the displacement radius.
by = FALSErestoresthe pre-0.10.0 per-row draw and warns (
masque_geo_ungrouped) if the inputcarries repeated coordinate pairs. It exists only so a table masked by an
earlier version can be reproduced.
A site that cannot be placed on land within
max_triesre-draws is now setto
NAon both axes. Leaving it unchanged shippedthe true coordinate inside a table the caller believed was masked, behind
a warning. The
masque_geo_unplacedwarning is unchanged in class and stillnames the count.
Genuinely point-level input -- every coordinate pair already distinct -- is
unaffected by all of the above, and reproduces its 0.9.2 output exactly under
the same seed.
One displacement per site holds within a table, not across rebuilds. Each
rebuild with a fresh seed is an independent draw, so a site kept in several
masked versions can be averaged back out: at the 5-20 km default, eight kept
rebuilds recover it to 4.5 km, below the floor. Reuse the seed and the
displacement is identical, so there is nothing to average. Documented under
"Rebuilding, and the displacement budget" in
?jitter_coordinates.New features
mask(coords = )acceptsbyin a coordinate spec, for examplecoords = list(list(lat = "GPS_S", lon = "GPS_E", by = "TRIAL")). Thegrouping is taken from the original table, so a site column that is
aliased, permuted or dropped in the synthetic still groups correctly.
The recipe records what happened to each declared coordinate pair: the jitter
parameters, the grouping it was masked under, the number of sites that
grouping produced, any consolidated sites, and any site set to
NA. It isshown by
print(recipe(m))and readable atrecipe(m)@coords. Recipeswritten before 0.10.0 read back with no such record.
Bug fixes
mask()gave every declared coordinate pair the same seed, so twolatitude/longitude pairs in one table received identical displacement
vectors. Each pair now draws from its own sub-stream, deterministically
derived from the
mask()seed.masque 0.9.2
New features
mask()gains acoordsargument to coarsen geographic coordinates as partof masking. Declare one or more latitude/longitude pairs (for example
coords = list(c(lat = "GPS_S", lon = "GPS_E"))) and each pair is passedthrough synthesis untouched and then displaced in place by the on-land
jitter_coordinates()geomask (a 5-20 km donut by default), instead of beingcopula-scrambled into implausible locations. A pair may be a bare
c(lat =, lon =)vector or alist(lat =, lon =, min_km =, max_km =, ...)that also carries jitter parameters; both spellings work (numbers supplied
through the vector form are coerced back from strings). A declared pair always
survives, coarsened; the recipe records that it was coarsened, and
apply_recipe()retargets a pipeline to the real coordinates.masque 0.9.1
Breaking changes
propose_roles()now protects a conservatively detectedmulti-environment allocation by default. Before 0.9.1, an environment
column not recognised by the name heuristic could remain
covariate/scramble, which moved observations between environments.High-confidence categorical environment columns now become
design/keepin local mode anddesign/aliasin collaborate mode.Numeric environment columns remain
design/keepand raise a classeddisclosure warning in collaborate mode. Disable the new scope path
explicitly when reproducing the former detector output:
New features
detect_design()gains an append-onlyenvargument and an orthogonalenvironment-scope contract. Use
env = NULLfor conservative automaticresolution,
env = c("site", "year")for an explicit composite, orenv = FALSEfor the whole-table compatibility path. The returneddesign_summaryrecords scope confidence, environment basis, experimentgroups, exact treatment-connectivity components, and advisory
per-environment design summaries without storing the source data.
print()on adesign_summarynow leads with scope and reports bounded,aggregated MET diagnostics.
plot()supports compact base and ggplot2environment overviews plus one selected field layout through
environment =. Connectivity and inner-design labels are diagnostic anddo not claim to reconstruct the original randomisation.
jitter_coordinates()coarsens latitude/longitude in place by anon-land geographic-masking jitter (donut or Gaussian), so a synthetic table
can carry realistic coordinates without revealing a true field or farm
location. The donut default moves every point 5-20 km in a random direction,
re-drawing until it lands on land (via
maps); the NA pattern and thelatitude/longitude pairing are preserved. It complements
synthesise_geospatial(), which instead re-anchors coordinates atuser-supplied fake centroids.
Bug fixes
Omitting
modefrommask()ormask_set()now inherits the mode storedby
propose_roles(); it no longer falls back silently to local defaults.An explicit collaborate-to-local change raises a classed
masque_mode_downgradewarning.mask_set()rejects role plans carryingmixed mode provenance.
mask()andmask_set()now warn (classedmasque_mode_unset) when nomodeis supplied and therolestable carries no mode provenance, insteadof silently masking in
localmode. Adata.table()wrap or asaveRDS()round-trip strips the tibble attribute that records the mode, so the warning
turns a silent local-mode downgrade into a recoverable one.
Site-only environment candidates now require replicated treatment evidence
across sites for automatic promotion. A block nested within one farm is
left for review and not classified as an environment.
Environment connectivity diagnostics now guard both dense incidence and
environment-adjacency allocations. Oversized problems return an explicit
not_computeddiagnostic instead of attempting an unsafe allocation.Experiment-group diagnostics tolerate a small, recorded treatment overlap
while rejecting genuinely interleaved groups. The evidence now reports the
confined and shared-treatment fractions.
Explicitly pinned role actions are no longer overwritten when a detected
environment recommendation promotes a column to the
designrole.A medium-confidence or ambiguous environment candidate is now preserved as
design/keepin both modes; it no longer falls through to acovariatedefault. Previously an unreplicated environment named with a site token
outside the design-name heuristic (for example
county,locationorfarm) could be silently row-permuted by a defaultmask(), movingobservations between environments. Such a column is kept byte-identical and
is never auto-aliased; pass
env =to enable environment-aware masking.Invalid (non-syntactic) column names are now legalised in every
cleanmode, not only"auto", and the repair is raised as a classedmasque_name_repairedwarning and recorded in the recipe. Previously,under
clean = "off"an invalid name likeGY_%VARMAXwas silentlyrewritten by
make.names()inside numeric synthesis with no maprecorded, so the synthesised column no longer matched its source: the
original column survived unmasked alongside the synthetic copy --
a leak -- and the round-trip broke.
mask()now legalises names upfront in all modes, remaps the
rolestable, records the map, andsynthesise_numeric_local()no longer rewrites names.unmask()now restores the original (pre-legalisation) column names,so a legalised name round-trips symmetrically with
apply_recipe().masque 0.8.2
New features
role_options()renders the two-axis vocabulary as data: everyrole and action pair, the storage kinds the pair works for, and the
reason it is constrained when it is. The grid is generated from the
same compatibility rules
roles_validate()enforces atmask()time, so it cannot drift from the validator.
role_options(kind = "factor")filters to the combinationsavailable for one column's storage kind.
Minor improvements and fixes
section: vectorised
set_role(), direct data-frame edits withNAaction re-resolution, the
role_options()grid, and hand-flaggingpii_suspectedon a sensitively valued column the name heuristiccannot see.
masque 0.8.1
Bug fixes
e) no longer destroys the sessionwhen the spreadsheet editor cannot start.
utils::edit()on a dataframe requires the X11 dataentry widget - missing on macOS without
XQuartz and on headless systems - and its error previously propagated
out of
masque(), losing the proposal and the user's place. Theeditor call is now guarded: on failure a dependency-free console
editor takes over (pick a column, then a role and an action from
numbered menus). Every console edit is applied through
set_role(),so vocabulary validation and default-action re-resolution match the
scriptable path exactly, and the roles table's provenance attributes
survive.
(
[e]is read ase).masque 0.8.0
Safety fix: HIGH leakage findings are never silenced (breaking behaviour)
This release corrects a safety-contract defect in the guided workflow.
In 0.5.0 through 0.7.1,
masque()andmask_set()wrapped theirinternal
mask()calls insuppressWarnings(), so the HIGH-leakagewarning raised by the collaborate-mode audit never reached the caller -
and the guided summary could then print sharing language for output the
audit had flagged. Generation must never imply release; the behaviour
now matches the documentation.
masque()andmask_set()no longer suppress warnings frommask().The HIGH-leakage finding is raised as a classed warning
(
masque_high_leakage) so callers can handle it programmatically.masque()'soutandwrite_set()-now refuse to write while a HIGH finding stands: nothing is
written, and the flagged columns are listed with the remedy. A new
allow_highargument (defaultFALSE) overrides the refusal afterthe custodian's own review; the override raises a
masque_high_overridewarning and is recorded in the recipe'swarnings, so the exception stays auditable. Local mode is never gated.
tally (HIGH / medium / low), prints a
BLOCKEDblock with the nextcorrective action when HIGH findings stand, and no longer prints
"share only the synthetic output".
print()on a masque object showsthe same tally.
warning(). Itforced callers (and this package's own examples) to blanket-suppress,
which is exactly how the HIGH warning was lost - an alarm-fatigue
defect. The reminder is recorded on the recipe and shown by
print()and the guided summary instead. Genuine findings keep the warning
channel to themselves.
mask()now errors on unused...arguments instead of silentlyignoring them, so a misspelled argument (e.g.
sedd = 1) can nolonger look like success.
describe the write gate, state plainly that
save_recipe()does notencrypt, that generating a synthetic table is not a release decision,
and that
masquecontributes evidence to Safe Data / Safe Outputswhile the remaining Five Safes are organisational.
masque 0.7.1
Minor improvements and fixes
masque_recipeno longer carries thedetect_design()artefactthat
propose_roles()stashes on the roles table (attr(roles, "design"), plus theproposed_actions/modeprovenance). Therecipe is contractually runtime-minimal, and the design object is not
part of its contract; it inflated the saved recipe and, because its
serialised size varies across R versions, could push
save_recipe()output past the documented small-file budget on R-devel. Saved recipes
are now smaller and deterministic across toolchains. No public API or
round-trip behaviour changes --
recipe(m)@roleskeeps every role andaction assignment.
masque 0.7.0
A conditional clone mode that preserves the treatment-to-outcome
relationship, so a causal model fitted on the synthetic recovers the
real treatment effect -- not just the marginal distribution.
New features
mask(),mask_set(), andmasque()gain aconditionalargument(default
FALSE). Withconditional = TRUE, the numeric block isre-simulated within each treatment-by-design stratum, not from
one global Gaussian copula, so a row's synthetic outcome inherits the
location of the treatment that row carries. The treatment-to-outcome
map -- the quantity a causal model reads -- survives the clone, within
sampling tolerance. The default
conditional = FALSEpreserves theexact 0.6.0 behaviour (global copula, marginal and structural fidelity
only), so existing scripts are byte-identical.
The motivation: structural and marginal fidelity alone are not
sufficient for causal inference. A clone that matches every marginal
and the global covariance can still sever the relationship between an
arm and its response, because the outcomes are drawn from the pooled
distribution and the treatment labels are relabelled independently. A
model fitted on such a clone returns a null effect.
Minor improvements and fixes
masque_reciperecords two new fields,conditional(logical)and
conditioning_cols(the treatment and retained design columns theclone stratified on), and
print(recipe(m))now reports the clonefidelity (marginal / structural versus conditional). Recipes written
by masque 0.6.0 and earlier read back with the back-compatible
defaults (
conditional = FALSE, no conditioning columns).graceful global fallback, so a conditional clone never fails on a
sparse design cell; with no treatment or design column to condition
on, the path degrades cleanly to the global copula with a note.
masque 0.6.0
A two-axis roles step, a hygiene layer, column-name aliasing, a
multi-table set layer, and a single guided verb turn masque into an
end-to-end tool for confidential tabular data.
Breaking changes
rolecolumn (what a columnis) and a new
actioncolumn (whatmask()does to it). The rolevocabulary changed -- the old
keepandignoreroles become thekeepanddropactions, and new rolesdate,id,text, andotherjoindesign/treatment/outcome/covariate.mask()no longer requires a column roledoutcome.automatically by
roles_validate()(and therefore bymask()) with aone-time deprecation warning that preserves the old mode semantics, so
existing scripts keep working. Re-run
propose_roles()to silence thewarning and adopt the new schema.
The guided verb
masque()is the front door: one call reads the input (a dataframe, a file, a folder, an Excel workbook, or a named list), proposes
column roles, pauses for review in an interactive session, masks,
audits, and - given an
outpath - writes the result. It dispatches asingle table through
mask()and a multi-table input throughmask_set(), and returns the same object the lower-level verbs do, soit stays fully scriptable: pass an edited
rolestable to skip theprompt. (The internal S7 result class previously bound to
masqueisnow
masque_obj; the class name is unchanged.)Two-axis roles (breaking, with an upgrade path)
role(whata column is) and
action(whatmask()does to it).roleisone of
design,treatment,outcome,covariate,date,id,text,other;actionis one ofkeep,scramble,alias,drop.propose_roles()resolves a mode-appropriate default actionfor every column, so the table you review is the masking plan that
runs.
first-class
daterole (date/time columns are row-permuted withclass and NA pattern preserved), and the explicit "retain untouched"
and "skip entirely" choices are now the
keepanddropactions,available on any column, where they were once only the
keepandignoreroles.propose_roles()gains amodeargument. The proposed actionsdiffer between
localandcollaborate(for example a treatment iskept locally but aliased for collaboration); the table records the
mode it was prepared for.
set_role()helper edits a roles table ergonomically.Re-assigning a column's role re-resolves its default action; passing
an explicit
actionpins the column so a later mode change leaves it alone. Direct
roles$role[...] <-edits still work.mask()no longer requires anoutcomecolumn. With none marked,the Gaussian copula simply re-simulates every scrambled numeric
column jointly.
role = "design", action = "alias"is a new opt-out frombyte-identical design preservation: the design structure is kept
but the site / block labels are hidden behind opaque aliases and
restored by
unmask(). The default for design columns remainskeep(byte-identical).roles_validate()validates the (role, action, kind) combination andfails closed on impossible pairings (a scrambled design column, an
aliased numeric, a scrambled id). It returns the validated table with
any
NAactions resolved.actioncolumn; thekeep/ignoreroles; themask_levelscolumn) are upgradedautomatically with a one-time deprecation warning, preserving the v1
mode semantics exactly.
Multi-table sets
mask_set()masks a whole multi-table dataset at once - a folderof CSV / TSV /
.fstfiles, a multi-sheet Excel workbook, or a namedlist of data frames - returning one synthetic table per input table
and a single private recipe bundle.
tables (a site code, a genotype name, a plot id) is detected as a
link and aliased identically everywhere it occurs, so a join written
against the synthetic set still resolves on the masked data. Links are
proposed automatically and printed; override with the
linksargument, or set
links = FALSEto mask each table independently.read_set()ingests the folder / workbook / list into a namedlist of data frames (clean rectangles only - a missing header row or
non-rectangular sheet fails with an explanatory error). New
write_set()writes a masked set back out, mirroring the input format(workbook in, workbook out; folder in, folder of CSVs out). The
private recipe bundle is never written by
write_set().synthetic(),recipe(),apply_recipe(), andunmask()alldispatch on the set:
synthetic()returns the named list of tables,recipe()the bundle, and the round-trip verbs operate table by table.data.tablejoins Imports (fastfread/fwrite);readxlandwritexlare Suggested for the Excel paths.Depth controls
alias_namesargument onmask()hides the column namesthemselves - the last identifying surface a kept or design column
exposes.
TRUEreplaces every retained name with an opaque alias(
col_001,col_002, ...); a character vector aliases just the namedcolumns. The original-to-alias map lives in the (private) recipe and
is inverted by
apply_recipe()andunmask(), so a pipeline writtenagainst the aliased synthetic round-trips. This realises the
long-reserved
column_name_maprecipe slot.reveal_maps()now alsoprints the column-name map.
Hygiene
clean_table()verb (and acleanargument onmask(), default"auto") tidies a dirty table before masking: column names arelegalised (valid, unique R names), and leading / trailing whitespace
is trimmed from names and from character / factor labels. Both fixes
are reported through
cli, recorded in the recipe, and re-applied byapply_recipe()so retargeting still lines up."north"vs"North") or by a single edit ("Compass"vs"Compas") - arereported, never merged: deciding whether two similar labels are the
same value is a judgement masque leaves to the user. Set
clean = "report"to preview fixes without applying them, orclean = "off"to skip hygiene entirely.Internal
MASSdependency: the copula now draws latent normalsthrough a Cholesky factor of the regularised covariance, removing a
hard import.
masque 0.5.0
New feature release: joint-treatment masking plus role-table usability
and first-class date handling.
mask()androles_validate()nowaccept designs with two or more treatment factors (factorial,
split-plot trials).
Usability
keeprole: users can now explicitly mark a column forbyte-identical pass-through in both local and collaborate mode. This is
distinct from
design, which remains for experimental-design structure,and from
ignore, which is still dropped in collaborate mode.propose_roles()now proposes Date / POSIX / difftime columns ascovariate, where they wereignore. Date/time covariates are row-permuted,retain their original class, and preserve the cell-level NA mask.
keepwith a clear note,avoiding a confusing attempt to synthesise objects masque does not know
how to mask.
Masking and recipes
matching the documented categorical-covariate contract.
apply_recipe()and
unmask()round-trip those aliases back to logical values.detect_design()and its candidate proposer now honour user-suppliedkeeproles so explicitly-kept columns are not promoted into designhints.
New features
mask()now masks every column flaggedtreatment, not just one.Each treatment factor is aliased independently. A single treatment
keeps the historical
trt_NNNcollaborate-mode alias prefix; withtwo or more, the column name is folded into the prefix
(
<col>_trt_NNN, e.g.variety_trt_001) so the opaque labels staydistinct and self-documenting, mirroring the categorical-covariate
<col>_LNNconvention. Aliasing each factor separately, notthe treatment combination, preserves the per-factor structure that
factorial models fit, and keeps each column's alias namespace -- and
therefore
audit_mask()'s leakage accounting -- unchanged.roles_validate()no longer errors when more than one column isflagged
treatment. The round-trip path (apply_recipe(),unmask()) already inverted multiple per-column level maps, sorecovery of every treatment factor works unchanged.
masque 0.4.1
Maintenance release: contract-sharpening corrections plus the
documentation and metadata that were prepared for v0.4.0 but not
released. No new public exports. The two behaviour changes below are
deliberate fail-closed corrections to existing exports; user code
that depended on the silent failure mode will need to be updated.
Behaviour: fail-closed corrections
apply_recipe()andunmask()now error when a non-NA value isnot present in the recipe's level map. Previously the row was
silently coerced to
NA, which could quietly poison downstreammodel matrices. Schema drift or a new treatment level in the input
now fails closed with the offending values listed.
apply_recipe()now verifies that the NA mask oforiginalmatchesthe recipe's recorded
integrity_fp. A mismatch errors withguidance. New
check_integrity = TRUEparameter (default) givesan escape hatch (
check_integrity = FALSE) for workflows where themissingness has legitimately changed since the recipe was built.
Bug fixes
unmask(x, rec)now passes through atomic numeric, integer,logical, and
Date/POSIXctvectors unchanged, matching thedocumented numeric pass-through contract. Previously these inputs
errored when the recipe held no level maps.
audit_mask()'sexact_match_pctnow divides by the number ofjointly-observed comparable cells, not by
nrow(df). Columnsdominated by NAs no longer underreport leakage. The audit tibble
gains a new
comparable_ncolumn for interpretability.synthesise_geospatial()now usesoriginal's NA mask as theauthority for cell-level preservation (previously used
synth'smask, which could let synthesised coordinates leak into rows that
the original had missing). Adds a
nrow(synth) == nrow(original)check.
Documentation
roles_validate()error message for the multiple-treatment case isrefreshed: drops the stale "v0.2 / deferred to v0.3" wording and
guides the user to either edit the roles tibble or call
propose_roles(df, detect = FALSE)for byte-stable v0.2.xbehaviour.
mask()'sroxygen removed.
recipe_io.Rdoc and therecipe_anatomyvignette reword theinclude_simulator = TRUEno-op without pinning it to v0.2 / v0.3.roadmapvignette restructured around feature areas. The hardversion pins ("v0.3", "v0.4") are gone -- v0.3 / v0.4 shipped
different features from the prior roadmap, so the pins were stale.
getting_startedvignette: "vignette('roadmap') -- what's plannedfor v0.3+" replaced by "features deliberately deferred from the
current release".
Test suite
test-mask-end-to-end.R,test-mask-roundtrip-integration.R) callpropose_roles(df, detect = FALSE)so the suite is clean againstthe maintainer's local fixtures while the multi-treatment design
decision remains roadmap.
HIGH-leakage warning now wrap with
expect_warning("HIGH leakage")so future warning regressions remain visible.
in
unmask(); fail-closed unknown-level handling inapply_recipe()andunmask();integrity_fpenforcement(positive, negative, and the
check_integrity = FALSEescapehatch);
synthesise_geospatial()NA-mask source authority androw-count check.
masque 0.4.0
Adds first-class geospatial synthesis. One new export, no breaking
changes to the v0.3.0 surface.
New export
synthesise_geospatial(synth, original, anchor_col, lat_col, lon_col, anchor_centroids, site_spread_deg, jitter_deg, seed)--re-anchors the latitude / longitude columns in a masqued data frame
at user-supplied centroids, while preserving (a) the count of
distinct sites per anchor level, (b) the per-site replication
distribution, and (c) within-site tight clustering with
between-site spread. The original positions are never published;
the function reads them only to count distinct sites. NA pattern
in coordinates is preserved cell-by-cell. RNG hygiene via
withr::local_preserve_seed().Motivated by the masque release walkthrough, where state-centroid
within-state clustering of real trial sites.
CRAN and r-universe readiness
cran-comments.mdfor first-submission notes..github/workflows/R-CMD-check.yaml(r-lib standard matrix:Linux release / devel / oldrel-1, macOS release, Windows release).
R CMD check --as-cranreports 0 errors, 0 warnings, 2 NOTEs(new-submission boilerplate and local HTML Tidy environmental).
Documentation
R/synthesise_geospatial.Rcarries the full roxygen doc + a\donttest{}example.masque 0.3.0
Adds automatic experimental-design detection and a sanity-check
visualisation. New public surface: 3 exports, 1 vignette.
New exports
detect_design(df, roles = NULL, interactive = FALSE, threshold = 0.5, tie_delta = 0.02)-- returns an S7design_summarywith the mostlikely design class (
CRD,RCBD,IBD/alpha-lattice,row-column,split-plot,factorial, ornone), per-rule scores,evidence, and a
recommended_rolestibble. Rule engine, not ML.design_summary-- S7 class wrapping the detection result.print()is cli-styled and surfaces top-3 alternates so the usercan see how confident the call was. Slots include
class_label,treatment_col,block_cols,whole_plot_col,sub_plot_col,spatial_cols,scores,evidence,recommended_roles,candidates,warnings.plot_design_summary(x, df, engine = c("base", "ggplot2"))-- alsoregistered as an S7
plot()method. Base-graphics sanity-checkvisualisation dispatched per class: replication tile, spatial
layout, factor-nesting tree, treatment-frequency + NA-pattern.
Behaviour change
propose_roles(df)flips todetect = TRUEby default. Thedetected design's
recommended_rolesare overlaid on the name-basedproposal, promoting structurally-identified treatments and blocks
even when their column names don't match the design / treatment
regexes (e.g.,
genin an alpha-lattice). Thedesign_summaryisstashed as
attr(roles, "design"). Passdetect = FALSEtorecover the v0.2.x byte-stable behaviour.
Design philosophy
mask()synthesis behaviour isunchanged. Only
propose_roles()consumes detection output, andonly as role hints.
returning a score in
[0, 1]with evidence; the orchestrator picksthe top above threshold, breaking ties in favour of the simpler
design (CRD < RCBD < factorial < IBD < row-column < split-plot).
layouts use
desplot::desplot()orggplot2-based packages.Suggests
agridat-- canonical fixtures for tests and the new vignette.ggplot2-- optional plot engine viaengine = "ggplot2"; basegraphics is the default and the fallback.
Limitations
factorial-in-blocks: both have the same data layout. The
whole-plot / sub-plot assignment uses cardinality (fewer levels =
whole-plot), which is heuristic.
detect = FALSEfor toy fixtures.masque 0.2.0
First public release of
masque-- a structurally faithful developmentsurrogate for tabular datasets. Successor to the unreleased
synthPRv0.1.0 (folder-scanning multi-file API), rewritten around a single-file
data-frame-first interface and a round-trippable
recipeobject.masqueis not an anonymisation or differential-privacy tool. It producesdevelopment surrogates suitable for building and debugging pipelines, and
a private
recipethat re-targets a pipeline built against the syntheticclone back onto the original data. See
vignette("confidentiality")forthe threat model.
Design
design,treatment,outcome,covariate,ignore. Multi-outcome supported. Date /POSIX columns and PII-pattern column names default to
ignore.local-- realistic dev surrogate for the data owner. Column namesand level vocabularies preserved. Treatment-level permutation is
opt-in. Issues a load-time warning when the synthetic is extracted.
collaborate-- give the synthetic to a collaborator while keepingthe recipe private. Treatment + categorical-covariate levels are
opaque-aliased (
trt_001,<col>_L01). Numeric draws arejittered within column resolution; integer columns are
stochastically rounded.
ignorecolumns are dropped.audit_mask()runs automatically and warns on HIGH leakage.Public API (11 exports)
propose_roles(df)-- heuristics-driven role tibble; the user editsand passes to
mask().roles_validate(roles, df)-- fail-closed structural + semantic check.mask(df, roles, mode, seed, ...)-- returns an S7masqueobject.synthetic(m)/recipe(m)-- accessors that hide S7.apply_recipe(original, recipe)-- forward translate original-namespacedata into the synthetic namespace.
unmask(x, recipe, column = NULL)-- inverse on a data frame or atomicvector; round-trips a pipeline back to the original.
save_recipe(rec, path, include_simulator = FALSE)/read_recipe(path)-- runtime-minimal.rdspersistence (under 10 KBon a 17,000-row, 38-column MET fixture).
audit_mask(m, original = NULL, print = TRUE)-- first-class leakageaudit returning the per-column severity tibble.
reveal_maps(recipe)-- explicit, banner-fenced unmasked-map reveal(never automatic;
print(recipe)is redacted by default).Synthesis engine
Pearson copula correlation matrix sampled via Gaussian copula.
set and marginal frequencies.
Confidentiality
withr::with_seed/local_preserve_seed);mask()does not mutate the caller's.Random.seed.recipeis runtime-minimal by default -- no copula matrix or rawmarginals stored. SHA-256 NA-mask fingerprint provided as an
integrity check, not a privacy primitive.
print(recipe)redacted by default;reveal_maps()is the onlyunmasked path.
audit_mask()flags retained PII-pattern columns, unaliasedtreatments under collaborate, rare-level leakage, and numeric exact-
match rates above the per-role thresholds.
Documentation
getting_started,confidentiality,recipe_anatomy,roadmap.inst/extdata/john_alpha.csv-- 72-row, 7-column public fixturederived from
agridat::john.alpha(John 1987, alpha design).History
Predecessor
synthPRv0.1.0 (folder-scanning, multi-file) is archivedat
_legacy/synthPR_v0.1.0/in the development workspace and is notdistributed.