diff --git a/databases/README.md b/databases/README.md index 45cf23bb..8abe9732 100644 --- a/databases/README.md +++ b/databases/README.md @@ -12,7 +12,7 @@ GECKO does not include BRENDA data. `fetchBrendaAsset` downloads the files of the [toolbox-assets](https://github.com/CompMetEng/toolbox-assets) release `brenda-2026.1-1` ([BRENDA](https://www.brenda-enzymes.org) 2026.1, CC BY 4.0) on first use into `databases/brenda-2026.1-1`, checked against the manifest checksums. geckopy uses the same release. A project can use its own BRENDA files instead, in `data/brenda`, written by the `geckopy brenda-refresh` CLI of [geckopy](https://github.com/CompMetEng/geckopy). -- `kcat.tsv`: kcat values per organism, reaction and substrate. Both a per-triple maximum (`kcat_max`) and median (`kcat_median`) are recorded. Which one is used is set by `params.kcatAggregation` on the model adapter, or per call with `fuzzyKcatMatching(model,'kcatAggregation','median')`; the default is `max`. +- `kcat.tsv`: kcat values per organism, reaction and substrate. Both a per-triple maximum (`kcat_max`) and median (`kcat_median`) are recorded. Which one is used is set by `aggregation` in `[kcat_sources.brenda]` of `model_adapter.toml` (the preset's: `median` for the default preset, `max` for `gecko_classic`), or per call with `fuzzyKcatMatching(model,'kcatAggregation','median')`. - `sa.tsv`: specific activities per organism, reaction and substrate, with a maximum (`sa_max`) and median (`sa_median`); `loadBRENDAdata` uses `sa_max`. - `mw.tsv`: molecular weights per organism and reaction (without explicitly referring to a protein identifier). - `measurements.tsv`: one row per wild-type kcat measurement with an EC number: EC number, substrate (name and InChIKey), organism, UniProt accession and sequence id, kcat (1/s; the geometric midpoint of a range), temperature, pH and PubMed id. Values more than four orders of magnitude below their EC class, and clusters of values below 0.01 /s that sit three orders of magnitude under the other values of their EC number, are left out as unit errors. `kcatsFromECTree` reads it (see `loadBrendaMeasurements`). diff --git a/src/deprecated/adapterTemplate.m b/src/deprecated/adapterTemplate.m index 9dcba3a2..053c8c0d 100644 --- a/src/deprecated/adapterTemplate.m +++ b/src/deprecated/adapterTemplate.m @@ -48,13 +48,6 @@ % Provide your organism scientific name obj.params.org_name = 'genus species'; - % Which BRENDA aggregate fuzzyKcatMatching reads for each - % (EC, substrate, organism) triple: 'max' (the highest - % turnover reported) or 'median' (the middle one). Can also - % be given per call as - % fuzzyKcatMatching(model,'kcatAggregation','median'). - obj.params.kcatAggregation = 'max'; - % Taxonomic identifier for Complex Portal obj.params.complex.taxonomicID = []; @@ -96,31 +89,6 @@ % this does not interfere with them catalyzing reactions in % different compartments). Typically, cytoplasm is chosen. obj.params.enzyme_comp = 'cytoplasm'; - - %% Hyperparameters for evotune (CMA-ES) kcat fitting - % Default initial uncertainty (standard deviation in log-space) for kcat values - obj.params.evotune.sigma0logDefault = 0.5; - % Data sources for kcat values, ordered from least to most trusted - obj.params.evotune.kcatSources = {'dlkcat','brenda','custom'}; - % Initial uncertainty for each source (lower = more trusted data) - obj.params.evotune.sigma0logSource = [0.4; 0.2; 0.1]; - - % Weight on the max-growth RMSE against the flux RMSE: - % (rmseFlux + w*rmseMaxGrowth) / (w+1). At 1 both count equally. - obj.params.evotune.maxGrowthWeight = 1.0; - % Weight on a prior term in the search objective, - % rmse + w*mean((log(k/k0)/sigma0log)^2). Keeps large corrections - % reproducible across seeds; 0 scores on RMSE alone. - obj.params.evotune.priorPenaltyWeight = 0.03; - % Give isozyme copies of one reaction (sharing a prior value and - % a source) a single shared kcat, so the search cannot invent a - % distinction the kcat assignment never made. - obj.params.evotune.tieIsozymes = true; - - % Stop optimization when RMSE falls below this threshold - obj.params.evotune.rmseThreshold = 0.2; - % Maximum number of CMA-ES generations before termination - obj.params.evotune.maxGenerations = 150; end % function ecModel = makeModelAnaerobic(ecModel) diff --git a/src/download_databases/loadBRENDAdata.m b/src/download_databases/loadBRENDAdata.m index 266f5220..d5753fab 100644 --- a/src/download_databases/loadBRENDAdata.m +++ b/src/download_databases/loadBRENDAdata.m @@ -18,8 +18,8 @@ % a loaded model adapter (default: the current default model adapter). % kcatAggregation : char % which aggregate to read from kcat.tsv and sa.tsv, 'max' or -% 'median' (default: params.kcatAggregation from the model adapter, -% or 'max' if the adapter does not set it). A triple's max is the +% 'median' (default: [kcat_sources.brenda] aggregation as resolved from the preset or the +% stages (median for the default preset, max when no stage sets it)). A triple's max is the % highest turnover reported for it; its median is the middle one. % filterOutlierKcats : logical % whether to drop a row whose kcat is implausibly low for its EC @@ -58,7 +58,7 @@ 'kcatAggregation', [], @(x) assert(isempty(x) || any(strcmpi(x, ... {'max','median'})), ... 'loadBRENDAdata:invalidKcatAggregation', ... - 'kcatAggregation must be ''max'' or ''median''.'); ... + 'aggregation must be max or median'); ... 'filterOutlierKcats', true, @(x) assert((islogical(x) || isnumeric(x)) && ... isscalar(x), ... 'loadBRENDAdata:invalidFilterOutlierKcats', ... diff --git a/src/download_databases/resolveKcatAggregation.m b/src/download_databases/resolveKcatAggregation.m index 9729e812..17db5df1 100644 --- a/src/download_databases/resolveKcatAggregation.m +++ b/src/download_databases/resolveKcatAggregation.m @@ -4,9 +4,11 @@ % % The BRENDA snapshot records both a maximum and a median per % (EC, substrate, organism) triple. Which one a run uses can be given per -% call, or once on the model adapter as params.kcatAggregation; an -% adapter that sets neither gets 'max', which is what GECKO has always -% used. +% call. Without one, it is [kcat_sources.brenda] aggregation of the kcat +% assignment configuration (kcatAssignmentConfig): the preset's, which is +% 'median' for the default preset brenda_ec_tree and 'max' for +% gecko_classic, or the one that model_adapter.toml sets. A configuration +% in which no stage sets an aggregation for brenda gives 'max'. % % Input Arguments % --------------- @@ -25,18 +27,17 @@ % loadBRENDAdata, fuzzyKcatMatching if isempty(kcatAggregation) - params = modelAdapter.getParameters(); - if isfield(params,'kcatAggregation') && ~isempty(params.kcatAggregation) - kcatAggregation = params.kcatAggregation; - else - kcatAggregation = 'max'; + config = kcatAssignmentConfig(modelAdapter.getParameters()); + kcatAggregation = 'max'; + if isfield(config.sources, 'brenda') && isfield(config.sources.brenda, 'aggregation') + kcatAggregation = config.sources.brenda.aggregation; end end if ~(ischar(kcatAggregation) || isstring(kcatAggregation)) || ... ~any(strcmpi(kcatAggregation,{'max','median'})) error('resolveKcatAggregation:invalidKcatAggregation', ... - 'kcatAggregation must be ''max'' or ''median''.') + 'aggregation must be max or median') end kcatAggregation = lower(char(kcatAggregation)); end diff --git a/src/gather_kcats/fuzzyKcatMatching.m b/src/gather_kcats/fuzzyKcatMatching.m index 4f0b0ff5..ba7eefa8 100644 --- a/src/gather_kcats/fuzzyKcatMatching.m +++ b/src/gather_kcats/fuzzyKcatMatching.m @@ -35,6 +35,10 @@ % forceWClvl : double % number of extra levels, from the right, replaced by a wildcard (on top % of any wildcard already in the code; default 0). +% kcatAggregation : char +% which BRENDA aggregate to read, 'max' or 'median' (default: +% [kcat_sources.brenda] aggregation as resolved from the preset or the +% stages (median for the default preset, max when no stage sets it)). % % Returns % ------- @@ -102,7 +106,7 @@ 'kcatAggregation', [], @(x) assert(isempty(x) || any(strcmpi(x, ... {'max','median'})), ... 'fuzzyKcatMatching:invalidKcatAggregation', ... - 'kcatAggregation must be ''max'' or ''median''.')}); + 'aggregation must be max or median')}); ecRxns = p.ecRxns; modelAdapter = p.modelAdapter; forceWClvl = p.forceWClvl; diff --git a/src/gather_kcats/kcatAssignmentConfig.m b/src/gather_kcats/kcatAssignmentConfig.m index 07a49b72..a27371d3 100644 --- a/src/gather_kcats/kcatAssignmentConfig.m +++ b/src/gather_kcats/kcatAssignmentConfig.m @@ -81,19 +81,23 @@ end end % The okp group: the OKP methods of [kcat_sources.okp] methods. -known = {'CataPro','CatPred','CatRange','DLKcat','EITLEM','KinForm-H','KinForm-L', ... - 'OmniESI','OmniESI-O2DENet','TurNup','UniKP'}; -% The default: KinForm-H, as the default of params.okp.method. +% The methods kcatsFromOKP accepts: those that predict from one enzyme and +% one substrate (CatPred and TurNup read whole reactions). +known = {'CataPro','CatRange','DLKcat','EITLEM','KinForm-H','KinForm-L', ... + 'OmniESI','OmniESI-O2DENet','UniKP'}; +% The default: KinForm-H. config.okpMethods = {'KinForm-H'}; if isfield(config.sources, 'okp') && isfield(config.sources.okp, 'methods') && ~isempty(config.sources.okp.methods) config.okpMethods = cellstr(config.sources.okp.methods); config.okpMethods = config.okpMethods(:)'; end -unknown = setdiff(config.okpMethods, known); +unknown = unique(config.okpMethods(~ismember(config.okpMethods, known)), 'stable'); if ~isempty(unknown) error('kcatAssignment:config', 'Unknown OKP methods in [kcat_sources.okp] methods: %s; methods: %s.', ... strjoin(unknown, ', '), strjoin(known, ', ')) end +% The file of a method: okp_ and the lower-case method name, with runs of +% characters other than letters and digits replaced by _. okpFiles = cellfun(@(m) ['okp_' lower(regexprep(regexprep(m, '[^A-Za-z0-9]+', '_'), '^_+|_+$', ''))], ... config.okpMethods, 'UniformOutput', false); for s = 1:numel(config.stages) diff --git a/src/gather_kcats/kcatsFromBrenda.m b/src/gather_kcats/kcatsFromBrenda.m index ec136f57..de65b67e 100644 --- a/src/gather_kcats/kcatsFromBrenda.m +++ b/src/gather_kcats/kcatsFromBrenda.m @@ -16,8 +16,8 @@ % -------------------- % aggregation : char % which BRENDA value per EC number, substrate and organism is used: -% 'max' or 'median' (default: params.kcatAggregation of the model -% adapter, or 'max'). +% 'max' or 'median' (default: [kcat_sources.brenda] aggregation as resolved from the preset or the +% stages (median for the default preset, max when no stage sets it)). % forceWClvl : double % number of extra levels, from the right, replaced by a wildcard in the % EC numbers (on top of any wildcard already in the code; default 0). diff --git a/src/gather_kcats/kcatsFromOKP.m b/src/gather_kcats/kcatsFromOKP.m index 76297dd8..4fbe09e5 100644 --- a/src/gather_kcats/kcatsFromOKP.m +++ b/src/gather_kcats/kcatsFromOKP.m @@ -32,9 +32,11 @@ % Name-Value Arguments % -------------------- % method : char -% OKP method, e.g. 'CataPro', 'CatPred', 'CatRange', 'DLKcat', -% 'EITLEM', 'KinForm-H', 'KinForm-L', 'OmniESI', 'TurNup', 'UniKP' -% (default: params.okp.method of the model adapter, or 'KinForm-H'). +% OKP method that predicts from one enzyme and one substrate: 'CataPro', +% 'CatRange', 'DLKcat', 'EITLEM', 'KinForm-H', 'KinForm-L', 'OmniESI', +% 'OmniESI-O2DENet' or 'UniKP' (default: the first entry of +% [kcat_sources.okp] methods of the model adapter, KinForm-H when it +% lists none). % wait : logical % wait until the job is done (default false). % resubmit : logical @@ -97,17 +99,16 @@ params = modelAdapter.params; method = char(p.method); if isempty(method) - if isfield(params, 'okp') && isfield(params.okp, 'method') && ~isempty(params.okp.method) - method = char(params.okp.method); - else - method = 'KinForm-H'; - end + config = kcatAssignmentConfig(params); + method = config.okpMethods{1}; end if any(strcmp(method, {'CatPred', 'TurNup'})) error('kcatsFromOKP:method', ['%s reads whole reactions (its substrate set, and products ' ... 'for TurNup); kcatsFromOKP submits enzyme-substrate pairs only.'], method) end -tag = lower(regexprep(method, '[^A-Za-z0-9]+', '_')); +% The file name of a method: lower case, runs of characters other than +% letters and digits replaced by _ (the same rule as okp_ sources). +tag = lower(regexprep(regexprep(method, '[^A-Za-z0-9]+', '_'), '^_+|_+$', '')); dataDir = fullfile(params.path, 'data'); jobFile = fullfile(dataDir, ['OKP_job_' tag '.txt']); outFile = fullfile(dataDir, ['OKP_output_' tag '.csv']); @@ -272,8 +273,7 @@ function checkOptions(args, names, functionName) error('ecRxns should be a logical vector with length equal to model.ec.rxns') end % Request options: defaults, then params.okp, then the method of this job. -okp = struct('method','KinForm-H', ... - 'handleLongSequences','truncate', ... +okp = struct('handleLongSequences','truncate', ... 'includeSimilarityColumns',true,'canonicalizeSubstrates',true); if isfield(params,'okp') fn = fieldnames(params.okp); diff --git a/src/gather_kcats/updateKcatSources.m b/src/gather_kcats/updateKcatSources.m index f1f212ad..52ca8ff7 100644 --- a/src/gather_kcats/updateKcatSources.m +++ b/src/gather_kcats/updateKcatSources.m @@ -24,8 +24,13 @@ % [kcat_sources.brenda] % aggregation = "median" % -% [kcat_sources.okp_catapro] -% method = "CataPro" +% [kcat_sources.okp] +% methods = ["CataPro", "KinForm-H"] +% +% The keys that a [kcat_sources.] table can set are aggregation and +% force_wildcard_level for brenda, k for ec_tree, k, min_identity and +% coverage for homologs, and methods for okp; the okp_ files +% have none (their method is the one whose file name they carry). % % Parameters % ---------- @@ -160,7 +165,7 @@ st = 'pending'; nt = 'OKP job for DLKcat still running; call again later'; end else % okp_ - method = okpMethod(name, wanted); + method = okpMethod(name); resolveOkpApiKey('', modelAdapter.params.path); % errors without a key [~, done] = kcatsFromOKP(model, 'method', method, 'saveAs', name, 'modelAdapter', modelAdapter); if ~done @@ -172,20 +177,18 @@ end end -function method = okpMethod(name, wanted) -% The OKP method of okp_: from [kcat_sources.okp_] method, -% or the known method whose name gives this file name. -if isfield(wanted, 'method') - method = char(wanted.method); - return -end -known = {'CataPro','CatPred','CatRange','DLKcat','EITLEM','KinForm-H','KinForm-L', ... - 'OmniESI','OmniESI-O2DENet','TurNup','UniKP'}; -tags = cellfun(@(m) ['okp_' lower(regexprep(m, '[^A-Za-z0-9]+', '_'))], known, 'UniformOutput', false); +function method = okpMethod(name) +% The OKP method of okp_: the known method whose file name is name, +% that is okp_ and the lower-case method name, with runs of characters +% other than letters and digits replaced by _. +known = {'CataPro','CatRange','DLKcat','EITLEM','KinForm-H','KinForm-L', ... + 'OmniESI','OmniESI-O2DENet','UniKP'}; +tags = cellfun(@(m) ['okp_' lower(regexprep(regexprep(m, '[^A-Za-z0-9]+', '_'), '^_+|_+$', ''))], ... + known, 'UniformOutput', false); hit = strcmp(tags, name); if ~any(hit) - error('updateKcatSources:okpMethod', ['Set the OKP method of %s in model_adapter.toml: ' ... - '[kcat_sources.%s] method = "..."'], name, name) + error('updateKcatSources:okpMethod', ['%s is not the file of a known OKP method; ' ... + 'the files are %s.'], name, strjoin(tags, ', ')) end method = known{hit}; end diff --git a/src/kcat_tuning/evotune/cmaesKcatTuning.m b/src/kcat_tuning/evotune/cmaesKcatTuning.m index 9011483f..48596790 100644 --- a/src/kcat_tuning/evotune/cmaesKcatTuning.m +++ b/src/kcat_tuning/evotune/cmaesKcatTuning.m @@ -8,7 +8,8 @@ % rate. % % Configuration comes entirely from modelAdapter.params.evotune: trust -% tiers (kcatSources/sigma0logSource/sigma0logDefault), maxGrowthWeight, +% tiers (kcatSources/sigma0logSource, from [evotune.source_groups], and +% sigma0logDefault), maxGrowthWeight, % priorPenaltyWeight and tieIsozymes control the search, and % maxGenerations/rmseThreshold are its stopping conditions. % diff --git a/src/kcat_tuning/evotune/resolveTunableContext.m b/src/kcat_tuning/evotune/resolveTunableContext.m index 143a39f8..b9b1bda9 100644 --- a/src/kcat_tuning/evotune/resolveTunableContext.m +++ b/src/kcat_tuning/evotune/resolveTunableContext.m @@ -5,22 +5,29 @@ % Shared by screenKcatLeverage and cmaesKcatTuning so a screen and the % tuning run it feeds are always looking at exactly the same parameters. % -% A kcat belongs to the group whose params.kcatSources entry equals the -% origin of its ec.source or its whole ec.source (case-insensitive, see -% parseKcatSource), so +% A kcat belongs to the group of the first params.kcatSources entry that +% matches its source, and is labelled with the name of that group +% (params.kcatGroups, one per entry). Without groups the error is raised: +% No evotune source groups: set [evotune.source_groups] in model_adapter.toml. An entry matches the origin of the kcat's ec.source +% or its whole ec.source (case-insensitive, see parseKcatSource), so % 'brenda (wc=0, origin=1)' is in the group of 'brenda', % 'combined (sd=0.10, n=2)' in the group of 'combined' and % 'brenda (wc=0, origin=1) | tuned=cmaes (prior=2)' in the group of 'brenda', -% with that group's sigma0logSource. When a source matches the entries of -% several groups, the first matching entry in params.kcatSources gives the -% group. +% with that group's sigma0logSource. An entry that ends in * is a prefix +% pattern on the origin ('okp_*' covers okp_eitlem_kinetics and +% okp_kinform_h). Exact entries are tried first, over all entries in order; +% then the patterns, the longest prefix first, and in the order of +% params.kcatSources among equal prefixes. Kcats that match no entry get +% sigma0logDefault; when more than 20 % of the tunable kcats match none, +% a warning (resolveTunableContext:unlabelled) lists up to five origins, +% the most common first and in alphabetical order among equal counts. % % Parameters % ---------- % ecModel : struct % ecModel with a populated ec.kcat. % params : struct -% modelAdapter.params.evotune (kcatSources, sigma0logSource, +% modelAdapter.params.evotune (kcatSources, kcatGroups, sigma0logSource, % sigma0logDefault, tieIsozymes). % tunableMask : logical % same length as ecModel.ec.rxns; [] to consider every kcat > 0. @@ -64,12 +71,34 @@ groups = repmat({'unlabelled'}, numel(tunableIdx), 1); sigma0log = params.sigma0logDefault * ones(numel(tunableIdx),1); assigned = false(numel(tunableIdx),1); -for i = 1:numel(params.kcatSources) - idx = ~assigned & (strcmpi(sourceNames, params.kcatSources{i}) | strcmpi(sources, params.kcatSources{i})); +entries = params.kcatSources(:)'; +if ~isfield(params, 'kcatGroups') || isempty(params.kcatGroups) + error('GECKO:resolveTunableContext', ... + 'No evotune source groups: set [evotune.source_groups] in model_adapter.toml') +end +entryGroup = params.kcatGroups(:)'; +isPattern = endsWith(entries, '*'); +patternLength = cellfun(@numel, entries); +[~, order] = sort(patternLength(isPattern), 'descend'); % stable: ties keep the group order +patternIdx = find(isPattern); +for i = [find(~isPattern), patternIdx(order)] + if isPattern(i) + idx = ~assigned & startsWith(sourceNames, entries{i}(1:end-1), 'IgnoreCase', true); + else + idx = ~assigned & (strcmpi(sourceNames, entries{i}) | strcmpi(sources, entries{i})); + end assigned(idx) = true; - groups(idx) = params.kcatSources(i); + groups(idx) = entryGroup(i); sigma0log(idx) = params.sigma0logSource(i); end +if mean(~assigned) > 0.2 + [names, ~, k] = unique(sourceNames(~assigned)); + [~, byCount] = sort(accumarray(k, 1), 'descend'); + names = names(byCount(1:min(5, end))); + warning('resolveTunableContext:unlabelled', ... + '%d of %d tunable kcats match no evotune source group and get sigma0_log_default (%g); origins: %s', ... + sum(~assigned), numel(assigned), params.sigma0logDefault, strjoin(names, ', ')) +end if isfield(params,'tieIsozymes') && params.tieIsozymes tieMap = isozymeTieMap(ecRxnIdsTunable, kcat0, 'sources', groups); diff --git a/src/kcat_tuning/sensitivityTuning.m b/src/kcat_tuning/sensitivityTuning.m index 4e3c51fa..97b397a2 100644 --- a/src/kcat_tuning/sensitivityTuning.m +++ b/src/kcat_tuning/sensitivityTuning.m @@ -19,7 +19,7 @@ % -------------------- % desiredGrowthRate : double % kcats will be relaxed until this growth rate is reached (default: the -% experimental growth rate params.gR_exp from the model adapter). +% experimental growth rate gr_exp from the model adapter). % modelAdapter : ModelAdapter % a loaded model adapter (default: the current default model adapter). % foldChange : double @@ -82,6 +82,9 @@ end params = modelAdapter.params; if isempty(desiredGrowthRate) + if ~isfield(params, 'gR_exp') || isempty(params.gR_exp) + error('GECKO:grExpNotSet', 'gr_exp is not set in model_adapter.toml') + end desiredGrowthRate = params.gR_exp; end diff --git a/src/kcat_tuning/sigmaFitter.m b/src/kcat_tuning/sigmaFitter.m index 7a253b25..3eedf138 100644 --- a/src/kcat_tuning/sigmaFitter.m +++ b/src/kcat_tuning/sigmaFitter.m @@ -71,7 +71,11 @@ Ptot = modelAdapter.getParameters().Ptot; end if isempty(growthRate) - growthRate = modelAdapter.getParameters().gR_exp; + params = modelAdapter.getParameters(); + if ~isfield(params, 'gR_exp') || isempty(params.gR_exp) + error('GECKO:grExpNotSet', 'gr_exp is not set in model_adapter.toml') + end + growthRate = params.gR_exp; end if ~(growthRate > 0) error('sigmaFitter:growthRate', 'The growth rate to fit sigma to must be positive, not %g.', growthRate) diff --git a/src/limit_proteins/flexibilizeEnzConcs.m b/src/limit_proteins/flexibilizeEnzConcs.m index 3dcbd5fd..5b71cc67 100644 --- a/src/limit_proteins/flexibilizeEnzConcs.m +++ b/src/limit_proteins/flexibilizeEnzConcs.m @@ -133,7 +133,10 @@ end if isempty(expGrowth) - expGrowth = modelAdapter.getParameters().gR_exp; + if ~isfield(params, 'gR_exp') || isempty(params.gR_exp) + error('GECKO:grExpNotSet', 'gr_exp is not set in model_adapter.toml') + end + expGrowth = params.gR_exp; end % If a zero value is defined, not iteration limit will be set diff --git a/src/limit_proteins/relaxEnzConcs.m b/src/limit_proteins/relaxEnzConcs.m index fc620aa5..bcb72fbc 100644 --- a/src/limit_proteins/relaxEnzConcs.m +++ b/src/limit_proteins/relaxEnzConcs.m @@ -25,7 +25,7 @@ % Name-Value Arguments % -------------------- % expGrowth : double -% the growth rate to reach (default: gR_exp of the model adapter). +% the growth rate to reach (default: gr_exp of the model adapter). % bioRxn : char % the biomass reaction (default: bioRxn of the model adapter). % perComplex : logical @@ -108,6 +108,9 @@ end params = modelAdapter.getParameters(); if isempty(expGrowth) + if ~isfield(params, 'gR_exp') || isempty(params.gR_exp) + error('GECKO:grExpNotSet', 'gr_exp is not set in model_adapter.toml') + end expGrowth = params.gR_exp; end if isempty(bioRxn) diff --git a/src/model_adapter/adapterParameters.tsv b/src/model_adapter/adapterParameters.tsv index dd19cd41..6b9f29b7 100644 --- a/src/model_adapter/adapterParameters.tsv +++ b/src/model_adapter/adapterParameters.tsv @@ -5,13 +5,10 @@ org_name org_name Scientific name of the organism sigma sigma 0.5 Average enzyme saturation factor p_tot Ptot 0.5 Total protein content [g protein/gDW] f f 0.5 Fraction of enzymes in the model [g enzyme/g protein] -gr_exp gR_exp 0.41 Growth rate the model should reach when not limited by nutrient uptake [1/h] +gr_exp gR_exp Growth rate the model should reach when not limited by nutrient uptake [1/h]; no default, functions that need it fail when it is not set c_source c_source "" Exchange reaction of the main carbon source bio_rxn bioRxn "" Biomass pseudoreaction enzyme_comp enzyme_comp "cytoplasm" Compartment of the protein pseudometabolites -kcat_aggregate_brenda kcatAggregation "max" BRENDA aggregate read by fuzzyKcatMatching: max or median -kcat_aggregate_candidates "max" Aggregation of several candidate kcats for one reaction (geckopy) -kcat_aggregate_isozymes "mean" Aggregation of isozyme kcats when filling a missing value (geckopy) kegg.id kegg.ID "" KEGG organism code kegg.gene_id kegg.geneID "kegg" Field in the KEGG entry that matches model gene identifiers uniprot.type uniprot.type "taxonomy" Whether uniprot.id is a taxonomy or a proteome identifier @@ -21,15 +18,13 @@ uniprot.reviewed uniprot.reviewed false Use only reviewed (Swiss-Prot) entries complex.taxonomic_id complex.taxonomicID Taxonomy identifier for Complex Portal spontaneous.name_contains spontaneous.name_contains Reactions whose name contains this text are spontaneous spontaneous.file spontaneous.file TSV of spontaneous reactions, relative to the project folder: reaction identifiers, optionally a 0/1 column -evotune.sigma0_log_default evotune.sigma0logDefault 0.5 Prior standard deviation (log) for kcats whose source is in no group -evotune.source_groups evotune.kcatSources Trust tiers: group name to the ec.source values it covers -evotune.sigma0_log_source evotune.sigma0logSource Prior standard deviation (log) per group +evotune.sigma0_log_default evotune.sigma0logDefault 0.5 Prior standard deviation of the natural log of the kcat, for kcats that match no source group +evotune.source_groups evotune.kcatSources Trust tiers, one table per group with sources (ec.source origins, or prefixes ending in *, such as okp_*) and sigma0_log (standard deviation of the natural log of the kcat; the sd of a combined kcat is log10). Default groups: custom 0.1, brenda 0.2, ec_tree 0.3, combined 0.4 evotune.rmse_threshold evotune.rmseThreshold 0.2 Stop when the best RMSE reaches this evotune.max_generations evotune.maxGenerations 150 Maximum number of CMA-ES generations evotune.max_growth_weight evotune.maxGrowthWeight 1.0 Weight of the maximum-growth RMSE against the flux RMSE evotune.prior_penalty_weight evotune.priorPenaltyWeight 0.03 Weight of the prior term in the search objective evotune.tie_isozymes evotune.tieIsozymes true Give isozyme copies of one reaction a single kcat -okp.method okp.method "KinForm-H" OpenKineticsPredictor method okp.handle_long_sequences okp.handleLongSequences "truncate" How OpenKineticsPredictor handles sequences that are too long okp.include_similarity_columns okp.includeSimilarityColumns true Return similarity to the method's training data okp.canonicalize_substrates okp.canonicalizeSubstrates true Canonicalize substrate SMILES before prediction @@ -37,6 +32,6 @@ kcat_assignment.preset Kcat assignment preset: brenda_ec_tree (used when neith kcat_assignment.isozymes true Fill reactions without a kcat from their isozymes after the stages kcat_assignment.standard true Give the remaining reactions the standard kcat kcat_assignment.stage Stages instead of a preset: action (select or combine), sources, criteria and BRENDA limits ([[kcat_assignment.stage]]) -kcat_sources Settings per kcat source file, e.g. [kcat_sources.brenda] aggregation; [kcat_sources.okp] methods lists the OKP methods of the okp source group +kcat_sources Settings per kcat source file: [kcat_sources.brenda] aggregation and force_wildcard_level, [kcat_sources.ec_tree] k, [kcat_sources.homologs] k, min_identity and coverage, [kcat_sources.okp] methods (the OKP methods of the okp source group; default KinForm-H) matlab.make_model_anaerobic matlab.makeModelAnaerobic MATLAB function that constrains the ecModel to anaerobic growth matlab.change_protein_biomass matlab.changeProteinBiomass MATLAB function that changes the protein content of the biomass diff --git a/src/model_adapter/adapterTemplate.toml b/src/model_adapter/adapterTemplate.toml index 5df8d865..761e3d67 100644 --- a/src/model_adapter/adapterTemplate.toml +++ b/src/model_adapter/adapterTemplate.toml @@ -16,9 +16,10 @@ sigma = 0.5 p_tot = 0.5 # Fraction of enzymes in the model [g enzyme/g protein]. f = 0.5 -# Growth rate the model should reach when not limited by nutrient uptake, -# for example the maximum growth rate [1/h]. -gr_exp = 0.41 +# Growth rate the model should reach when not limited by nutrient uptake +# [1/h]. Required by the functions that tune kcats or fit sigma; there is no +# default. +# gr_exp = # Exchange reaction of glucose, or another preferred carbon source. c_source = "r_1714" @@ -29,10 +30,6 @@ bio_rxn = "r_4041" # cytoplasm. enzyme_comp = "cytoplasm" -# Which BRENDA aggregate fuzzyKcatMatching reads for each (EC, substrate, -# organism): "max" (the highest turnover reported) or "median". -kcat_aggregate_brenda = "max" - [kegg] # KEGG organism code, from https://www.genome.jp/kegg/catalog/org_list.html, # for example "sce" for Saccharomyces cerevisiae. Empty: KEGG is not used. @@ -91,21 +88,25 @@ rmse_threshold = 0.2 # Maximum number of CMA-ES generations. max_generations = 150 -# Kcat sources, from least to most trusted, each with its prior width -# (lower is more trusted). -[evotune.source_groups.dlkcat] -sources = ["dlkcat"] - -[evotune.source_groups.brenda] -sources = ["brenda"] - -[evotune.source_groups.custom] -sources = ["custom"] - -[evotune.sigma0_log_source] -dlkcat = 0.4 -brenda = 0.2 -custom = 0.1 +# Trust tiers: group name -> the ec.source labels it covers and its +# sigma0_log (the standard deviation of the natural log of the kcat; lower is +# more trusted). A label ending in * is a prefix pattern ("okp_*" covers every +# OKP method). Groups given in the TOML replace the defaults. +# [evotune.source_groups.custom] +# sources = ["custom"] +# sigma0_log = 0.1 +# +# [evotune.source_groups.brenda] +# sources = ["brenda"] +# sigma0_log = 0.2 +# +# [evotune.source_groups.ec_tree] +# sources = ["ec_tree"] +# sigma0_log = 0.3 +# +# [evotune.source_groups.combined] +# sources = ["combined"] +# sigma0_log = 0.4 # How kcats are assigned from the kcat source files in data/kcats (see # collectKcats and assignKcats): a preset, or [[kcat_assignment.stage]] @@ -128,8 +129,13 @@ preset = "brenda_ec_tree" isozymes = true standard = true -# Settings per kcat source, used by updateKcatSources, for example the OKP -# methods the presets combine (default: KinForm-H): +# Settings per kcat source, used by updateKcatSources: aggregation +# ("max" or "median") and force_wildcard_level for brenda, k for ec_tree, +# k, min_identity and coverage for homologs, and methods for okp, the OKP +# methods the presets combine (default: KinForm-H). The BRENDA aggregation +# of the preset is median, or max for gecko_classic. +# [kcat_sources.brenda] +# aggregation = "median" # [kcat_sources.okp] # methods = ["CataPro", "KinForm-H", "UniKP"] diff --git a/src/model_adapter/convertAdapterToToml.m b/src/model_adapter/convertAdapterToToml.m index 7924a8d1..5a0db450 100644 --- a/src/model_adapter/convertAdapterToToml.m +++ b/src/model_adapter/convertAdapterToToml.m @@ -29,6 +29,10 @@ % report : struct % - file: the file written; % - unmapped: parameters written under [matlab.params]; +% - notCarried: parameters that model_adapter.toml has no key for and +% that are not written: kcatAggregation (set aggregation in +% [kcat_sources.brenda]) and evotune.kcatSources and +% evotune.sigma0logSource (write [evotune.source_groups] tables); % - methods: overridden methods that need carrying over by hand. % % Examples @@ -66,7 +70,7 @@ used = {}; for k = 1:numel(map.tomlKey) field = map.matlabField{k}; - if isempty(field) || any(strcmp(field, {'evotune.kcatSources','evotune.sigma0logSource'})) + if isempty(field) || strcmp(field, 'evotune.kcatSources') continue end [found, value] = getDotted(params, field); @@ -84,21 +88,15 @@ end used{end+1} = 'path'; -% Trust tiers: one group per ec.source value. -if isfield(params,'evotune') && isfield(params.evotune,'kcatSources') - used = [used, {'evotune.kcatSources','evotune.sigma0logSource'}]; - sources = params.evotune.kcatSources; - widths = params.evotune.sigma0logSource; - groups = struct(); - sigma0 = struct(); - for k = 1:numel(sources) - name = matlab.lang.makeValidName(sources{k}); - groups.(name).sources = sources(k); - sigma0.(name) = widths(k); +% Fields that model_adapter.toml has no key for: they are not written. +notCarried = {'kcatAggregation', 'evotune.kcatSources', 'evotune.sigma0logSource'}; +report.notCarried = {}; +for k = 1:numel(notCarried) + if getDotted(params, notCarried{k}) + report.notCarried{end+1} = notCarried{k}; end - data.evotune.source_groups = groups; - data.evotune.sigma0_log_source = sigma0; end +used = [used, notCarried, {'evotune.kcatGroups'}]; % Everything else is kept for GECKO under [matlab.params]. extra = unusedFields(params, '', used); @@ -120,6 +118,9 @@ if ~isempty(report.unmapped) fprintf('Kept for GECKO only, under [matlab.params]: %s\n', strjoin(report.unmapped, ', ')); end +if ~isempty(report.notCarried) + printOrange(sprintf('Not carried over to model_adapter.toml: %s.\n', strjoin(report.notCarried, ', '))); +end if ~isempty(report.methods) printOrange(sprintf(['%s overrides methods that a TOML file cannot hold: %s.\n' ... 'Set [spontaneous] for getSpontaneousReactions, [matlab] function names for\n' ... diff --git a/src/model_adapter/loadAdapterParams.m b/src/model_adapter/loadAdapterParams.m index 2e2eccef..bdccdd37 100644 --- a/src/model_adapter/loadAdapterParams.m +++ b/src/model_adapter/loadAdapterParams.m @@ -6,6 +6,9 @@ % they can be used as ModelAdapter.params. Keys are translated with % adapterParameters.tsv (for instance p_tot becomes Ptot); keys that are % not given get the default listed there, which is geckopy's default. +% A key that is not known is an error, also inside [evotune.source_groups], +% [[kcat_assignment.stage]] and [kcat_sources.]; the keys of a kcat +% source are those that updateKcatSources reads. % % Parameters % ---------- @@ -19,8 +22,11 @@ % - path: the project folder (the folder of the TOML file); % - convGEM and spontaneous.file: absolute paths; % - evotune.kcatSources and evotune.sigma0logSource: built from the -% evotune.source_groups and evotune.sigma0_log_source tables, one -% entry per ec.source value; +% evotune.source_groups tables, one entry per element of sources +% (a prefix pattern keeps its trailing *), in group order, with the +% sigma0_log of its group, and evotune.kcatGroups the group name of +% each entry. Without [evotune.source_groups] the +% groups are custom 0.1, brenda 0.2, ec_tree 0.3 and combined 0.4; % - fields under [matlab.params] are copied as they are, for % parameters that only GECKO uses. % @@ -45,14 +51,15 @@ map = adapterParameterMap(); % Unknown keys are errors, as in geckopy, so that a typo is not ignored. -known = [map.tomlKey; {'evotune.source_groups'; 'evotune.sigma0_log_source'}]; +known = map.tomlKey; checkKnownKeys(data, '', known, tomlFile); +checkNested(data, tomlFile); params = struct(); params.path = projectPath; for k = 1:numel(map.tomlKey) key = map.tomlKey{k}; - if any(strcmp(key, {'evotune.source_groups', 'evotune.sigma0_log_source'})) + if strcmp(key, 'evotune.source_groups') continue end [found, value] = getDotted(data, key); @@ -83,9 +90,10 @@ params.spontaneous.file = projectFile(projectPath, params.spontaneous.file); end -params = sourceGroups(params, data, tomlFile); +params = sourceGroups(params, data); if isfield(data, 'matlab') && isfield(data.matlab, 'params') + checkMatlabParams(data.matlab.params, tomlFile); params = mergeStruct(params, data.matlab.params); end end @@ -139,57 +147,162 @@ function checkKnownKeys(s, prefix, known, tomlFile) end continue end - if any(startsWith(key, {'evotune.source_groups.', 'evotune.sigma0_log_source.'})) + unknownKey(key, tomlFile); +end +end + +function checkMatlabParams(extra, tomlFile) +% [matlab.params] is for parameters that only GECKO uses; the settings that +% have a key of their own are not set there. +fields = fieldnames(extra); +for k = 1:numel(fields) + if strcmp(fields{k}, 'kcatAggregation') + where = '[kcat_sources.brenda] aggregation'; + elseif strcmp(fields{k}, 'evotune') + where = '[evotune]'; + else continue end - error('GECKO:loadAdapterParams', ['Unknown key "%s" in %s. Parameters that only ' ... - 'GECKO uses go under [matlab.params].'], key, tomlFile); + error('GECKO:loadAdapterParams', '%s: matlab.params.%s is not allowed; set it in %s', ... + tomlFile, fields{k}, where); end end -function params = sourceGroups(params, data, tomlFile) -% evotune.source_groups and evotune.sigma0_log_source become the -% kcatSources and sigma0logSource lists that evotune reads: one entry per -% ec.source value, with its group's prior width. -groups = struct(); -widths = struct(); +function unknownKey(key, tomlFile) +error('GECKO:loadAdapterParams', '%s: unknown key %s in model_adapter.toml', tomlFile, key); +end + +function checkNested(data, tomlFile) +% Keys inside the tables that adapterParameters.tsv lists as one key. if isfield(data, 'evotune') && isfield(data.evotune, 'source_groups') groups = data.evotune.source_groups; - if isfield(data.evotune, 'sigma0_log_source') - widths = data.evotune.sigma0_log_source; + names = fieldnames(groups); + for k = 1:numel(names) + g = groups.(names{k}); + prefix = ['evotune.source_groups.' names{k}]; + if ~isstruct(g) + error('GECKO:loadAdapterParams', ... + '%s: %s must be a table with sources and sigma0_log', tomlFile, prefix); + end + checkKeys(g, {'sources', 'sigma0_log'}, prefix, tomlFile); + if ~isfield(g, 'sources') || ~isfield(g, 'sigma0_log') + error('GECKO:loadAdapterParams', ... + '%s: %s needs both sources and sigma0_log', tomlFile, prefix); + end + if ~(iscell(g.sources) && ~isempty(g.sources) && all(cellfun(@ischar, g.sources))) + error('GECKO:loadAdapterParams', ... + '%s: %s.sources must be a non-empty list of strings', tomlFile, prefix); + end + w = g.sigma0_log; + if ~(isnumeric(w) && isscalar(w) && isfinite(w) && w > 0) + error('GECKO:loadAdapterParams', ... + '%s: %s.sigma0_log must be a positive number', tomlFile, prefix); + end + end +end +if isfield(data, 'kcat_assignment') && isfield(data.kcat_assignment, 'stage') + stages = data.kcat_assignment.stage; + for k = 1:numel(stages) + checkKeys(stages(k), {'action', 'sources', 'criteria', 'top_origin_limit', ... + 'bottom_origin_limit', 'wildcard_limit'}, 'kcat_assignment.stage', tomlFile); + end +end +if isfield(data, 'kcat_sources') && isstruct(data.kcat_sources) + % The sources that GECKO knows and the keys that updateKcatSources + % reads for each; dlkcat, custom and the okp_ files take none. + allowed = struct('brenda', {{'aggregation', 'force_wildcard_level'}}, ... + 'ec_tree', {{'k'}}, 'homologs', {{'k', 'min_identity', 'coverage'}}, ... + 'okp', {{'methods'}}, 'dlkcat', {{}}, 'custom', {{}}); + names = fieldnames(data.kcat_sources); + for k = 1:numel(names) + if isfield(allowed, names{k}) + keys = allowed.(names{k}); + elseif any(strcmp(names{k}, okpFileNames())) + keys = {}; + else + unknownKey(['kcat_sources.' names{k}], tomlFile); + end + source = data.kcat_sources.(names{k}); + if ~isstruct(source) + error('GECKO:loadAdapterParams', '%s: kcat_sources.%s must be a table', tomlFile, names{k}); + end + checkKeys(source, keys, ['kcat_sources.' names{k}], tomlFile); + if strcmp(names{k}, 'brenda') && isfield(source, 'aggregation') && ... + ~(ischar(source.aggregation) && any(strcmp(source.aggregation, {'max', 'median'}))) + error('GECKO:loadAdapterParams', ... + '%s: kcat_sources.brenda.aggregation must be max or median', tomlFile); + end + if strcmp(names{k}, 'okp') && isfield(source, 'methods') + checkOkpMethods(source.methods, tomlFile); + end + end +end +end + +function checkOkpMethods(methods, tomlFile) +% [kcat_sources.okp] methods: a list of the OKP methods that kcatsFromOKP accepts. +if ~(iscell(methods) && all(cellfun(@ischar, methods))) + error('GECKO:loadAdapterParams', '%s: kcat_sources.okp.methods must be a list of strings', tomlFile); +end +known = okpMethodNames(); +unknown = unique(methods(~ismember(methods, known)), 'stable'); +if ~isempty(unknown) + error('kcatAssignment:config', 'Unknown OKP methods in [kcat_sources.okp] methods: %s; methods: %s.', ... + strjoin(unknown, ', '), strjoin(known, ', ')) +end +end + +function known = okpMethodNames() +% The methods that predict from one enzyme and one substrate (CatPred and +% TurNup read whole reactions). +known = {'CataPro','CatRange','DLKcat','EITLEM','KinForm-H','KinForm-L', ... + 'OmniESI','OmniESI-O2DENet','UniKP'}; +end + +function files = okpFileNames() +% The files of the OKP methods that kcatsFromOKP accepts: okp_ and the +% lower-case method name, with runs of characters other than letters and +% digits replaced by _. +known = okpMethodNames(); +files = cellfun(@(m) ['okp_' lower(regexprep(regexprep(m, '[^A-Za-z0-9]+', '_'), '^_+|_+$', ''))], ... + known, 'UniformOutput', false); +end + +function checkKeys(s, allowed, prefix, tomlFile) +fields = fieldnames(s); +for k = 1:numel(fields) + if ~any(strcmp(fields{k}, allowed)) + unknownKey([prefix '.' fields{k}], tomlFile); end +end +end + +function params = sourceGroups(params, data) +% evotune.source_groups becomes the kcatSources and sigma0logSource lists +% that evotune reads: one entry per element of sources, in group order, +% with the sigma0_log of its group. +groups = struct(); +if isfield(data, 'evotune') && isfield(data.evotune, 'source_groups') + groups = data.evotune.source_groups; else - % geckopy's defaults (EvotuneParams.source_groups, sigma0_log_source). - groups.dlkcat.sources = {'dlkcat'}; - groups.brenda.sources = {'brenda'}; - groups.custom.sources = {'custom'}; - widths.dlkcat = 0.4; - widths.brenda = 0.2; - widths.custom = 0.1; + defaults = {'custom', 0.1; 'brenda', 0.2; 'ec_tree', 0.3; 'combined', 0.4}; + for k = 1:size(defaults, 1) + groups.(defaults{k,1}).sources = defaults(k,1); + groups.(defaults{k,1}).sigma0_log = defaults{k,2}; + end end names = fieldnames(groups); -if ~isempty(setxor(names, fieldnames(widths))) - error('GECKO:loadAdapterParams', ['%s: evotune.sigma0_log_source must have ' ... - 'the same group names as evotune.source_groups'], tomlFile); -end kcatSources = {}; +kcatGroups = {}; sigma0logSource = []; for k = 1:numel(names) - group = groups.(names{k}); - sources = {}; - if isfield(group, 'sources') - sources = group.sources; - if ischar(sources) - sources = {sources}; - end - end - if isfield(group, 'match_okp') && group.match_okp && isfield(params, 'okp') - sources{end+1} = params.okp.method; %#ok - end + sources = groups.(names{k}).sources(:)'; kcatSources = [kcatSources, sources]; %#ok - sigma0logSource = [sigma0logSource; repmat(widths.(names{k}), numel(sources), 1)]; %#ok + kcatGroups = [kcatGroups, repmat(names(k), 1, numel(sources))]; %#ok + sigma0logSource = [sigma0logSource; repmat(groups.(names{k}).sigma0_log, numel(sources), 1)]; %#ok end params.evotune.kcatSources = kcatSources; +params.evotune.kcatGroups = kcatGroups; params.evotune.sigma0logSource = sigma0logSource; end diff --git a/src/model_adapter/showAdapterParams.m b/src/model_adapter/showAdapterParams.m new file mode 100644 index 00000000..06f65040 --- /dev/null +++ b/src/model_adapter/showAdapterParams.m @@ -0,0 +1,226 @@ +function T = showAdapterParams(modelAdapter) +% showAdapterParams Print the parameters of a model adapter and where each comes from. +% +% Prints one line per parameter of adapterParameters.tsv, in that order: +% +% = [] +% +% The origin is file (set in model_adapter.toml, or by an adapter class), +% default (the default of adapterParameters.tsv or of the code), preset +% (given by the kcat assignment preset) or not set (no value; the +% line is " = [not set]"). Strings are shown bare, booleans as true or +% false, lists comma-joined, integer-valued numbers whole and other +% numbers with %g. The resolved kcat +% assignment is listed as kcat_assignment.stage[] lines (the stages of the +% preset or the file, with the settings that the file sets or that differ +% from the defaults criteria max, top_origin_limit 4, bottom_origin_limit 6 +% and wildcard_limit 3), the settings of each kcat source as +% kcat_sources.. lines and the evotune source groups, in group +% order, as evotune.source_groups. lines with their sigma0_log. +% geckopy's "geckopy adapter show" prints the same lines. +% +% Parameters +% ---------- +% modelAdapter : ModelAdapter, optional +% a loaded model adapter (default: the current default model adapter). +% +% Returns +% ------- +% T : table, optional +% the listing, with the columns key, value and origin. Without an +% output argument the lines are printed. +% +% Examples +% -------- +% showAdapterParams(ModelAdapterManager.getDefault()) +% +% See also +% -------- +% loadAdapterParams, kcatAssignmentConfig + +if nargin < 1 || isempty(modelAdapter) + modelAdapter = ModelAdapterManager.getDefault(); + if isempty(modelAdapter) + error('Either send in a modelAdapter or set the default model adapter in the ModelAdapterManager.') + end +end +params = modelAdapter.getParameters(); +tomlFile = fullfile(params.path, 'model_adapter.toml'); +hasFile = isfile(tomlFile); +data = struct(); +if hasFile + data = readAdapterToml(tomlFile); +end +config = kcatAssignmentConfig(params); + +file = fullfile(fileparts(mfilename('fullpath')), 'adapterParameters.tsv'); +fid = fopen(file, 'r'); +cols = textscan(fid, '%s %s %s %s', 'Delimiter', '\t', 'HeaderLines', 1, 'Whitespace', ''); +fclose(fid); + +key = {}; +value = {}; +origin = {}; + function add(k, v, o) + key{end+1, 1} = k; + value{end+1, 1} = v; + origin{end+1, 1} = o; + end + +fromPreset = 'file'; +if ~isempty(config.preset) + fromPreset = ['preset ' config.preset]; +end +for k = 1:numel(cols{1}) + tomlKey = cols{1}{k}; + field = cols{2}{k}; + if isempty(field) + field = tomlKey; + end + switch tomlKey + case 'evotune.source_groups' + if ~isfield(params, 'evotune') || ~isfield(params.evotune, 'kcatGroups') + add(tomlKey, '', 'not set'); + continue + end + entries = params.evotune.kcatSources; + groups = params.evotune.kcatGroups; + o = originOf(data, tomlKey, hasFile); + [names, first] = unique(groups, 'stable'); + for j = 1:numel(names) + in = strcmp(groups, names{j}); + add(['evotune.source_groups.' names{j}], sprintf('%s; sigma0_log %s', ... + strjoin(entries(in), ', '), formatValue(params.evotune.sigma0logSource(first(j)))), o); + end + case 'kcat_assignment.preset' + if isempty(config.preset) + add(tomlKey, 'none (stages are given)', 'file'); + else + add(tomlKey, config.preset, originOf(data, tomlKey, hasFile)); + end + case 'kcat_assignment.stage' + given = struct(); + if isempty(config.preset) + given = params.kcat_assignment.stage; + end + % A stage shows the settings that the file sets, or, for a preset, + % those that are not the default. + limits = {'criteria', 'criteria', 'max'; 'top_origin_limit', 'topOriginLimit', 4; ... + 'bottom_origin_limit', 'bottomOriginLimit', 6; 'wildcard_limit', 'wildcardLimit', 3}; + for j = 1:numel(config.stages) + st = config.stages(j); + text = sprintf('%s: %s', st.action, strjoin(st.sources, ', ')); + for i = 1:size(limits, 1) + if isempty(config.preset) + shown = isfield(given(j), limits{i,1}) && ~isempty(given(j).(limits{i,1})); + else + shown = ~isequal(st.(limits{i,2}), limits{i,3}); + end + if shown + text = [text '; ' limits{i,1} '=' formatValue(st.(limits{i,2}))]; %#ok + end + end + add(sprintf('%s[%d]', tomlKey, j), text, fromPreset); + end + case 'kcat_sources' + sources = config.sources; + if ~(isfield(sources, 'okp') && isfield(sources.okp, 'methods')) + sources.okp.methods = config.okpMethods; + end + % brenda, ec_tree, homologs and okp first, other sources and the + % keys of each source alphabetically + names = fieldnames(sources); + first = {'brenda', 'ec_tree', 'homologs', 'okp'}; + names = [first(ismember(first, names)), sort(names(~ismember(names, first)))']; + for j = 1:numel(names) + keys = sort(fieldnames(sources.(names{j}))); + for i = 1:numel(keys) + given = isfield(params, 'kcat_sources') && isfield(params.kcat_sources, names{j}) ... + && isfield(params.kcat_sources.(names{j}), keys{i}); + o = fromPreset; + if given + o = 'file'; + elseif strcmp(names{j}, 'okp') && strcmp(keys{i}, 'methods') + o = 'default'; + end + add(sprintf('%s.%s.%s', tomlKey, names{j}, keys{i}), ... + formatValue(sources.(names{j}).(keys{i})), o); + end + end + otherwise + found = false; + v = params; + for part = strsplit(field, '.') + if isstruct(v) && isfield(v, part{1}) + v = v.(part{1}); + found = true; + else + found = false; + break + end + end + if ~found || (isnumeric(v) && isempty(v)) + add(tomlKey, '', 'not set'); + continue + end + o = originOf(data, tomlKey, hasFile); + add(tomlKey, formatValue(v), o); + end +end + +T = table(key, value, origin, 'VariableNames', {'key', 'value', 'origin'}); +if nargout == 0 + for k = 1:height(T) + if strcmp(origin{k}, 'not set') + fprintf('%s = [not set]\n', key{k}); + else + fprintf('%s = %s [%s]\n', key{k}, value{k}, origin{k}); + end + end + clear T +end +end + +function o = originOf(data, tomlKey, hasFile) +% file when the TOML sets the key, default otherwise, file for an adapter +% class without a TOML file (it sets the value). +inFile = true; +w = data; +for part = strsplit(tomlKey, '.') + inFile = isstruct(w) && isfield(w, part{1}); + if ~inFile + break + end + w = w.(part{1}); +end +if inFile || ~hasFile + o = 'file'; +else + o = 'default'; +end +end + +function text = formatValue(v) +if ischar(v) || isstring(v) + text = char(v); +elseif islogical(v) + text = char(string(v)); +elseif isnumeric(v) + % integer-valued numbers whole, others with %g + text = strjoin(arrayfun(@(x) numberText(x), v(:)', 'UniformOutput', false), ', '); +elseif iscell(v) + text = strjoin(cellfun(@formatValue, v(:)', 'UniformOutput', false), ', '); +elseif isstruct(v) + text = sprintf('struct with fields %s', strjoin(fieldnames(v), ', ')); +else + text = class(v); +end +end + +function text = numberText(x) +if isfinite(x) && x == round(x) + text = sprintf('%.0f', x); +else + text = sprintf('%g', x); +end +end diff --git a/test/unit_tests/adapterTomlTests.m b/test/unit_tests/adapterTomlTests.m index 020fda6c..683508bd 100644 --- a/test/unit_tests/adapterTomlTests.m +++ b/test/unit_tests/adapterTomlTests.m @@ -24,7 +24,7 @@ function testReadAdapterToml_tc0061(testCase) '', ... '[evotune.source_groups.okp]', ... 'sources = ["OKP"]', ... - 'match_okp = true', ... + 'sigma0_log = 0.25', ... '', ... '[[stage]]', ... 'action = "select"', ... @@ -42,7 +42,7 @@ function testReadAdapterToml_tc0061(testCase) verifyEqual(testCase, d.empty, {}) verifyEqual(testCase, d.kegg.id, 'sce') verifyEqual(testCase, d.evotune.source_groups.okp.sources, {'OKP'}) - verifyTrue(testCase, d.evotune.source_groups.okp.match_okp) + verifyEqual(testCase, d.evotune.source_groups.okp.sigma0_log, 0.25) verifyEqual(testCase, numel(d.stage), 2) verifyEqual(testCase, d.stage(2).action, 'fill') verifyEqual(testCase, d.stage(2).sources, {'isozymes'}) @@ -87,9 +87,12 @@ function testLoadAdapterParamsDefaults_tc0064(testCase) verifyEqual(testCase, params.sigma, 0.5) verifyEqual(testCase, params.kegg.ID, '') verifyEqual(testCase, params.uniprot.ID, '') - verifyEqual(testCase, params.kcatAggregation, 'max') - verifyEqual(testCase, params.evotune.kcatSources, {'dlkcat','brenda','custom'}) - verifyEqual(testCase, params.evotune.sigma0logSource, [0.4; 0.2; 0.1]) + verifyFalse(testCase, isfield(params, 'kcatAggregation')) + verifyFalse(testCase, isfield(params, 'gR_exp')) + verifyFalse(testCase, isfield(params.okp, 'method')) + verifyEqual(testCase, params.evotune.kcatSources, {'custom','brenda','ec_tree','combined'}) + verifyEqual(testCase, params.evotune.sigma0logSource, [0.1; 0.2; 0.3; 0.4]) + verifyEqual(testCase, params.evotune.kcatGroups, params.evotune.kcatSources) verifyEqual(testCase, params.evotune.maxGenerations, 150) verifyEqual(testCase, params.okp.handleLongSequences, 'truncate') verifyEmpty(testCase, params.complex.taxonomicID) @@ -99,18 +102,17 @@ function testLoadAdapterParamsDefaults_tc0064(testCase) end function testLoadAdapterParamsSourceGroups_tc0065(testCase) - % Trust tiers with several sources and the OKP method. + % Trust tiers with several sources, a prefix pattern and a group width. folder = tempname; mkdir(folder); writeTemp({'conv_gem = "x.xml"', 'org_name = "x"', ... - '[okp]', 'method = "EITLEM"', ... - '[evotune.source_groups.okp]', 'sources = ["OpenKineticsPredictor"]', 'match_okp = true', ... - '[evotune.source_groups.database]', 'sources = ["brenda", "okp_brenda"]', ... - '[evotune.sigma0_log_source]', 'okp = 0.25', 'database = 0.2', ... + '[evotune.source_groups.okp]', 'sources = ["okp_*"]', 'sigma0_log = 0.25', ... + '[evotune.source_groups.database]', 'sources = ["brenda", "okp_brenda"]', 'sigma0_log = 0.2', ... '[matlab]', 'make_model_anaerobic = "anaerobicModel_GECKO"', ... '[matlab.params]', 'c_UptakeExp = 0.64'}, fullfile(folder, 'model_adapter.toml')); params = loadAdapterParams(fullfile(folder, 'model_adapter.toml')); - verifyEqual(testCase, params.evotune.kcatSources, {'OpenKineticsPredictor','EITLEM','brenda','okp_brenda'}) - verifyEqual(testCase, params.evotune.sigma0logSource, [0.25; 0.25; 0.2; 0.2]) + verifyEqual(testCase, params.evotune.kcatSources, {'okp_*','brenda','okp_brenda'}) + verifyEqual(testCase, params.evotune.kcatGroups, {'okp','database','database'}) + verifyEqual(testCase, params.evotune.sigma0logSource, [0.25; 0.2; 0.2]) verifyEqual(testCase, params.matlab.makeModelAnaerobic, 'anaerobicModel_GECKO') verifyEqual(testCase, params.c_UptakeExp, 0.64) end @@ -131,6 +133,15 @@ function testConvertAdapterToToml_tc0066(testCase) verifyTrue(testCase, any(strcmp(report.methods, 'getSpontaneousReactions'))) converted = TomlAdapter(file); expected = original.params; + % Fields without a key in model_adapter.toml are not carried over. + if k == 2 + verifyEqual(testCase, report.notCarried, ... + {'kcatAggregation', 'evotune.kcatSources', 'evotune.sigma0logSource'}) + expected = rmfield(expected, 'kcatAggregation'); + expected.evotune = rmfield(expected.evotune, {'kcatSources', 'sigma0logSource'}); + else + verifyEmpty(testCase, report.notCarried) + end actual = converted.params; % The project folder is where the TOML file is; the model path is % relative to it. @@ -144,8 +155,9 @@ function testConvertAdapterToToml_tc0066(testCase) end function lines = legacyAdapterClass() -% An adapter class as written for GECKO 3 and 4: evotune trust tiers and a -% parameter that geckopy does not know. +% An adapter class as written for GECKO 3 and 4: fields that +% model_adapter.toml has no key for (kcatAggregation, evotune trust tiers) +% and a parameter that geckopy does not know. lines = { ... 'classdef LegacyFixtureAdapter < ModelAdapter', ... ' methods', ... @@ -212,10 +224,423 @@ function testCreateGECKOprojectToml_tc0068(testCase) verifyEqual(testCase, strtrim(fileread(fullfile(folder, '.gitignore'))), 'data/okpApiKey.txt') adapter = ModelAdapterManager.getAdapter(folder); verifyEqual(testCase, adapter.params.org_name, 'genus species') - verifyEqual(testCase, adapter.params.evotune.kcatSources, {'dlkcat','brenda','custom'}) + verifyEqual(testCase, adapter.params.evotune.kcatSources, {'custom','brenda','ec_tree','combined'}) + verifyEqual(testCase, adapter.params.evotune.sigma0logSource, [0.1; 0.2; 0.3; 0.4]) + verifyFalse(testCase, isfield(adapter.params, 'gR_exp')) verifyEqual(testCase, adapter.params.spontaneous.name_contains, 'spontaneous') end +function testUnknownKeys_tc0732(testCase) + % An unknown key is an error in every table, naming the file and the key. + base = {'conv_gem = "x.xml"', 'org_name = "x"'}; + cases = { ... + {'sigmaa = 1'}, 'sigmaa'; ... + {'[okp]', 'method = "EITLEM"'}, 'okp.method'; ... + {'[evotune.source_groups.a]', 'sources = ["a"]', 'sigma0_log = 0.1', 'match_okp = true'}, ... + 'evotune.source_groups.a.match_okp'; ... + {'[evotune.sigma0_log_source]', 'a = 0.1'}, 'evotune.sigma0_log_source'; ... + {'[[kcat_assignment.stage]]', 'sources = ["custom"]', 'limit = 2'}, 'kcat_assignment.stage.limit'; ... + {'[kcat_sources.brenda]', 'aggregate = "max"'}, 'kcat_sources.brenda.aggregate'; ... + {'[kcat_sources.ec_tree]', 'min_identity = 40'}, 'kcat_sources.ec_tree.min_identity'; ... + {'[kcat_sources.okp_kinform_h]', 'method = "KinForm-H"'}, 'kcat_sources.okp_kinform_h.method'; ... + {'[kcat_sources.okp]', 'method = "KinForm-H"'}, 'kcat_sources.okp.method'; ... + {'[kcat_sources.foo]'}, 'kcat_sources.foo'; ... + {'[kcat_sources.okp_nosuchmethod]'}, 'kcat_sources.okp_nosuchmethod'; ... + {'[kcat_sources.dlkcat]', 'k = 1'}, 'kcat_sources.dlkcat.k'}; + for k = 1:size(cases, 1) + folder = tempname; mkdir(folder); + file = fullfile(folder, 'model_adapter.toml'); + writeTemp([base, cases{k,1}], file); + try + loadAdapterParams(file); + err = []; + catch err + end + verifyNotEmpty(testCase, err, cases{k,2}) + if ~isempty(err) + verifyEqual(testCase, err.message, ... + sprintf('%s: unknown key %s in model_adapter.toml', file, cases{k,2})) + end + end + % The keys that updateKcatSources reads are known, and an empty table of + % a known source loads. + folder = tempname; mkdir(folder); + writeTemp([base, {'[kcat_sources.dlkcat]', '[kcat_sources.okp_unikp]', '[kcat_sources.custom]', ... + '[kcat_sources.brenda]', 'aggregation = "max"', 'force_wildcard_level = 1', ... + '[kcat_sources.homologs]', 'k = 3', 'min_identity = 40', 'coverage = 0.5', ... + '[kcat_sources.okp]', 'methods = ["UniKP"]'}], fullfile(folder, 'model_adapter.toml')); + params = loadAdapterParams(folder); + verifyEqual(testCase, params.kcat_sources.brenda.aggregation, 'max') +end + +function testSourceGroupValidation_tc0733(testCase) + % Each fault of an evotune source group has its own message. + base = {'conv_gem = "x.xml"', 'org_name = "x"'}; + cases = { ... + {'[evotune.source_groups]', 'a = 0.1'}, 'evotune.source_groups.a must be a table with sources and sigma0_log'; ... + {'[evotune.source_groups.a]', 'sources = ["a"]'}, 'evotune.source_groups.a needs both sources and sigma0_log'; ... + {'[evotune.source_groups.a]', 'sigma0_log = 0.1'}, 'evotune.source_groups.a needs both sources and sigma0_log'; ... + {'[evotune.source_groups.a]', 'sources = "a"', 'sigma0_log = 0.1'}, 'evotune.source_groups.a.sources must be a non-empty list of strings'; ... + {'[evotune.source_groups.a]', 'sources = [1, 2]', 'sigma0_log = 0.1'}, 'evotune.source_groups.a.sources must be a non-empty list of strings'; ... + {'[evotune.source_groups.a]', 'sources = []', 'sigma0_log = 0.1'}, 'evotune.source_groups.a.sources must be a non-empty list of strings'; ... + {'[evotune.source_groups.a]', 'sources = ["a"]', 'sigma0_log = -0.1'}, 'evotune.source_groups.a.sigma0_log must be a positive number'; ... + {'[evotune.source_groups.a]', 'sources = ["a"]', 'sigma0_log = 0'}, 'evotune.source_groups.a.sigma0_log must be a positive number'; ... + {'[evotune.source_groups.a]', 'sources = ["a"]', 'sigma0_log = "wide"'}, 'evotune.source_groups.a.sigma0_log must be a positive number'; ... + {'[evotune.source_groups.a]', 'sources = ["a"]', 'sigma0_log = [0.1, 0.2]'}, 'evotune.source_groups.a.sigma0_log must be a positive number'}; + for k = 1:size(cases, 1) + folder = tempname; mkdir(folder); + file = fullfile(folder, 'model_adapter.toml'); + writeTemp([base, cases{k,1}], file); + try + loadAdapterParams(file); + err = []; + catch err + end + verifyNotEmpty(testCase, err, cases{k,2}) + if ~isempty(err) + verifyEqual(testCase, err.message, [file ': ' cases{k,2}]) + end + end +end + +function testResolveTunableContextPatterns_tc0734(testCase) + % Exact entries first, then prefix patterns with the longest prefix + % first, and the first group among equal prefixes. + model.ec.rxns = {'R1'; 'R2'; 'R3'; 'R4'; 'R5'}; + model.ec.kcat = ones(5, 1); + model.ec.source = {'okp_kinform_h'; 'okp_eitlem_kinetics'; 'OKP_other'; 'brenda'; 'okp_unknown (n=2)'}; + params.kcatSources = {'okp_*', 'okp_k*', 'okp_*', 'okp_eitlem_kinetics', 'brenda'}; + params.kcatGroups = {'okp', 'kinform', 'okp', 'eitlem', 'brenda'}; + params.sigma0logSource = [0.1; 0.2; 0.1; 0.4; 0.5]; + params.sigma0logDefault = 1; + params.tieIsozymes = false; + [~, ~, ~, groups, sigma0log] = resolveTunableContext(model, params, []); + verifyEqual(testCase, groups, {'kinform'; 'eitlem'; 'okp'; 'brenda'; 'okp'}) + verifyEqual(testCase, sigma0log, [0.2; 0.4; 0.1; 0.5; 0.1]) +end + +function testResolveTunableContextDefaultGroups_tc0741(testCase) + % The default groups label kcats by group name, end to end from a TOML + % without [evotune.source_groups]; the label standard is in no group. + folder = tempname; mkdir(folder); + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"'}, fullfile(folder, 'model_adapter.toml')); + params = loadAdapterParams(folder); + model.ec.rxns = {'R1'; 'R2'; 'R3'; 'R4'; 'R5'}; + model.ec.kcat = ones(5, 1); + model.ec.source = {'combined (sd=0.10, n=2)'; 'ec_tree'; 'brenda (wc=0, origin=1)'; 'custom'; 'standard'}; + params.evotune.tieIsozymes = false; + [~, ~, ~, groups, sigma0log] = resolveTunableContext(model, params.evotune, []); + verifyEqual(testCase, groups, {'combined'; 'ec_tree'; 'brenda'; 'custom'; 'unlabelled'}) + verifyEqual(testCase, sigma0log, [0.4; 0.3; 0.2; 0.1; 0.5]) +end + +function testResolveTunableContextUnlabelledOrder_tc0742(testCase) + % The origins in the warning: most common first, then alphabetical. + model.ec.rxns = {'R1'; 'R2'; 'R3'; 'R4'; 'R5'}; + model.ec.kcat = ones(5, 1); + model.ec.source = {'brenda'; 'zeta'; 'beta'; 'alpha'; 'beta'}; + params.kcatSources = {'brenda'}; + params.kcatGroups = {'brenda'}; + params.sigma0logSource = 0.2; + params.sigma0logDefault = 0.5; + params.tieIsozymes = false; + lastwarn(''); + verifyWarning(testCase, @() resolveTunableContext(model, params, []), 'resolveTunableContext:unlabelled') + verifyEqual(testCase, lastwarn, ['4 of 5 tunable kcats match no evotune source group and get ' ... + 'sigma0_log_default (0.5); origins: beta, alpha, zeta']) +end + +function testResolveTunableContextUnlabelled_tc0735(testCase) + % A warning when more than 20 % of the tunable kcats are in no group. + model.ec.rxns = {'R1'; 'R2'; 'R3'; 'R4'; 'R5'}; + model.ec.kcat = ones(5, 1); + model.ec.source = {'brenda'; 'standard'; 'standard'; 'dlkcat'; 'brenda'}; + params.kcatSources = {'brenda'}; + params.kcatGroups = {'brenda'}; + params.sigma0logSource = 0.2; + params.sigma0logDefault = 0.5; + params.tieIsozymes = false; + lastwarn(''); + verifyWarning(testCase, @() resolveTunableContext(model, params, []), 'resolveTunableContext:unlabelled') + verifyEqual(testCase, lastwarn, ['3 of 5 tunable kcats match no evotune source group and get ' ... + 'sigma0_log_default (0.5); origins: standard, dlkcat']) + model.ec.source(2:4) = {'brenda'}; + verifyWarningFree(testCase, @() resolveTunableContext(model, params, [])) +end + +function testBrendaAggregationFromPreset_tc0736(testCase) + % Without an argument the BRENDA aggregate is the one the kcat + % assignment configuration gives brenda. + adapter = ModelAdapterManager.getAdapter(fullfile(findGECKOroot,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + verifyEqual(testCase, resolveKcatAggregation([], adapter), 'median') + adapter.params.kcat_assignment = struct('preset', 'gecko_classic'); + verifyEqual(testCase, resolveKcatAggregation([], adapter), 'max') + verifyEqual(testCase, resolveKcatAggregation('median', adapter), 'median') + adapter.params.kcat_assignment = struct('preset', 'brenda_ec_tree'); + adapter.params.kcat_sources.brenda = struct('aggregation', 'max'); + verifyEqual(testCase, resolveKcatAggregation([], adapter), 'max') + adapter.params.kcat_assignment = struct('stage', struct('action', 'select', 'sources', {{'custom'}})); + adapter.params = rmfield(adapter.params, 'kcat_sources'); + verifyEqual(testCase, resolveKcatAggregation([], adapter), 'max') +end + +function testBrendaDirectCallsUsePresetAggregation_tc0743(testCase) + % kcatsFromBrenda and fuzzyKcatMatching without an aggregation: the + % median of the default preset, or max when stages give no brenda table. + adapter = ModelAdapterManager.getAdapter(fullfile(findGECKOroot,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + ecModel = getECfromGEM(makeEcModel(getGeckoTestModel(), false, adapter)); + medianK = [0.5;0.5;4;4;40;0.5]; + maxK = [1;1;10;10;100;1]; + list = kcatsFromBrenda(ecModel, 'save', false, 'modelAdapter', adapter); + verifyEqual(testCase, list.kcats, medianK) + list = fuzzyKcatMatching(ecModel, 'ecRxns', [], 'modelAdapter', adapter); + verifyEqual(testCase, list.kcats, medianK) + adapter.params.kcat_assignment = struct('stage', struct('action', 'select', 'sources', {{'custom'}})); + list = kcatsFromBrenda(ecModel, 'save', false, 'modelAdapter', adapter); + verifyEqual(testCase, list.kcats, maxK) + list = fuzzyKcatMatching(ecModel, 'ecRxns', [], 'modelAdapter', adapter); + verifyEqual(testCase, list.kcats, maxK) +end + +function testKcatsFromOKPDefaultMethod_tc0737(testCase) + % Without a method, kcatsFromOKP uses the first of [kcat_sources.okp] methods. + adapter = ModelAdapterManager.getAdapter(fullfile(findGECKOroot,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + scratchDir = tempname; + mkdir(fullfile(scratchDir, 'data')); + cleanupDir = onCleanup(@() rmdir(scratchDir, 's')); + adapter.params.path = scratchDir; + ecModel.ec.rxns = {'R1'}; + try + kcatsFromOKP(ecModel, 'useStored', true, 'save', false, 'modelAdapter', adapter); + err = []; + catch err + end + verifyEqual(testCase, err.identifier, 'kcatsFromOKP:noStored') + verifySubstring(testCase, err.message, 'KinForm-H') + adapter.params.kcat_sources.okp.methods = {'UniKP', 'CataPro'}; + try + kcatsFromOKP(ecModel, 'useStored', true, 'save', false, 'modelAdapter', adapter); + err = []; + catch err + end + verifyEqual(testCase, err.identifier, 'kcatsFromOKP:noStored') + verifySubstring(testCase, err.message, 'UniKP') +end + +function testOkpMethodsAndFiles_tc0738(testCase) + % CatPred and TurNup are not accepted in [kcat_sources.okp] methods, and + % an okp_ file that is no known method's file fails naming the file. + verifyError(testCase, @() kcatAssignmentConfig(struct('kcat_sources', ... + struct('okp', struct('methods', {{'CatPred'}})))), 'kcatAssignment:config') + known = 'CataPro, CatRange, DLKcat, EITLEM, KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet, UniKP'; + try + kcatAssignmentConfig(struct('kcat_sources', struct('okp', struct('methods', {{'Zed', 'UniKP', 'Zed', 'Abc'}})))); + err = []; + catch err + end + verifyEqual(testCase, err.message, ['Unknown OKP methods in [kcat_sources.okp] methods: Zed, Abc; methods: ' known '.']) + verifyError(testCase, @() kcatAssignmentConfig(struct('kcat_sources', ... + struct('okp', struct('methods', {{'TurNup'}})))), 'kcatAssignment:config') + folder = tempname; mkdir(folder); + cleanupDir = onCleanup(@() rmdir(folder, 's')); + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"', '[[kcat_assignment.stage]]', ... + 'sources = ["okp_nosuchmethod", "okp_kinform_h"]'}, fullfile(folder, 'model_adapter.toml')); + adapter = ModelAdapterManager.getAdapter(folder); + report = evalc('r = updateKcatSources(struct(), ''modelAdapter'', adapter);'); %#ok + row = strcmp(r.source, 'okp_nosuchmethod'); + verifyEqual(testCase, r.status{row}, 'failed') + verifyEqual(testCase, r.note{row}, ['okp_nosuchmethod is not the file of a known OKP method; the files are ' ... + 'okp_catapro, okp_catrange, okp_dlkcat, okp_eitlem, okp_kinform_h, okp_kinform_l, okp_omniesi, ' ... + 'okp_omniesi_o2denet, okp_unikp.']) +end + +function testOkpMethodsAtLoad_tc0744(testCase) + % The methods are checked when the file is read. + base = {'conv_gem = "x.xml"', 'org_name = "x"', '[kcat_sources.okp]'}; + known = 'CataPro, CatRange, DLKcat, EITLEM, KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet, UniKP'; + cases = { ... + {'methods = "KinForm-H"'}, 'kcat_sources.okp.methods must be a list of strings'; ... + {'methods = [1, 2]'}, 'kcat_sources.okp.methods must be a list of strings'; ... + {'methods = ["Zed", "UniKP", "Zed", "CatPred"]'}, ... + ['Unknown OKP methods in [kcat_sources.okp] methods: Zed, CatPred; methods: ' known '.']}; + for k = 1:size(cases, 1) + folder = tempname; mkdir(folder); + file = fullfile(folder, 'model_adapter.toml'); + writeTemp([base, cases{k,1}], file); + try + loadAdapterParams(file); + err = []; + catch err + end + verifyNotEmpty(testCase, err) + if k < 3 + verifyEqual(testCase, err.message, [file ': ' cases{k,2}]) + else + verifyEqual(testCase, err.message, cases{k,2}) + end + end +end + +function testKcatSourcesTableAndAggregation_tc0745(testCase) + base = {'conv_gem = "x.xml"', 'org_name = "x"'}; + cases = { ... + {'[kcat_sources]', 'brenda = "max"'}, 'kcat_sources.brenda must be a table'; ... + {'[kcat_sources.brenda]', 'aggregation = "mean"'}, 'kcat_sources.brenda.aggregation must be max or median'; ... + {'[kcat_sources.brenda]', 'aggregation = 1'}, 'kcat_sources.brenda.aggregation must be max or median'; ... + {'[matlab.params]', 'kcatAggregation = "max"'}, 'matlab.params.kcatAggregation is not allowed; set it in [kcat_sources.brenda] aggregation'; ... + {'[matlab.params.evotune]', 'maxGenerations = 3'}, 'matlab.params.evotune is not allowed; set it in [evotune]'}; + for k = 1:size(cases, 1) + folder = tempname; mkdir(folder); + file = fullfile(folder, 'model_adapter.toml'); + writeTemp([base, cases{k,1}], file); + try + loadAdapterParams(file); + err = []; + catch err + end + verifyNotEmpty(testCase, err, cases{k,2}) + if ~isempty(err) + verifyEqual(testCase, err.message, [file ': ' cases{k,2}]) + end + end + % A function argument gives its own text. + adapter = ModelAdapterManager.getAdapter(fullfile(findGECKOroot,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + try + resolveKcatAggregation('mean', adapter); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'aggregation must be max or median') + try + fuzzyKcatMatching(struct(), 'kcatAggregation', 'mean'); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'aggregation must be max or median') + try + loadBRENDAdata('kcatAggregation', 'mean'); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'aggregation must be max or median') +end + +function testNoEvotuneGroups_tc0746(testCase) + % evotune needs groups; an adapter class without them gets a message. + model.ec.rxns = {'R1'}; + model.ec.kcat = 1; + model.ec.source = {'brenda'}; + params = struct('sigma0logDefault', 0.5, 'tieIsozymes', false, 'kcatSources', {{'brenda'}}, ... + 'sigma0logSource', 0.2); + try + resolveTunableContext(model, params, []); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'No evotune source groups: set [evotune.source_groups] in model_adapter.toml') + % showAdapterParams of an adapter class without evotune settings. + adapter = ModelAdapterManager.getAdapter(fullfile(findGECKOroot,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + lines = splitlines(strtrim(evalc('showAdapterParams(adapter)'))); + verifyTrue(testCase, ismember('evotune.source_groups = [not set]', lines)) +end + +function testShowAdapterParamsOrder_tc0747(testCase) + % Lines follow the parameter table; kcat sources and their keys are + % ordered; whole numbers print without decimals at any size. + folder = tempname; mkdir(folder); + cleanupDir = onCleanup(@() rmdir(folder, 's')); + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"', 'f = 1000000', ... + '[kcat_sources.homologs]', 'min_identity = 40', 'k = 3', 'coverage = 0.5', ... + '[kcat_sources.okp_unikp]'}, fullfile(folder, 'model_adapter.toml')); + lines = splitlines(strtrim(evalc('showAdapterParams(ModelAdapterManager.getAdapter(folder))'))); + verifyTrue(testCase, ismember('f = 1000000 [file]', lines)) + fid = fopen(fullfile(findGECKOroot, 'src', 'model_adapter', 'adapterParameters.tsv'), 'r'); + cols = textscan(fid, '%s %*[^\n]', 'Delimiter', '\t', 'HeaderLines', 1); + fclose(fid); + position = zeros(numel(cols{1}), 1); + for k = 1:numel(cols{1}) + position(k) = find(startsWith(lines, [cols{1}{k} ' =']) | startsWith(lines, [cols{1}{k} '.']) | ... + startsWith(lines, [cols{1}{k} '[']), 1); + end + verifyEqual(testCase, position, sort(position)) + sourceLines = lines(startsWith(lines, 'kcat_sources.')); + verifyEqual(testCase, cellfun(@(l) l(1:find(l == ' ', 1)-1), sourceLines, 'UniformOutput', false), ... + {'kcat_sources.brenda.aggregation'; 'kcat_sources.ec_tree.k'; 'kcat_sources.homologs.coverage'; ... + 'kcat_sources.homologs.k'; 'kcat_sources.homologs.min_identity'; 'kcat_sources.okp.methods'}) +end + +function testGrExpNotSet_tc0739(testCase) + % gr_exp has no default; a function that needs it says so. + folder = tempname; mkdir(folder); + cleanupDir = onCleanup(@() rmdir(folder, 's')); + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"'}, fullfile(folder, 'model_adapter.toml')); + adapter = ModelAdapterManager.getAdapter(folder); + try + sensitivityTuning(struct(), 'modelAdapter', adapter); + err = []; + catch err + end + verifyEqual(testCase, err.identifier, 'GECKO:grExpNotSet') + verifyEqual(testCase, err.message, 'gr_exp is not set in model_adapter.toml') + try + sigmaFitter(struct(), 'modelAdapter', adapter, 'makePlot', false); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'gr_exp is not set in model_adapter.toml') + try + flexibilizeEnzConcs(struct(), 'modelAdapter', adapter); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'gr_exp is not set in model_adapter.toml') + try + relaxEnzConcs(struct(), 'modelAdapter', adapter); + err = []; + catch err + end + verifyEqual(testCase, err.message, 'gr_exp is not set in model_adapter.toml') +end + +function testShowAdapterParams_tc0740(testCase) + % One line per item, = []. + folder = tempname; mkdir(folder); + cleanupDir = onCleanup(@() rmdir(folder, 's')); + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"', 'p_tot = 0.4', ... + '[kcat_assignment]', 'preset = "gecko_classic"', ... + '[kcat_sources.okp]', 'methods = ["UniKP", "CataPro"]'}, fullfile(folder, 'model_adapter.toml')); + lines = splitlines(strtrim(evalc('showAdapterParams(ModelAdapterManager.getAdapter(folder))'))); + verifyTrue(testCase, ismember('p_tot = 0.4 [file]', lines)) + verifyTrue(testCase, ismember('sigma = 0.5 [default]', lines)) + verifyTrue(testCase, ismember('gr_exp = [not set]', lines)) + verifyTrue(testCase, ismember('uniprot.reviewed = false [default]', lines)) + verifyTrue(testCase, ismember('kegg.id = [default]', lines)) + verifyTrue(testCase, ismember('complex.taxonomic_id = [not set]', lines)) + verifyTrue(testCase, ismember('kcat_assignment.preset = gecko_classic [file]', lines)) + verifyTrue(testCase, ismember(['kcat_assignment.stage[1] = select: custom, database_top, dlkcat, database_bottom; ' ... + 'top_origin_limit=6 [preset gecko_classic]'], lines)) + verifyTrue(testCase, ismember('kcat_sources.brenda.aggregation = max [preset gecko_classic]', lines)) + verifyTrue(testCase, ismember('kcat_sources.okp.methods = UniKP, CataPro [file]', lines)) + verifyTrue(testCase, ismember('evotune.source_groups.custom = custom; sigma0_log 0.1 [default]', lines)) + verifyTrue(testCase, ismember('evotune.source_groups.combined = combined; sigma0_log 0.4 [default]', lines)) + % Integer-valued numbers print whole, other numbers with %g. + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"', 'f = 0.25', ... + '[complex]', 'taxonomic_id = 5592920'}, fullfile(folder, 'model_adapter.toml')); + lines = splitlines(strtrim(evalc('showAdapterParams(ModelAdapterManager.getAdapter(folder))'))); + verifyTrue(testCase, ismember('complex.taxonomic_id = 5592920 [file]', lines)) + verifyTrue(testCase, ismember('f = 0.25 [file]', lines)) + % Stages from the file show only what they set. + writeTemp({'conv_gem = "x.xml"', 'org_name = "x"', ... + '[[kcat_assignment.stage]]', 'sources = ["custom", "okp"]', 'criteria = "median"', ... + '[evotune.source_groups.a]', 'sources = ["a", "b_*"]', 'sigma0_log = 0.25'}, fullfile(folder, 'model_adapter.toml')); + lines = splitlines(strtrim(evalc('showAdapterParams(ModelAdapterManager.getAdapter(folder))'))); + verifyTrue(testCase, ismember('kcat_assignment.preset = none (stages are given) [file]', lines)) + verifyTrue(testCase, ismember('kcat_assignment.stage[1] = select: custom, okp; criteria=median [file]', lines)) + verifyTrue(testCase, ismember('evotune.source_groups.a = a, b_*; sigma0_log 0.25 [file]', lines)) + T = showAdapterParams(ModelAdapterManager.getAdapter(folder)); + verifyEqual(testCase, T.Properties.VariableNames, {'key', 'value', 'origin'}) +end + function s = keepFields(s, template) % s reduced to the fields (recursively) that template has. fields = fieldnames(s); diff --git a/test/unit_tests/geckoCoreFunctionTests.m b/test/unit_tests/geckoCoreFunctionTests.m index 3f45a4c5..431d44fe 100644 --- a/test/unit_tests/geckoCoreFunctionTests.m +++ b/test/unit_tests/geckoCoreFunctionTests.m @@ -315,7 +315,8 @@ function testfuzzyKcatMatchingMedian_tc0060(testCase) ecModel = makeEcModel(model, false, adapter); ecModel = getECfromGEM(ecModel); - kcatListMax = fuzzyKcatMatching(ecModel, [], adapter); + kcatListMax = fuzzyKcatMatching(ecModel, 'ecRxns', [], ... + 'modelAdapter', adapter, 'kcatAggregation', 'max'); verifyEqual(testCase, kcatListMax.kcats, [1;1;10;10;100;1]) kcatListMedian = fuzzyKcatMatching(ecModel, 'ecRxns', [], ... @@ -328,10 +329,10 @@ function testfuzzyKcatMatchingMedian_tc0060(testCase) verifyEqual(testCase, kcatListMedian.wildcardLvl, kcatListMax.wildcardLvl) verifyEqual(testCase, kcatListMedian.origin, kcatListMax.origin) - % 'max' is the default, and an unknown value is refused. - kcatListExplicitMax = fuzzyKcatMatching(ecModel, 'ecRxns', [], ... - 'modelAdapter', adapter, 'kcatAggregation', 'max'); - verifyEqual(testCase, kcatListExplicitMax.kcats, kcatListMax.kcats) + % Without an argument the aggregate of the default preset (median) is + % used, and an unknown value is refused. + kcatListDefault = fuzzyKcatMatching(ecModel, [], adapter); + verifyEqual(testCase, kcatListDefault.kcats, kcatListMedian.kcats) verifyError(testCase, @() fuzzyKcatMatching(ecModel, 'ecRxns', [], ... 'modelAdapter', adapter, 'kcatAggregation', 'mean'), ... 'fuzzyKcatMatching:invalidKcatAggregation') @@ -353,6 +354,7 @@ function testfuzzyKcatMatching_tc0010(testCase) % resort. geckoPath = findGECKOroot; adapter = ModelAdapterManager.getAdapter(fullfile(geckoPath,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + adapter.params.kcat_sources.brenda.aggregation = 'max'; % the default preset's median is not what this test reads model = getGeckoTestModel(); %First all rxns, full model ecModel = makeEcModel(model, false, adapter); @@ -412,6 +414,7 @@ function testfuzzyKcatMatching_tc0010(testCase) function testKcats_tc0011(testCase) geckoPath = findGECKOroot; adapter = ModelAdapterManager.getAdapter(fullfile(geckoPath,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + adapter.params.kcat_sources.brenda.aggregation = 'max'; % the default preset's median is not what this test reads model = getGeckoTestModel(); %add an extra R3 reaction to be able to check that wildcards go in if there is no kcat in the dlkcat list rxnsToAdd = struct(); @@ -1912,6 +1915,7 @@ function testLoadBRENDAdataParsesNewTsvFormat_tc0043(testCase) % is the value used. geckoPath = findGECKOroot; adapter = ModelAdapterManager.getAdapter(fullfile(geckoPath,'test','unit_tests','ecTestGEM', 'TestGEMAdapter.m')); + adapter.params.kcat_sources.brenda.aggregation = 'max'; % the default preset's median is not what this test reads scratchDir = fullfile(tempname); cleanupDir = onCleanup(@() rmdir(scratchDir, 's')); @@ -2938,11 +2942,12 @@ function testResolveTunableContextFirstGroupWins_tc0719(testCase) model.ec.kcat = [1; 2]; model.ec.source = {'brenda (wc=0, origin=1)'; 'custom'}; params.kcatSources = {'brenda', 'custom', 'brenda'}; + params.kcatGroups = {'first', 'custom', 'second'}; params.sigma0logSource = [0.2; 0.1; 0.4]; params.sigma0logDefault = 1; params.tieIsozymes = false; [~, ~, ~, groups, sigma0log] = resolveTunableContext(model, params, []); - verifyEqual(testCase, groups, {'brenda'; 'custom'}) + verifyEqual(testCase, groups, {'first'; 'custom'}) verifyEqual(testCase, sigma0log, [0.2; 0.1]) end @@ -2965,16 +2970,6 @@ function testKcatSourcesTableMergesWithPreset_tc0721(testCase) verifyEqual(testCase, config.sources.brenda.aggregation, 'max') end -function testOkpMethodDefault_tc0722(testCase) - % Without an [okp] table, the OpenKineticsPredictor method is KinForm-H. - folder = tempname; mkdir(folder); - cleanupDir = onCleanup(@() rmdir(folder, 's')); - writelines(["conv_gem = ""models/x.xml"""; "org_name = ""testus testus"""], ... - fullfile(folder, 'model_adapter.toml')); - params = loadAdapterParams(folder); - verifyEqual(testCase, params.okp.method, 'KinForm-H') -end - function testOkpSourceGroupDefault_tc0723(testCase) % The okp source group of the presets is KinForm-H, unless the project % sets [kcat_sources.okp] methods. diff --git a/test/unit_tests/kcatContractTests.m b/test/unit_tests/kcatContractTests.m index dd820f61..f4aa0096 100644 --- a/test/unit_tests/kcatContractTests.m +++ b/test/unit_tests/kcatContractTests.m @@ -196,7 +196,7 @@ function testResolveTunableContextMatchesSourceBeforeDetail_tc0312(testCase) [model, ~] = kcatProject(); model.ec.kcat(1:4) = 10; model.ec.source(1:4) = {'brenda (wc=0, origin=1)'; 'combined (sd=0.10, n=2)'; 'DLKcat'; 'other (x)'}; - params = struct('kcatSources', {{'brenda', 'combined', 'dlkcat'}}, 'sigma0logSource', [0.2; 0.3; 0.4], ... + params = struct('kcatSources', {{'brenda', 'combined', 'dlkcat'}}, 'kcatGroups', {{'brenda', 'combined', 'dlkcat'}}, 'sigma0logSource', [0.2; 0.3; 0.4], ... 'sigma0logDefault', 0.5, 'tieIsozymes', false); mask = false(numel(model.ec.rxns), 1); mask(1:4) = true; diff --git a/test/unit_tests/kcatListTests.m b/test/unit_tests/kcatListTests.m index db343938..53cdca6f 100644 --- a/test/unit_tests/kcatListTests.m +++ b/test/unit_tests/kcatListTests.m @@ -46,6 +46,7 @@ function testLoadBRENDAdataLowercaseEc_tc0105(testCase) function adapter = getTestAdapter() geckoPath = findGECKOroot; adapter = ModelAdapterManager.getAdapter(fullfile(geckoPath,'test','unit_tests','ecTestGEM','TestGEMAdapter.m')); + adapter.params.kcat_sources.brenda.aggregation = 'max'; end function writeTable(file, header, row) diff --git a/test/unit_tests/kcatSourceFormatTests.m b/test/unit_tests/kcatSourceFormatTests.m index 68894ed5..204339cd 100644 --- a/test/unit_tests/kcatSourceFormatTests.m +++ b/test/unit_tests/kcatSourceFormatTests.m @@ -197,7 +197,7 @@ function testResolveTunableContextGroupsATunedSourceByItsOrigin_tc0712(testCase) model.ec.kcat(1:4) = 10; model.ec.source(1:4) = {'brenda | tuned=cmaes (prior=2)'; 'Combined (sd=0.10, n=2) | tuned=sensitivity (prior=1)'; ... 'DLKcat'; 'other | tuned=cmaes (prior=2)'}; - params = struct('kcatSources', {{'brenda', 'combined', 'dlkcat'}}, 'sigma0logSource', [0.2; 0.3; 0.4], ... + params = struct('kcatSources', {{'brenda', 'combined', 'dlkcat'}}, 'kcatGroups', {{'brenda', 'combined', 'dlkcat'}}, 'sigma0logSource', [0.2; 0.3; 0.4], ... 'sigma0logDefault', 0.5, 'tieIsozymes', false); mask = false(numel(model.ec.rxns), 1); mask(1:4) = true; diff --git a/tutorials/full_ecModel/model_adapter.toml b/tutorials/full_ecModel/model_adapter.toml index 7d416a61..0910b66b 100644 --- a/tutorials/full_ecModel/model_adapter.toml +++ b/tutorials/full_ecModel/model_adapter.toml @@ -64,27 +64,26 @@ rmse_threshold = 0.2 max_generations = 212 # Kcat sources, from least to most trusted, each with its prior width -# (lower is more trusted). The groups match the labels of ec.source that -# the preset brenda_ec_tree gives: combined (the KinForm-H predictions of -# the okp stage), ec_tree, brenda and custom. Kcats with another label, -# such as standard, take sigma0_log_default. +# sigma0_log (the standard deviation of the natural log of the kcat; lower +# is more trusted). The groups match the labels of ec.source that the preset +# brenda_ec_tree gives: combined (the KinForm-H predictions of the okp +# stage), ec_tree, brenda and custom. Kcats with another label, such as +# standard, take sigma0_log_default. [evotune.source_groups.combined] sources = ["combined"] +sigma0_log = 0.4 [evotune.source_groups.ec_tree] sources = ["ec_tree"] +sigma0_log = 0.3 [evotune.source_groups.brenda] sources = ["brenda"] +sigma0_log = 0.2 [evotune.source_groups.custom] sources = ["custom"] - -[evotune.sigma0_log_source] -combined = 0.4 -ec_tree = 0.3 -brenda = 0.2 -custom = 0.1 +sigma0_log = 0.1 # Kcats are assigned with the default preset: custom kcats # (data/kcats/custom.tsv), the BRENDA median of measurements of the organism diff --git a/tutorials/full_ecModel/protocol.m b/tutorials/full_ecModel/protocol.m index 2d97d95d..9dfb460a 100644 --- a/tutorials/full_ecModel/protocol.m +++ b/tutorials/full_ecModel/protocol.m @@ -243,11 +243,11 @@ % against these data and searches with CMA-ES, an evolutionary optimiser. % screenKcatLeverage first ranks the kcats by their effect on the fit, and % the search varies the top kcats of the screen. The trust in each kcat -% source ([evotune.source_groups] and [evotune.sigma0_log_source] of -% model_adapter.toml) limits how far each kcat moves, and -% prior_penalty_weight penalises large changes. The settings depend on the -% model, on the certainty of its kcats and on the data, and the search runs -% many FBA problems, so it is shown here and not run: +% source ([evotune.source_groups] of model_adapter.toml) limits how far +% each kcat moves, and prior_penalty_weight penalises large changes. The +% settings depend on the model, on the certainty of its kcats and on the +% data, and the search runs many FBA problems, so it is shown here and not +% run: % screen = screenKcatLeverage(ecModel, 'nProc', 8); % [ecModel, result] = cmaesKcatTuning(ecModel, 'screen', screen, 'nProc', 8, 'seed', 0); % The standard and the advanced tuning are alternatives: cmaesKcatTuning