diff --git a/CHANGELOG.md b/CHANGELOG.md index 7516173..c48a046 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,14 @@ * `upgrade_project` (`geckopy upgrade`) reports the calls of removed or replaced functions in the project's scripts with their replacement, for example `submit_open_kinetics_predictor`, `fetch_open_kinetics_predictor`, `build_okp_input_csv` and `truncate_values`. * The deprecated aliases with GECKO 3 names are removed from `geckopy`: `run_dlkcat`, `read_dlkcat_output`, `write_dlkcat_input`, `apply_custom_kcats`, `get_kcat_across_isozymes`, `get_standard_kcat`, `select_kcat_value`, `merge_dlkcat_and_fuzzy_kcats`, `get_ec_from_database`, `get_ec_from_gem`, `sigma_fitter` and `constrain_flux_data`, and `find_max_value`, which ports `findMaxValue` that GECKO 4 removed. `upgrade_project` (`geckopy upgrade`) reports each of them in a project's scripts with its replacement, which `geckopy.upgrade.RENAMED` lists. `extract_enzyme_substrate_pairs` stays public. * The `okp` source group of the presets is the OpenKineticsPredictor method KinForm-H alone by default, instead of the eight methods with combination parameters; a project that sets `[kcat_sources.okp] methods` keeps its list. `update_kcat_sources` then gathers `okp_kinform_h.tsv` only. GECKO uses the same default. -* The default of `okp.method` (the OpenKineticsPredictor method of `kcats_from_okp` without `method`) is `KinForm-H` instead of `CataPro`, as in GECKO. +* `okp.method` is removed from model_adapter.toml; `[kcat_sources.okp] methods` is the only setting that names OpenKineticsPredictor methods (default `["KinForm-H"]`). `kcats_from_okp` without `method` uses the first entry of `methods`; it used `okp.method`. `[kcat_sources.okp_] method` is removed: the method of the file `okp_` is the known method whose file name is `okp_` (`okp_` and the method in lower case, each run of other characters than letters and digits as `_`), and `update_kcat_sources` reports any other `okp_` as failed (` is not the file of a known OKP method; the files are ...`). `Unknown OKP methods in [kcat_sources.okp] methods: ...` lists the names comma-separated. `CatPred` and `TurNup` are no longer accepted in `[kcat_sources.okp] methods`: `kcats_from_okp` rejects them, so the configuration now fails early with `Unknown OKP methods`. +* Evotune source groups are one table each, `[evotune.source_groups.]` with `sources` and `sigma0_log`, both required. `[evotune.sigma0_log_source]` and `SourceGroupRule.match_okp` are removed, and so is the `okp_method` argument of `classify_kcat_source`, `classify_kcat_sources`, `screen_kcat_leverage`, `cmaes_kcat_tuning`, `sweep_tunable_mask_size` and the other tuning functions; a group covers OpenKineticsPredictor kcats with a pattern such as `okp_*`. `sigma0_log` is a standard deviation of the natural log of the kcat. A `sources` entry ending in `*` is a prefix pattern on the origin (case-insensitive); exact matches over all groups are tried first, then patterns, the longest prefix first, and groups of equal length in file order. Without `[evotune.source_groups]` the groups are `custom` 0.1, `brenda` 0.2, `ec_tree` 0.3 and `combined` 0.4 (they were `dlkcat` 0.4, `brenda` 0.2 and `custom` 0.1); groups in the file replace the defaults entirely. Tuning warns when more than 20 % of the tunable kcats are in no group: ` of tunable kcats match no evotune source group and get sigma0_log_default (); origins: `. +* `kcat_aggregate_brenda` is removed from model_adapter.toml (and `ModelParameters`); `[kcat_sources.brenda] aggregation` is the only BRENDA aggregation setting. `kcats_from_brenda` and `fuzzy_kcat_matching` without an explicit aggregation use it as the kcat assignment config resolves it: median for the default preset `brenda_ec_tree` and the other median presets, max for `gecko_classic`, and max when neither the preset nor the file sets it. They used `kcat_aggregate_brenda`, which defaulted to `"max"`, so a call without an aggregation under the default preset now takes the median. +* `kcat_aggregate_candidates` and `kcat_aggregate_isozymes` are removed from model_adapter.toml (and `ModelParameters`). `apply_kcat_list` takes `criteria="max"` and `fill_kcats_from_isozymes` takes `aggregate="mean"` as their defaults, as before; they no longer read an adapter value. +* `gr_exp` has no default (it was `0.41`, the yeast growth rate). `sensitivity_tuning`, `fit_sigma`, `flexibilize_enz_concs` and `relax_enz_concs` called without a growth rate raise `ValueError: gr_exp is not set in model_adapter.toml` when the file does not set it; `geckopy init` writes it commented out, noting that the functions that tune kcats or fit sigma require it. A project that relied on `0.41` sets `gr_exp`. +* An unknown key in model_adapter.toml is an error, whatever its place: `ModelAdapter.from_folder` raises `ValueError` with `: unknown key in model_adapter.toml`. This includes keys under `evotune.source_groups.`, the `kcat_assignment.stage` entries and `kcat_sources.`. The keys of `[kcat_sources.]` are `aggregation` and `force_wildcard_level` for `brenda`, `k` for `ec_tree`, `k`, `min_identity` and `coverage` for `homologs`, `methods` for `okp`, and none for `custom`, `dlkcat` and `okp_`; another source name is an unknown key. A malformed evotune group raises `: evotune.source_groups. must be a table with sources and sigma0_log`, `... needs both sources and sigma0_log`, `... .sources must be a list of strings` or `... .sigma0_log must be a positive number`. The removed keys are unknown keys: no alias or warning remains for them. +* `geckopy adapter show [FOLDER]` (default: the current folder) and `show_adapter_params(adapter)` print one line per item in the order of `adapterParameters.tsv`: ` = []` with the origin `file`, `default`, `preset ` or `not set` (` = [not set]`), without a header. The kcat assignment stages (`kcat_assignment.stage[]`), the `kcat_sources..` settings and the evotune source groups with their `sigma0_log` are shown as resolved. GECKO has `showAdapterParams`. +* `ModelAdapter.from_folder` checks more of model_adapter.toml when it loads it: `[kcat_sources.okp] methods` must be a list of strings and name known OKP methods (`Unknown OKP methods in [kcat_sources.okp] methods: ; methods: .`), a `[kcat_sources]` entry must be a table (`: kcat_sources. must be a table`), `[kcat_sources.brenda] aggregation` must be `max` or `median`, an evotune group's `sources` must be a non-empty list of strings, and `sigma0_log` must be finite and positive. A function given another BRENDA aggregation raises `aggregation must be max or median`. The unlabelled-kcat warning of evotune tuning is also issued as a `UserWarning`. `geckopy adapter show` prints whole-valued numbers without decimals and lists the `kcat_sources` settings by source (brenda, ec_tree, homologs, okp, then others alphabetically) and key. * `submit_open_kinetics_predictor`, `fetch_open_kinetics_predictor`, `build_okp_input_csv` and `parse_okp_output` are private (`_submit_open_kinetics_predictor` and so on) and no longer exported from `geckopy`, `geckopy.gather_kcats` or `geckopy.gather_kcats.open_kinetics_predictor`: `kcats_from_okp` and `kcats_from_dlkcat(via="okp")` submit and fetch OpenKineticsPredictor jobs. GECKO moves its OKP engine functions into `kcatsFromOKP` in the same way. `OKPClient` and `OKPError` stay public. * `convert_custom_kcats` (or `geckopy upgrade`) turns `data/customKcats.tsv` into `data/kcats/custom.tsv`. `collect_kcats` and `assign_kcats` do not apply `data/customKcats.tsv`: when a stage lists custom, `data/kcats/custom.tsv` does not exist and `data/customKcats.tsv` does, they raise `ValueError` naming `convert_custom_kcats`, before they change the model. The decisions have no stage `custom`. GECKO's `collectKcats` does the same. * `download_uniprot` and `load_uniprot_tsv` download reviewed and unreviewed entries by default (`reviewed=False`), as the adapter default `uniprot.reviewed = false`; they downloaded reviewed entries only. diff --git a/docs/evotune_kcat_tuning.md b/docs/evotune_kcat_tuning.md index 79a66cf..21a1062 100644 --- a/docs/evotune_kcat_tuning.md +++ b/docs/evotune_kcat_tuning.md @@ -86,31 +86,34 @@ sigma0_log_default = 0.3 # trust for any source not listed below max_growth_weight = 2.0 # weight the growth-rate data double against flux data [evotune.source_groups.okp] -sources = ["OpenKineticsPredictor"] -match_okp = true +sources = ["okp_*"] # a trailing * is a prefix pattern on the origin +sigma0_log = 0.25 [evotune.source_groups.brenda] sources = ["brenda"] +sigma0_log = 0.2 [evotune.source_groups.custom] sources = ["custom"] - -[evotune.sigma0_log_source] -okp = 0.25 -brenda = 0.2 -custom = 0.1 +sigma0_log = 0.1 ``` What each knob means: -- **Trust tiers** (`source_groups` + `sigma0_log_source` + - `sigma0_log_default`). Group your kcats by how much you trust their - origin, and give each group a standard deviation in log-space: - smaller means more trusted, and the search needs stronger evidence - before it moves that kcat far. `custom` (curated by hand) at 0.1 is - three times more trusted than an unlabelled prediction at 0.3. This - is the one section with no universal default -- your model's own - source labels determine it. +- **Trust tiers** (`source_groups` + `sigma0_log_default`). Group your + kcats by how much you trust their origin. Each group is one table + with `sources`, the `ec.source` labels it covers, and `sigma0_log`, + the standard deviation of the natural log of the kcat: smaller means + more trusted, and the search needs stronger evidence before it moves + that kcat far. `custom` (curated by hand) at 0.1 is three times more + trusted than a kcat in no group at 0.3 (`sigma0_log_default`). A + `sources` entry ending in `*` is a prefix pattern on the origin + (`okp_*` covers `okp_kinform_h` and `okp_eitlem_kinetics`); exact + entries are tried first over all groups, then patterns, the longest + prefix first. Without `[evotune.source_groups]` the groups are + `custom` 0.1, `brenda` 0.2, `ec_tree` 0.3 and `combined` 0.4; groups + in the file replace them. When more than 20 % of the tunable kcats + are in no group, tuning warns and names the most common origins. - **`max_growth_weight`** (default `1.0`). The combined score is `(rmse_flux + w * rmse_max_growth) / (w + 1)`. At `2`, growth-rate error counts double against flux error -- useful when growth rate is diff --git a/docs/future_improvements.md b/docs/future_improvements.md index 716c7dd..dddccc6 100644 --- a/docs/future_improvements.md +++ b/docs/future_improvements.md @@ -64,7 +64,7 @@ keep an in-flight release on schedule. a range (e.g. `"0.1-2.5"`) is collapsed to its upper bound unconditionally in `parse_brenda_json`. After the kcat-aggregation refactor moved the rest of the pipeline to be configurable - (`kcat_aggregate_brenda = "max" | "median"`), this is the one + (`[kcat_sources.brenda] aggregation = "max" | "median"`), this is the one residual max-leaning step. Honouring the adapter setting here would make every layer consistent. Small effect (only rows reported as ranges) — deferred. diff --git a/docs/kcat_aggregation.md b/docs/kcat_aggregation.md index a28eff1..a94f284 100644 --- a/docs/kcat_aggregation.md +++ b/docs/kcat_aggregation.md @@ -5,12 +5,14 @@ organism) max and the per-triple median of all raw measurements that fell into the triple. They live in the same `kcat.tsv` / `sa.tsv` file distinguished by an `aggregation` column; `load_brenda_data` splits them into `BrendaData.kcat_max` / `.kcat_median` (and `.sa_*`) so -consumers never see doubled-up rows. The default -(`kcat_aggregate_brenda = "max"` on `ModelParameters`) preserves -MATLAB-GECKO behaviour; flipping that field to `"median"` (or passing -`fuzzy_kcat_matching(aggregate="median")` per call) makes both -aggregation layers — the per-triple snapshot view and the per-EC -runtime collapse — use median together. +consumers never see doubled-up rows. The setting +`[kcat_sources.brenda] aggregation` chooses between them: the default +preset `brenda_ec_tree` uses `"median"`, and `gecko_classic` uses `"max"`, +which is the GECKO 3 behaviour. A call such as `kcats_from_brenda` or +`fuzzy_kcat_matching` without an explicit aggregation takes that value +(`fuzzy_kcat_matching(aggregate="max")` overrides it per call). Either +value makes both aggregation layers — the per-triple snapshot view and +the per-EC runtime collapse — use the same aggregation together. This document quantifies the two layers separately by walking the raw BRENDA JSON (BRENDA release 2026.1, ~57 900 raw kcat rows, ~39 000 @@ -56,7 +58,7 @@ triples to a single per-EC kcat. There are four possible |---|---|---|---| | max | max | **maxmax** | historical default (MATLAB GECKO) | | max | median | **maxmed** | the runtime-only median flag we had before | -| median | median | **medmed** | adapter `kcat_aggregate_brenda="median"` | +| median | median | **medmed** | `[kcat_sources.brenda] aggregation = "median"` | | median | max | medmax | unused; included for symmetry | `log10` spreads: @@ -81,7 +83,7 @@ typical BRENDA assay; for 8.6 % it is ≥ 1 000× above. Most of that comes from the runtime aggregation layer (`maxmax / maxmed` ≈ 1.38), with a small further bias from the snapshot layer (`maxmed / medmed` ≈ 0.06). Both swap together when the user sets -`kcat_aggregate_brenda = "median"`. +`[kcat_sources.brenda] aggregation = "median"`. | EC | n | max [1/s] | median [1/s] | max / median | |---:|---:|---:|---:|---:| diff --git a/docs/migrating_from_gecko_matlab.md b/docs/migrating_from_gecko_matlab.md index 1757d59..bfb0359 100644 --- a/docs/migrating_from_gecko_matlab.md +++ b/docs/migrating_from_gecko_matlab.md @@ -228,7 +228,7 @@ Each row maps a MATLAB function to its geckopy equivalent. The | `applyKcatConstraints` | `apply_kcat_constraints` | **`[3→4]`** Writes coefficients with the forward-direction `usage_prot_*` / `prot_pool_exchange` convention. | | `findMetSmiles` | `find_met_smiles` | **`[Py]`** Asynchronous PubChem cache via `requests`; uses a TSV cache (`data/smilesDB.tsv`) so re-runs are offline. | | `getComplexData` | `get_complex_data` | — | -| `getKcatAcrossIsozymes` | `fill_kcats_from_isozymes` | **`[Py]`** Renamed. Aggregation strategy (`max` / `median` / `mean`) is adapter-configurable via `kcat_aggregate_isozymes`. | +| `getKcatAcrossIsozymes` | `fill_kcats_from_isozymes` | **`[Py]`** Renamed. The `aggregate` argument (`mean`, `median` or `max`) defaults to `mean`. | | `getReactionsFromEnzyme` | `get_reactions_from_enzyme` | **`[Py]`** Returns one `DataFrame` instead of five parallel cells. | | `setKcatForReactions` | `set_kcat_for_reactions` | Matches IDs without regard to letter case, ignores an unknown ID and does not apply the constraints unless `apply=True`, as `setKcatForReactions`; a light-model row is addressed by its `###_` ID. | | `setProtPoolSize` | `set_prot_pool_size` | **`[3→4]`** Sets the **upper** bound of `prot_pool_exchange` (forward direction). | @@ -422,8 +422,10 @@ JSON; bare EC numbers, plain organism names, `#` release header, `references` column). GECKO 3's `max_KCAT.txt` etc. are no longer read. **`[Py]`** geckopy additionally includes both **max** and **median** rows per (ec, substrate, organism) triple in one file, -distinguished by an `aggregation` column. The default -`kcat_aggregate_brenda = "max"` preserves MATLAB-GECKO behaviour. +distinguished by an `aggregation` column. The +setting `[kcat_sources.brenda] aggregation` chooses one: the default +preset `brenda_ec_tree` uses `"median"`; `gecko_classic` uses `"max"`, the +GECKO 3 behaviour. See [kcat_aggregation.md](kcat_aggregation.md). --- diff --git a/src/geckopy/__init__.py b/src/geckopy/__init__.py index db639fd..45a58eb 100644 --- a/src/geckopy/__init__.py +++ b/src/geckopy/__init__.py @@ -9,7 +9,7 @@ The most-used names, grouped by stage of a typical workflow: -- **Project setup**: ``ModelAdapter`` +- **Project setup**: ``ModelAdapter``, ``show_adapter_params`` - **Build**: ``load_conventional_gem``, ``make_ec_model`` - **Enrich kcats**: ``update_kcat_sources`` (``kcats_from_brenda``, ``kcats_from_dlkcat``, ``kcats_from_okp``, ...), ``collect_kcats``, @@ -38,7 +38,7 @@ __version__ = "0.0.0+unknown" # Adapter -from geckopy.adapter import ModelAdapter, ModelParameters +from geckopy.adapter import ModelAdapter, ModelParameters, show_adapter_params # Databases (loaders + dataclasses) from geckopy.databases import ( @@ -171,6 +171,7 @@ # Adapter "ModelAdapter", "ModelParameters", + "show_adapter_params", # Core data + builder + per-enzyme accessor "EcData", "EcModel", diff --git a/src/geckopy/adapter/__init__.py b/src/geckopy/adapter/__init__.py index 696e518..084d637 100644 --- a/src/geckopy/adapter/__init__.py +++ b/src/geckopy/adapter/__init__.py @@ -6,7 +6,8 @@ ModelParameters, UniprotParams, ) -from .resolve import resolve_adapter, resolve_param +from .resolve import resolve_adapter, resolve_gr_exp, resolve_param +from .show import format_adapter_params, show_adapter_params __all__ = [ "ComplexParams", @@ -14,6 +15,9 @@ "ModelAdapter", "ModelParameters", "UniprotParams", + "format_adapter_params", "resolve_adapter", + "resolve_gr_exp", "resolve_param", + "show_adapter_params", ] diff --git a/src/geckopy/adapter/adapter.py b/src/geckopy/adapter/adapter.py index e5590d4..f897c80 100644 --- a/src/geckopy/adapter/adapter.py +++ b/src/geckopy/adapter/adapter.py @@ -33,7 +33,7 @@ from pathlib import Path from typing import TYPE_CHECKING -from .params import ModelParameters +from .params import ModelParameters, check_adapter_keys if TYPE_CHECKING: import cobra @@ -82,6 +82,8 @@ def from_folder(cls, folder: str | Path) -> "ModelAdapter": with open(config_path, "rb") as f: data = tomllib.load(f) + check_adapter_keys(data, config_path) + # The path field is injected from the folder location, not the TOML, # so that the TOML file is location-independent. data["path"] = folder diff --git a/src/geckopy/adapter/adapterParameters.tsv b/src/geckopy/adapter/adapterParameters.tsv index dd19cd4..6b9f29b 100644 --- a/src/geckopy/adapter/adapterParameters.tsv +++ b/src/geckopy/adapter/adapterParameters.tsv @@ -5,13 +5,10 @@ org_name org_name Scientific name of the organism sigma sigma 0.5 Average enzyme saturation factor p_tot Ptot 0.5 Total protein content [g protein/gDW] f f 0.5 Fraction of enzymes in the model [g enzyme/g protein] -gr_exp gR_exp 0.41 Growth rate the model should reach when not limited by nutrient uptake [1/h] +gr_exp gR_exp Growth rate the model should reach when not limited by nutrient uptake [1/h]; no default, functions that need it fail when it is not set c_source c_source "" Exchange reaction of the main carbon source bio_rxn bioRxn "" Biomass pseudoreaction enzyme_comp enzyme_comp "cytoplasm" Compartment of the protein pseudometabolites -kcat_aggregate_brenda kcatAggregation "max" BRENDA aggregate read by fuzzyKcatMatching: max or median -kcat_aggregate_candidates "max" Aggregation of several candidate kcats for one reaction (geckopy) -kcat_aggregate_isozymes "mean" Aggregation of isozyme kcats when filling a missing value (geckopy) kegg.id kegg.ID "" KEGG organism code kegg.gene_id kegg.geneID "kegg" Field in the KEGG entry that matches model gene identifiers uniprot.type uniprot.type "taxonomy" Whether uniprot.id is a taxonomy or a proteome identifier @@ -21,15 +18,13 @@ uniprot.reviewed uniprot.reviewed false Use only reviewed (Swiss-Prot) entries complex.taxonomic_id complex.taxonomicID Taxonomy identifier for Complex Portal spontaneous.name_contains spontaneous.name_contains Reactions whose name contains this text are spontaneous spontaneous.file spontaneous.file TSV of spontaneous reactions, relative to the project folder: reaction identifiers, optionally a 0/1 column -evotune.sigma0_log_default evotune.sigma0logDefault 0.5 Prior standard deviation (log) for kcats whose source is in no group -evotune.source_groups evotune.kcatSources Trust tiers: group name to the ec.source values it covers -evotune.sigma0_log_source evotune.sigma0logSource Prior standard deviation (log) per group +evotune.sigma0_log_default evotune.sigma0logDefault 0.5 Prior standard deviation of the natural log of the kcat, for kcats that match no source group +evotune.source_groups evotune.kcatSources Trust tiers, one table per group with sources (ec.source origins, or prefixes ending in *, such as okp_*) and sigma0_log (standard deviation of the natural log of the kcat; the sd of a combined kcat is log10). Default groups: custom 0.1, brenda 0.2, ec_tree 0.3, combined 0.4 evotune.rmse_threshold evotune.rmseThreshold 0.2 Stop when the best RMSE reaches this evotune.max_generations evotune.maxGenerations 150 Maximum number of CMA-ES generations evotune.max_growth_weight evotune.maxGrowthWeight 1.0 Weight of the maximum-growth RMSE against the flux RMSE evotune.prior_penalty_weight evotune.priorPenaltyWeight 0.03 Weight of the prior term in the search objective evotune.tie_isozymes evotune.tieIsozymes true Give isozyme copies of one reaction a single kcat -okp.method okp.method "KinForm-H" OpenKineticsPredictor method okp.handle_long_sequences okp.handleLongSequences "truncate" How OpenKineticsPredictor handles sequences that are too long okp.include_similarity_columns okp.includeSimilarityColumns true Return similarity to the method's training data okp.canonicalize_substrates okp.canonicalizeSubstrates true Canonicalize substrate SMILES before prediction @@ -37,6 +32,6 @@ kcat_assignment.preset Kcat assignment preset: brenda_ec_tree (used when neith kcat_assignment.isozymes true Fill reactions without a kcat from their isozymes after the stages kcat_assignment.standard true Give the remaining reactions the standard kcat kcat_assignment.stage Stages instead of a preset: action (select or combine), sources, criteria and BRENDA limits ([[kcat_assignment.stage]]) -kcat_sources Settings per kcat source file, e.g. [kcat_sources.brenda] aggregation; [kcat_sources.okp] methods lists the OKP methods of the okp source group +kcat_sources Settings per kcat source file: [kcat_sources.brenda] aggregation and force_wildcard_level, [kcat_sources.ec_tree] k, [kcat_sources.homologs] k, min_identity and coverage, [kcat_sources.okp] methods (the OKP methods of the okp source group; default KinForm-H) matlab.make_model_anaerobic matlab.makeModelAnaerobic MATLAB function that constrains the ecModel to anaerobic growth matlab.change_protein_biomass matlab.changeProteinBiomass MATLAB function that changes the protein content of the biomass diff --git a/src/geckopy/adapter/params.py b/src/geckopy/adapter/params.py index 32d257e..e963b4d 100644 --- a/src/geckopy/adapter/params.py +++ b/src/geckopy/adapter/params.py @@ -2,13 +2,14 @@ ``KeggParams``, ``UniprotParams``, ``ComplexParams``, and the core fields of ``ModelParameters`` mirror the ``obj.params.*`` fields set in GECKO -MATLAB's adapter template. ``EvotuneParams`` holds the evotune settings; ``OkpParams`` is a -geckopy-only extension with no MATLAB equivalent. +MATLAB's adapter template. ``EvotuneParams`` holds the evotune settings and +``OkpParams`` the OpenKineticsPredictor request settings. Ported from GECKO MATLAB: src/deprecated/adapterTemplate.m. """ +import math from pathlib import Path -from typing import Any, Literal, Optional +from typing import Any, Literal, Optional, get_args, get_origin from pydantic import BaseModel, ConfigDict, Field, model_validator @@ -160,24 +161,25 @@ def _preset_or_stages(self) -> "KcatAssignmentParams": class SourceGroupRule(BaseModel): """One trust-tier entry in ``EvotuneParams.source_groups``. - ``ec.source`` holds literal strings like ``"dlkcat"``, ``"brenda"``, - and -- for OpenKineticsPredictor kcats -- the raw predictor method - name (e.g. ``"CataPro"``), not a generic ``"okp"`` tag. A rule - matches a given ``ec.source`` value if it's listed in ``sources``, - or (when ``match_okp`` is True) if it equals the project's - configured OKP method (``OkpParams.method``). + ``ec.source`` holds the label of the source of a kcat, such as + ``"custom"``, ``"brenda"``, ``"ec_tree"``, ``"dlkcat"``, ``"combined"`` or + ``"okp_kinform_h"``. An entry of ``sources`` matches the origin of the + label or the whole label, in either case without regard to case. An entry + ending in ``*`` is a prefix pattern on the origin: ``"okp_*"`` covers + ``okp_eitlem_kinetics`` and ``okp_kinform_h``. """ model_config = ConfigDict(extra="forbid") sources: list[str] = Field( - default_factory=list, - description="Literal ec.source strings belonging to this group.", + description=( + "ec.source labels belonging to this group; an entry ending in * " + "is a prefix pattern." + ), ) - match_okp: bool = Field( - default=False, + sigma0_log: float = Field( description=( - "Also match ec.source values equal to the project's " - "configured OpenKineticsPredictor method (OkpParams.method)." + "Prior standard deviation of the natural log of the kcat for this " + "group. Lower is more trusted, and narrows that group's proposal width." ), ) @@ -185,33 +187,30 @@ class SourceGroupRule(BaseModel): class EvotuneParams(BaseModel): """Hyperparameters for kcat tuning against experimental data. - Sources not matched by any ``source_groups`` entry fall back to the - ``*_default`` fields (matching MATLAB's ``noKcatSource`` behaviour: - ``sigma0logDefault`` etc. apply to every kcat first, then only ones - with a recognised source get overridden by their group's value). + A kcat whose source is in no ``source_groups`` entry takes + ``sigma0_log_default``. """ model_config = ConfigDict(extra="forbid") sigma0_log_default: float = Field( default=0.5, description=( - "Prior standard deviation in log-space for kcats whose source " - "matches no source_groups entry." + "Prior standard deviation of the natural log of the kcat for kcats " + "whose source matches no source_groups entry." ), ) source_groups: dict[str, SourceGroupRule] = Field( default_factory=lambda: { - "dlkcat": SourceGroupRule(sources=["dlkcat"]), - "brenda": SourceGroupRule(sources=["brenda"]), - "custom": SourceGroupRule(sources=["custom"]), + "custom": SourceGroupRule(sources=["custom"], sigma0_log=0.1), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "ec_tree": SourceGroupRule(sources=["ec_tree"], sigma0_log=0.3), + "combined": SourceGroupRule(sources=["combined"], sigma0_log=0.4), }, - description="Trust tiers: group name -> which ec.source values it covers.", - ) - sigma0_log_source: dict[str, float] = Field( - default_factory=lambda: {"dlkcat": 0.4, "brenda": 0.2, "custom": 0.1}, description=( - "Prior log-space standard deviation per group. Lower is more " - "trusted, and narrows that group's proposal width." + "Trust tiers: group name -> the ec.source labels it covers (an entry " + "ending in * is a prefix pattern, e.g. okp_*) and its sigma0_log, the " + "standard deviation of the natural log of the kcat. Groups given in " + "the TOML replace the defaults." ), ) @@ -268,39 +267,18 @@ class EvotuneParams(BaseModel): ), ) - @model_validator(mode="after") - def _check_group_keys(self) -> "EvotuneParams": - """The per-source dicts must have exactly ``source_groups``' - keys, otherwise a downstream lookup silently falls back to a - default.""" - group_names = set(self.source_groups) - for name in ("sigma0_log_source",): - keys = set(getattr(self, name)) - if keys != group_names: - raise ValueError( - f"EvotuneParams.{name} keys {sorted(keys)} must match " - f"source_groups keys {sorted(group_names)}." - ) - return self - class OkpParams(BaseModel): """OpenKineticsPredictor settings (used by kcats_from_okp). + The OKP methods are set in ``[kcat_sources.okp] methods``. + The API key is intentionally NOT here (it is a secret): provide it as a function argument, the OKP_API_KEY environment variable, or the file ``/data/okpApiKey.txt``. """ model_config = ConfigDict(extra="forbid") - method: str = Field( - default="KinForm-H", - description=( - "Predictor method: CataPro, CatPred, CatRange, DLKcat, EITLEM, " - "KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet, TurNup, UniKP " - "(see GET /api/v1/methods/)." - ), - ) handle_long_sequences: str = Field( default="truncate", description="How OKP handles sequences exceeding a method's max length.", @@ -333,7 +311,14 @@ class ModelParameters(BaseModel): sigma: float = Field(default=0.5, description="Average enzyme saturation factor") p_tot: float = Field(default=0.5, description="Total protein content [g/gDw]") f: float = Field(default=0.5, description="Fraction of enzymes in model [g/g]") - gr_exp: float = Field(default=0.41, description="Reference growth rate [1/h]") + gr_exp: Optional[float] = Field( + default=None, + description=( + "Growth rate the model should reach when not limited by nutrient " + "uptake [1/h]. Required by the functions that tune kcats or fit " + "sigma; there is no default." + ), + ) c_source: str = Field( default="", description="Reaction ID for preferred carbon source exchange" @@ -346,38 +331,6 @@ class ModelParameters(BaseModel): description="Compartment where protein pseudometabolites are placed", ) - # kcat-aggregation defaults applied when the matching function is called - # without an explicit aggregate/criteria argument. One field per function - # so each can keep its MATLAB-GECKO default while still being centrally - # overridable on the adapter. See docs/kcat_aggregation.md for the - # empirical rationale for offering `median` as an alternative to `max`. - kcat_aggregate_brenda: Literal["max", "median"] = Field( - default="max", - description=( - "Default aggregation for `fuzzy_kcat_matching` when no `aggregate` " - "is passed: collapse the BRENDA rows matched at one search level " - "to a single kcat. `max` matches MATLAB GECKO; `median` is more " - "robust to engineered-mutant/non-physiological-substrate tails." - ), - ) - kcat_aggregate_candidates: Literal["max", "min", "median", "mean"] = Field( - default="max", - description=( - "Default aggregation for `apply_kcat_list` when no `criteria` is " - "passed: collapse multiple candidate kcats for the same reaction " - "(e.g. several EC tokens per reaction) to one. `max` matches " - "MATLAB GECKO." - ), - ) - kcat_aggregate_isozymes: Literal["max", "median", "mean"] = Field( - default="mean", - description=( - "Default aggregation for `fill_kcats_from_isozymes` when no " - "`aggregate` is passed: combine known kcats across sibling " - "isozymes to fill a missing entry. `mean` matches MATLAB GECKO." - ), - ) - kegg: KeggParams = Field(default_factory=KeggParams) uniprot: UniprotParams = Field(default_factory=UniprotParams) complex: ComplexParams = Field(default_factory=ComplexParams) @@ -390,3 +343,99 @@ class ModelParameters(BaseModel): description="Settings per kcat source file, e.g. kcat_sources.brenda.aggregation.", ) matlab: MatlabParams = Field(default_factory=MatlabParams) + + +#: The keys of ``[kcat_sources.]`` that the kcat source gathering reads +#: (``update_kcat_sources``): the source name and its keys. A source file +#: ``okp_`` has no keys. +KCAT_SOURCE_KEYS: dict[str, tuple[str, ...]] = { + "brenda": ("aggregation", "force_wildcard_level"), + "ec_tree": ("k",), + "homologs": ("k", "min_identity", "coverage"), + "okp": ("methods",), + "dlkcat": (), + "custom": (), +} + + +def _kcat_source_keys(name: str) -> Optional[tuple[str, ...]]: + """The allowed keys of ``[kcat_sources.]``, or None for an unknown source.""" + from ..gather_kcats.kcat_sources import OKP_METHODS, okp_file + + if name in {okp_file(m) for m in OKP_METHODS}: + return () + return KCAT_SOURCE_KEYS.get(name) + + +def _check_table(cls: type[BaseModel], data: dict, prefix: str, file: Path | str) -> None: + for key, value in data.items(): + dotted = f"{prefix}{key}" + field = cls.model_fields.get(key) + if field is None or (cls is ModelParameters and key == "path"): + raise ValueError(f"{file}: unknown key {dotted} in model_adapter.toml") + annotation = field.annotation + args = [a for a in get_args(annotation) if a is not type(None)] + if get_origin(annotation) is None and isinstance(annotation, type) and issubclass(annotation, BaseModel): + if isinstance(value, dict): + _check_table(annotation, value, dotted + ".", file) + elif get_origin(annotation) is list and args and issubclass(args[0], BaseModel): + for item in value if isinstance(value, list) else []: + if isinstance(item, dict): + _check_table(args[0], item, dotted + ".", file) + elif get_origin(annotation) is dict and len(args) == 2 and isinstance(args[1], type) and issubclass(args[1], BaseModel): + for name, item in value.items() if isinstance(value, dict) else []: + if cls is EvotuneParams: + _check_group(item, f"{dotted}.{name}", file) + if isinstance(item, dict): + _check_table(args[1], item, f"{dotted}.{name}.", file) + elif cls is ModelParameters and key == "kcat_sources" and isinstance(value, dict): + for name, table in value.items(): + allowed = _kcat_source_keys(name) + if allowed is None: + raise ValueError(f"{file}: unknown key {dotted}.{name} in model_adapter.toml") + if not isinstance(table, dict): + raise ValueError(f"{file}: {dotted}.{name} must be a table") + for k in table: + if k not in allowed: + raise ValueError(f"{file}: unknown key {dotted}.{name}.{k} in model_adapter.toml") + _check_source_values(name, table, file) + + +def _check_source_values(name: str, table: dict, file: Path | str) -> None: + """The values of ``[kcat_sources.okp] methods`` and ``[kcat_sources.brenda] aggregation``.""" + if name == "okp" and "methods" in table: + methods = table["methods"] + if not isinstance(methods, list) or not all(isinstance(m, str) for m in methods): + raise ValueError(f"{file}: kcat_sources.okp.methods must be a list of strings") + from ..gather_kcats.kcat_sources import check_okp_methods + + check_okp_methods(methods) + if name == "brenda" and "aggregation" in table and table["aggregation"] not in ("max", "median"): + raise ValueError(f"{file}: kcat_sources.brenda.aggregation must be max or median") + + +def _check_group(item: Any, dotted: str, file: Path | str) -> None: + """The shape of one ``[evotune.source_groups.]`` table.""" + if not isinstance(item, dict): + raise ValueError(f"{file}: {dotted} must be a table with sources and sigma0_log") + unknown = [k for k in item if k not in ("sources", "sigma0_log")] + if unknown: + raise ValueError(f"{file}: unknown key {dotted}.{unknown[0]} in model_adapter.toml") + if "sources" not in item or "sigma0_log" not in item: + raise ValueError(f"{file}: {dotted} needs both sources and sigma0_log") + sources, width = item["sources"], item["sigma0_log"] + if not isinstance(sources, list) or not sources or not all(isinstance(s, str) for s in sources): + raise ValueError(f"{file}: {dotted}.sources must be a non-empty list of strings") + if isinstance(width, bool) or not isinstance(width, (int, float)) or not math.isfinite(width) or not width > 0: + raise ValueError(f"{file}: {dotted}.sigma0_log must be a positive number") + + +def check_adapter_keys(data: dict, file: Path | str) -> None: + """Raise ``ValueError`` for a key of model_adapter.toml that no parameter uses. + + The message is ``: unknown key in model_adapter.toml``. + Every table is checked: the sections, ``evotune.source_groups.``, the + ``kcat_assignment.stage`` entries and ``kcat_sources.`` (see + :data:`KCAT_SOURCE_KEYS`). ``matlab.params`` is free-form. + """ + _check_table(ModelParameters, data, "", file) diff --git a/src/geckopy/adapter/resolve.py b/src/geckopy/adapter/resolve.py index 2e93bb8..eb101bd 100644 --- a/src/geckopy/adapter/resolve.py +++ b/src/geckopy/adapter/resolve.py @@ -107,3 +107,27 @@ def resolve_param( return explicit resolved = resolve_adapter(model, adapter, purpose=purpose) return getattr(resolved.params, attr) + + +def resolve_gr_exp( + model, + explicit=None, + *, + adapter: Optional["ModelAdapter"] = None, + purpose: str = "this function", +) -> float: + """Return an explicit growth rate, else ``adapter.params.gr_exp``. + + Raises + ------ + ValueError + If ``explicit`` is ``None`` and the adapter has no ``gr_exp``: + ``gr_exp is not set in model_adapter.toml``. ``resolve_adapter`` + raises if there is no adapter either. + """ + if explicit is not None: + return float(explicit) + gr_exp = resolve_adapter(model, adapter, purpose=purpose).params.gr_exp + if gr_exp is None: + raise ValueError("gr_exp is not set in model_adapter.toml") + return float(gr_exp) diff --git a/src/geckopy/adapter/show.py b/src/geckopy/adapter/show.py new file mode 100644 index 0000000..9c29401 --- /dev/null +++ b/src/geckopy/adapter/show.py @@ -0,0 +1,120 @@ +"""Show the resolved parameters of an adapter and where each value comes from. + +One line per parameter, in the order of ``adapterParameters.tsv``, with the +value in effect and its origin: ``file`` (set in model_adapter.toml), +``preset `` (given by the kcat assignment preset) or ``default``. The +kcat assignment stages, the ``[kcat_sources]`` settings and the evotune source +groups are shown as resolved. GECKO (MATLAB) has the same function as +``showAdapterParams``; the command line is ``geckopy adapter show``. +""" +from __future__ import annotations + +import csv +from pathlib import Path +from typing import TYPE_CHECKING, Any + + +if TYPE_CHECKING: + from .adapter import ModelAdapter + +_TABLE = Path(__file__).with_name("adapterParameters.tsv") + + +def _keys() -> list[str]: + with open(_TABLE, newline="", encoding="utf-8") as f: + return [r["toml_key"] for r in csv.DictReader(f, delimiter="\t", quoting=csv.QUOTE_NONE)] + + +def _text(value: Any) -> str: + """A value as ``adapter show`` prints it: strings bare, booleans true/false, + lists comma-joined, floats with ``%g`` (whole-valued ones without decimals).""" + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, float): + return str(int(value)) if value.is_integer() else f"{value:g}" + if isinstance(value, (list, tuple)): + return ", ".join(_text(v) for v in value) + if isinstance(value, Path): + return value.as_posix() + return str(value) + + +def _line(key: str, value: Any, origin: str) -> str: + if value is None: + return f"{key} = [not set]" + return f"{key} = {_text(value)} [{origin}]" + + +#: The settings of a stage that are shown when set (file) or not at their default (preset). +_STAGE_DEFAULTS = {"criteria": "max", "top_origin_limit": 4, "bottom_origin_limit": 6, "wildcard_limit": 3} + + +def _stage_line(i: int, stage, shown: list[str], origin: str) -> str: + parts = [f"{stage.action}: {', '.join(stage.sources)}"] + parts.extend(f"{k}={_text(getattr(stage, k))}" for k in _STAGE_DEFAULTS if k in shown) + return f"kcat_assignment.stage[{i}] = {'; '.join(parts)} [{origin}]" + + +def format_adapter_params(adapter: "ModelAdapter") -> str: + """The resolved parameters of ``adapter`` as text (see :func:`show_adapter_params`).""" + from ..gather_kcats.kcat_assignment import DEFAULT_OKP_METHODS, kcat_assignment_config + + params = adapter.params + config = kcat_assignment_config(adapter) + preset_origin = f"preset {config.preset}" if config.preset else "file" + out: list[str] = [] + for key in _keys(): + owner: Any = params + *parents, leaf = key.split(".") + for name in parents: + owner = getattr(owner, name) + value = getattr(owner, leaf) + origin = "file" if leaf in owner.model_fields_set else "default" + if key == "kcat_assignment.preset": + if params.kcat_assignment.stage: + out.append("kcat_assignment.preset = none (stages are given) [file]") + else: + out.append(_line(key, config.preset, origin)) + elif key == "kcat_assignment.stage": + for i, stage in enumerate(config.stages, start=1): + if params.kcat_assignment.stage: + shown = [k for k in _STAGE_DEFAULTS if k in params.kcat_assignment.stage[i - 1].model_fields_set] + else: + shown = [k for k, d in _STAGE_DEFAULTS.items() if getattr(stage, k) != d] + out.append(_stage_line(i, stage, shown, "file" if params.kcat_assignment.stage else preset_origin)) + elif key == "kcat_sources": + sources = {name: dict(keys) for name, keys in config.sources.items()} + if "methods" not in sources.get("okp", {}): + sources.setdefault("okp", {})["methods"] = list(DEFAULT_OKP_METHODS) + order = ["brenda", "ec_tree", "homologs", "okp"] + for name in sorted(sources, key=lambda n: (order.index(n), "") if n in order else (len(order), n)): + keys = sources[name] + for k in sorted(keys): + v = keys[k] + if k in params.kcat_sources.get(name, {}): + where = "file" + elif name == "okp" and k == "methods": + where = "default" + else: + where = preset_origin + out.append(_line(f"kcat_sources.{name}.{k}", v, where)) + elif key == "evotune.source_groups": + for name, rule in value.items(): + out.append(f"evotune.source_groups.{name} = {_text(rule.sources)}; sigma0_log {_text(rule.sigma0_log)} [{origin}]") + else: + out.append(_line(key, value, origin)) + return "\n".join(out) + + +def show_adapter_params(adapter: "ModelAdapter") -> None: + """Print every adapter parameter with its value and where it comes from. + + One line per item, in the order of ``adapterParameters.tsv``: + `` = []``. The origin is ``file`` (set in + model_adapter.toml), ``default``, ``preset `` (given by the kcat + assignment preset) or ``not set`` (no value: `` = [not set]``). + The kcat assignment stages (``kcat_assignment.stage[]``), the + ``kcat_sources..`` settings and the evotune source groups, in + matching order with their ``sigma0_log``, are shown as resolved. + """ + print(format_adapter_params(adapter)) diff --git a/src/geckopy/adapter/template.py b/src/geckopy/adapter/template.py index f90cb11..9acd688 100644 --- a/src/geckopy/adapter/template.py +++ b/src/geckopy/adapter/template.py @@ -38,6 +38,24 @@ ], } +# Written last: [kcat_sources.] tables follow every other section's keys. +_KCAT_SOURCES_EXAMPLE = [ + "# Settings per kcat source. A preset sets the keys it needs; a table here", + "# overrides only the keys it sets. The keys each source reads:", + "# [kcat_sources.brenda]", + '# aggregation = "median" # "max" or "median"', + "# force_wildcard_level = 0", + "# [kcat_sources.ec_tree]", + "# k = 5", + "# [kcat_sources.homologs]", + "# k = 5", + "# min_identity = 0.4", + "# coverage = 0.8", + "# [kcat_sources.okp]", + '# methods = ["KinForm-H"] # the first is the default method of kcats_from_okp', + "", +] + def _format_toml_value(value: Any) -> str | None: """Format a Python value as a TOML literal, or None if unrepresentable.""" @@ -138,7 +156,7 @@ def generate_template_toml(*, project_name: str = "my-ecmodel") -> str: sections: list[tuple[str, type[BaseModel]]] = [] for name, field in ModelParameters.model_fields.items(): - if name in _INJECTED_FIELDS: + if name in _INJECTED_FIELDS or name == "kcat_sources": continue if _is_nested_model(field.annotation): sections.append((name, field.annotation)) @@ -164,7 +182,12 @@ def generate_template_toml(*, project_name: str = "my-ecmodel") -> str: for name, model_cls in sections: lines.append(f"[{name}]") instance = model_cls() + tables: list[tuple[str, Any]] = [] for sub_name, sub_field in model_cls.model_fields.items(): + value = getattr(instance, sub_name) + if isinstance(value, dict) and value and all(isinstance(v, BaseModel) for v in value.values()): + tables.append((sub_name, sub_field)) # written last: sub-tables follow the section's keys + continue if (name, sub_name) in _SET_IN_TEMPLATE: value = _format_toml_value(getattr(instance, sub_name)) lines.extend([*_SET_IN_TEMPLATE[(name, sub_name)], f"{sub_name} = {value}", ""]) @@ -175,5 +198,12 @@ def generate_template_toml(*, project_name: str = "my-ecmodel") -> str: default_value=getattr(instance, sub_name), required=False, )) - + for sub_name, sub_field in tables: + lines.append(f"# {sub_field.description}") + for group, rule in getattr(instance, sub_name).items(): + lines.append(f"# [{name}.{sub_name}.{group}]") + lines.extend(f"# {key} = {_format_toml_value(val)}" for key, val in rule.model_dump().items()) + lines.append("") + + lines.extend(_KCAT_SOURCES_EXAMPLE) return "\n".join(lines) + "\n" diff --git a/src/geckopy/cli.py b/src/geckopy/cli.py index 24c73ec..b911b7e 100644 --- a/src/geckopy/cli.py +++ b/src/geckopy/cli.py @@ -143,6 +143,14 @@ def cmd_upgrade(args: argparse.Namespace) -> int: return 0 +def cmd_adapter_show(args: argparse.Namespace) -> int: + """Print every parameter of the project's adapter, with its value and origin.""" + from .adapter import ModelAdapter, show_adapter_params + + show_adapter_params(ModelAdapter.from_folder(args.folder)) + return 0 + + def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser( prog="geckopy", description="Command-line interface for geckopy." @@ -227,6 +235,15 @@ def main(argv: list[str] | None = None) -> int: ) upgrade_parser.set_defaults(func=cmd_upgrade) + adapter_parser = sub.add_parser("adapter", help="Inspect the model adapter of a project") + adapter_sub = adapter_parser.add_subparsers(dest="adapter_command", required=True) + show_parser = adapter_sub.add_parser( + "show", + help="Print every parameter of model_adapter.toml with its value and origin (file, preset or default)", + ) + show_parser.add_argument("folder", nargs="?", default=".", help="Project folder (default: the current folder)") + show_parser.set_defaults(func=cmd_adapter_show) + args = parser.parse_args(argv) return args.func(args) diff --git a/src/geckopy/databases/brenda_loader.py b/src/geckopy/databases/brenda_loader.py index 82aeefa..7377649 100644 --- a/src/geckopy/databases/brenda_loader.py +++ b/src/geckopy/databases/brenda_loader.py @@ -78,9 +78,7 @@ def kcat_for(self, aggregation: Literal["max", "median"]) -> pd.DataFrame: return self.kcat_median if aggregation == "max": return self.kcat_max - raise ValueError( - f"aggregation must be 'max' or 'median', got {aggregation!r}" - ) + raise ValueError("aggregation must be max or median") def sa_for(self, aggregation: Literal["max", "median"]) -> pd.DataFrame: """Return the SA-derived-kcat view for the requested aggregation.""" @@ -88,9 +86,7 @@ def sa_for(self, aggregation: Literal["max", "median"]) -> pd.DataFrame: return self.sa_median if aggregation == "max": return self.sa_max - raise ValueError( - f"aggregation must be 'max' or 'median', got {aggregation!r}" - ) + raise ValueError("aggregation must be max or median") def load_brenda_data( @@ -118,7 +114,7 @@ def load_brenda_data( Both kcat and SA are loaded as separate max and median views over the raw measurements aggregated per (ec, substrate, organism) triple, so a project can opt into median aggregation via - ``adapter.params.kcat_aggregate_brenda`` without regenerating the + ``[kcat_sources.brenda] aggregation`` without regenerating the snapshot. See ``docs/kcat_aggregation.md`` for the rationale. Malformed lines (wrong number of fields, non-numeric value column) diff --git a/src/geckopy/ec_model/pipeline/fill_kcats.py b/src/geckopy/ec_model/pipeline/fill_kcats.py index 576ea1c..584155f 100644 --- a/src/geckopy/ec_model/pipeline/fill_kcats.py +++ b/src/geckopy/ec_model/pipeline/fill_kcats.py @@ -7,7 +7,7 @@ import logging import re -from typing import TYPE_CHECKING, Optional +from typing import TYPE_CHECKING import numpy as np @@ -32,7 +32,7 @@ def fill_kcats_from_isozymes( model: "EcModel", *, apply: bool = False, - aggregate: Optional[str] = None, + aggregate: str = "mean", ) -> None: """Fill in missing kcats by combining known kcats across sibling isozymes. @@ -67,9 +67,7 @@ def fill_kcats_from_isozymes( aggregate How to combine sibling kcats: ``"mean"`` (matches MATLAB GECKO), ``"median"`` (more robust for log-distributed rate constants), or - ``"max"``. ``None`` (default) reads - ``model.adapter.params.kcat_aggregate_isozymes`` if an adapter is - attached, falling back to ``"mean"`` otherwise. + ``"max"``. Default ``"mean"``. Raises ------ @@ -83,13 +81,6 @@ def fill_kcats_from_isozymes( "fill_kcats_from_isozymes: not applicable to gecko-light models." ) - if aggregate is None: - adapter = getattr(model, "adapter", None) - aggregate = ( - adapter.params.kcat_aggregate_isozymes - if adapter is not None else "mean" - ) - kcat = model.ec.kcat known_mask = kcat > 0 missing_mask = kcat == 0 diff --git a/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py b/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py index 4fc0573..c725ed7 100644 --- a/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py +++ b/src/geckopy/gather_kcats/fuzzy_kcat_matching/__init__.py @@ -156,9 +156,9 @@ def fuzzy_kcat_matching( How to collapse the BRENDA rows matched at a search level to one kcat: ``"max"`` (the highest reported turnover) or ``"median"`` (more robust to assay outliers / engineered mutants). ``None`` - (default) reads the value from - ``model.adapter.params.kcat_aggregate_brenda``, which itself - defaults to ``"max"``. + (default) reads ``[kcat_sources.brenda] aggregation`` of the adapter + as the kcat assignment resolves it: median for the default preset + ``brenda_ec_tree``, max for ``gecko_classic``. The choice sets both aggregation layers consistently: it selects the matching snapshot view (``brenda.kcat_for(aggregate)`` / @@ -199,7 +199,9 @@ def fuzzy_kcat_matching( ) if aggregate is None: - aggregate = adapter.params.kcat_aggregate_brenda + from ..kcat_assignment import brenda_aggregation + + aggregate = brenda_aggregation(adapter) organism_name = (adapter.params.org_name or "").strip() n = model.ec.n_rxns diff --git a/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py b/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py index 102b89f..f293505 100644 --- a/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py +++ b/src/geckopy/gather_kcats/fuzzy_kcat_matching/_brenda_query.py @@ -125,7 +125,7 @@ def _aggregate_kcats(kcats: np.ndarray, how: str) -> float: return float(np.median(kcats)) if how == "max": return float(np.max(kcats)) - raise ValueError(f"aggregate must be 'max' or 'median', got {how!r}") + raise ValueError("aggregation must be max or median") def match_kcat( diff --git a/src/geckopy/gather_kcats/kcat_assignment.py b/src/geckopy/gather_kcats/kcat_assignment.py index d75530c..37f6293 100644 --- a/src/geckopy/gather_kcats/kcat_assignment.py +++ b/src/geckopy/gather_kcats/kcat_assignment.py @@ -10,7 +10,6 @@ import json import logging -import re from dataclasses import dataclass, field from pathlib import Path from typing import TYPE_CHECKING, Any, Optional @@ -22,7 +21,14 @@ from ..ec_model.pipeline.fill_kcats import fill_kcats_from_isozymes from .get_standard_kcat import assign_standard_kcat from .kcat_source_format import format_kcat_source, is_standard_source -from .kcat_sources import load_kcat_source, read_kcat_source +from .kcat_sources import ( + DEFAULT_OKP_METHODS, + check_okp_methods, + load_kcat_source, + okp_file, + okp_method_of_file, + read_kcat_source, +) from .kcats_from import _brenda_release, _measurements, kcats_from_brenda, kcats_from_dlkcat, kcats_from_ec_tree, kcats_from_okp from .merge_kcats import _strip_okp_prefix, merge_kcats, normalize_source from .select_kcat_value import apply_kcat_list @@ -36,20 +42,6 @@ PRESETS = ("brenda_ec_tree", "ec_tree_homologs", "ec_tree_okp", "okp", "gecko_median", "gecko_classic") _DATABASE_TOKENS = {"brenda", "sabio_rk"} -OKP_METHODS = ("CataPro", "CatPred", "CatRange", "DLKcat", "EITLEM", "KinForm-H", "KinForm-L", - "OmniESI", "OmniESI-O2DENet", "TurNup", "UniKP") -#: OKP methods that read whole reactions (substrate set, and products for TurNup) -#: instead of enzyme-substrate pairs; kcats_from_okp submits pairs only. -REACTION_OKP_METHODS = ("CatPred", "TurNup") -#: The OKP methods of the ``okp`` source group by default; a project lists others in -#: ``[kcat_sources.okp] methods`` (those that read enzyme-substrate pairs and have -#: combination parameters: not CatRange, which predicts classes, nor the reaction methods). -DEFAULT_OKP_METHODS = ("KinForm-H",) - - -def okp_file(method: str) -> str: - """The source file name of an OKP method, e.g. ``okp_kinform_h``.""" - return "okp_" + re.sub(r"[^A-Za-z0-9]+", "_", method).lower().strip("_") @dataclass @@ -126,13 +118,26 @@ def _source_file(token: str) -> str: def okp_methods(config: KcatAssignmentConfig) -> list[str]: """The OKP methods of the ``okp`` source group (``[kcat_sources.okp] methods``).""" methods = config.sources.get("okp", {}).get("methods") or list(DEFAULT_OKP_METHODS) - unknown = [m for m in methods if m not in OKP_METHODS] - if unknown: - raise ValueError(f"Unknown OKP methods in [kcat_sources.okp] methods: {unknown}; " - f"methods: {', '.join(OKP_METHODS)}.") + check_okp_methods(methods) return list(methods) +def brenda_aggregation(adapter: "ModelAdapter") -> str: + """The BRENDA aggregation of the adapter: ``[kcat_sources.brenda] aggregation``. + + As the kcat assignment config resolves it: the preset's value (median for + ``brenda_ec_tree`` and the other median presets, max for ``gecko_classic``) + unless the file sets it. ``max`` when neither the preset nor the file sets it + (the ``okp`` preset, or stages without a ``[kcat_sources.brenda]`` table). + """ + return str(kcat_assignment_config(adapter).sources.get("brenda", {}).get("aggregation", "max")) + + +def default_okp_method(adapter: "ModelAdapter") -> str: + """The OKP method used when none is given: the first of ``[kcat_sources.okp] methods``.""" + return okp_methods(kcat_assignment_config(adapter))[0] + + def _expand(tokens: list[str], config: KcatAssignmentConfig) -> list[str]: """Stage tokens with the ``okp`` group replaced by its methods' files.""" out: list[str] = [] @@ -409,15 +414,6 @@ def assign_kcats( return decisions, report -def _okp_method(name: str, wanted: dict[str, Any]) -> str: - if "method" in wanted: - return str(wanted["method"]) - for m in OKP_METHODS: - if "okp_" + re.sub(r"[^A-Za-z0-9]+", "_", m).lower() == name: - return m - raise ValueError(f'Set the OKP method of {name} in model_adapter.toml: [kcat_sources.{name}] method = "..."') - - def _same(a: Any, b: Any) -> bool: def norm(v: Any) -> Any: return v.lower() if isinstance(v, str) else (int(v) if isinstance(v, bool) else v) @@ -508,7 +504,7 @@ def _gather(model: "EcModel", name: str, wanted: dict[str, Any], adapter: "Model if not done: return "pending", "OKP job for DLKcat still running; call again later" else: - method = _okp_method(name, wanted) + method = okp_method_of_file(name) _, done = kcats_from_okp(model, method=method, save_as=name, adapter=adapter) if not done: return "pending", f"OKP job for {method} still running; call again later" diff --git a/src/geckopy/gather_kcats/kcat_sources.py b/src/geckopy/gather_kcats/kcat_sources.py index 2731024..246a70f 100644 --- a/src/geckopy/gather_kcats/kcat_sources.py +++ b/src/geckopy/gather_kcats/kcat_sources.py @@ -40,6 +40,48 @@ _NUMBER_RE = re.compile(r"^[-+]?([0-9]+\.?[0-9]*|\.[0-9]+)([eE][-+]?[0-9]+)?$|^[-+]?Inf$") _META_RE = re.compile(r"^\s*#\s*([A-Za-z][A-Za-z0-9_]{0,62})\s*:\s?(.*)$", re.ASCII) +#: The OKP methods that ``[kcat_sources.okp] methods`` and ``kcats_from_okp`` accept. +#: The reaction methods CatPred and TurNup are left out: they read whole reactions +#: (substrate set, and products for TurNup), while ``kcats_from_okp`` submits +#: enzyme-substrate pairs. +OKP_METHODS = ("CataPro", "CatRange", "DLKcat", "EITLEM", "KinForm-H", "KinForm-L", + "OmniESI", "OmniESI-O2DENet", "UniKP") +#: The OKP methods of the ``okp`` source group by default; a project lists others in +#: ``[kcat_sources.okp] methods`` (those that have combination parameters: not +#: CatRange, which predicts classes). +DEFAULT_OKP_METHODS = ("KinForm-H",) + + +def okp_file(method: str) -> str: + """The source file name of an OKP method, e.g. ``okp_kinform_h``. + + ``okp_`` and the method in lower case, with each run of characters other + than letters and digits as ``_`` and no ``_`` at either end. + """ + return "okp_" + re.sub(r"[^A-Za-z0-9]+", "_", method).lower().strip("_") + + +def check_okp_methods(methods: list[str]) -> None: + """Raise ``ValueError`` naming the entries of ``methods`` that are no known OKP method. + + The names are in the order given, each once. + """ + unknown = list(dict.fromkeys(m for m in methods if m not in OKP_METHODS)) + if unknown: + raise ValueError(f"Unknown OKP methods in [kcat_sources.okp] methods: {', '.join(unknown)}; " + f"methods: {', '.join(OKP_METHODS)}.") + + +def okp_method_of_file(name: str) -> str: + """The OKP method whose source file is ``name``, e.g. ``KinForm-H`` for ``okp_kinform_h``.""" + for method in OKP_METHODS: + if okp_file(method) == name: + return method + raise ValueError( + f"{name} is not the file of a known OKP method; the files are " + f"{', '.join(okp_file(m) for m in OKP_METHODS)}." + ) + def ec_rxn_keys(model: "EcModel") -> tuple[list[str], list[str]]: """Build-independent keys of the ``model.ec`` reactions. diff --git a/src/geckopy/gather_kcats/kcats_from.py b/src/geckopy/gather_kcats/kcats_from.py index 65e036b..a4f6433 100644 --- a/src/geckopy/gather_kcats/kcats_from.py +++ b/src/geckopy/gather_kcats/kcats_from.py @@ -22,7 +22,7 @@ from ..databases import load_brenda_data, load_brenda_measurements, load_dlkcat_ignore_lists, load_phyl_dist from .ec_tree import ECTree from .fuzzy_kcat_matching import fuzzy_kcat_matching -from .kcat_sources import format_number, kcat_source_file, save_kcat_source +from .kcat_sources import format_number, kcat_source_file, okp_file, save_kcat_source from .open_kinetics_predictor.run import _fetch_open_kinetics_predictor, _submit_open_kinetics_predictor from .open_kinetics_predictor.parse_output import _parse_okp_output from .read_dlkcat_output import _read_dlkcat_output @@ -91,7 +91,9 @@ def kcats_from_brenda( EcModel with EC numbers. aggregation Which BRENDA value per EC number, substrate and organism is used: - ``"max"`` or ``"median"`` (default: ``params.kcat_aggregate_brenda``). + ``"max"`` or ``"median"`` (default: ``[kcat_sources.brenda] aggregation`` + as the kcat assignment resolves it: median for the default preset + ``brenda_ec_tree``, max for ``gecko_classic``). force_wildcard_level Force a minimum wildcard level in the EC numbers. ec_rxns @@ -113,10 +115,12 @@ def kcats_from_brenda( """ adapter = _adapter(model, adapter, "kcats_from_brenda needs the BRENDA folder and params") if aggregation is None: - aggregation = adapter.params.kcat_aggregate_brenda + from .kcat_assignment import brenda_aggregation + + aggregation = brenda_aggregation(adapter) aggregation = aggregation.lower() if aggregation not in ("max", "median"): - raise ValueError("aggregation must be 'max' or 'median'") + raise ValueError("aggregation must be max or median") ec_rxns = _rxn_subset(ec_rxns) if save: path = kcat_source_file(save_as, model=model, adapter=adapter) # fails before the matching @@ -417,8 +421,8 @@ def kcats_from_okp( model EcModel with metabolite SMILES and protein sequences. method - OKP method, e.g. ``"CataPro"``, ``"CatRange"``, ``"UniKP"`` (default: - ``params.okp.method``). + OKP method, e.g. ``"CataPro"``, ``"CatRange"``, ``"UniKP"`` (default: the + first of ``[kcat_sources.okp] methods``, ``"KinForm-H"`` unless set). wait Wait until the job is done. resubmit @@ -442,16 +446,19 @@ def kcats_from_okp( """ adapter = _adapter(model, adapter, "kcats_from_okp needs params.path") if not method: - method = adapter.params.okp.method + from .kcat_assignment import default_okp_method + + method = default_okp_method(adapter) if method in ("CatPred", "TurNup"): raise ValueError(f"{method} reads whole reactions (substrate set{', and products' if method == 'TurNup' else ''}); " "kcats_from_okp submits enzyme-substrate pairs only.") ec_rxns = _rxn_subset(ec_rxns) - tag = re.sub(r"[^A-Za-z0-9]+", "_", method).lower() + name_default = okp_file(method) + tag = name_default[len("okp_"):] data_dir = Path(adapter.params.path) / "data" job_file = data_dir / f"OKP_job_{tag}.txt" out_file = data_dir / f"OKP_output_{tag}.csv" - name = save_as or f"okp_{tag}" + name = save_as or name_default job = _read_job_file(job_file) submitting = not use_stored and (resubmit or "jobId" not in job) if ec_rxns is not None and not submitting: diff --git a/src/geckopy/gather_kcats/open_kinetics_predictor/run.py b/src/geckopy/gather_kcats/open_kinetics_predictor/run.py index 1719d32..c4457bf 100644 --- a/src/geckopy/gather_kcats/open_kinetics_predictor/run.py +++ b/src/geckopy/gather_kcats/open_kinetics_predictor/run.py @@ -70,8 +70,8 @@ def _submit_open_kinetics_predictor( OKP API key. If omitted, resolved from ``OKP_API_KEY`` or ``data/okpApiKey.txt`` (see the module docstring). method - Predictor method. Resolved as: this arg -> ``params.okp.method`` - -> ``'KinForm-H'``. + Predictor method. Resolved as: this arg -> the first of + ``[kcat_sources.okp] methods`` (``'KinForm-H'`` unless set). ec_rxns Optional iterable of reaction IDs to include (default all). only_with_smiles @@ -96,7 +96,10 @@ def _submit_open_kinetics_predictor( ) data_dir = Path(adapter.params.path) / "data" api_key = _resolve_api_key(api_key, adapter) - method = method or adapter.params.okp.method + if not method: + from ..kcat_assignment import default_okp_method + + method = default_okp_method(adapter) okp = adapter.params.okp okp_csv = data_dir / "OKP.csv" diff --git a/src/geckopy/gather_kcats/select_kcat_value.py b/src/geckopy/gather_kcats/select_kcat_value.py index e61fcb1..53eaa0b 100644 --- a/src/geckopy/gather_kcats/select_kcat_value.py +++ b/src/geckopy/gather_kcats/select_kcat_value.py @@ -5,7 +5,7 @@ """ from __future__ import annotations -from typing import TYPE_CHECKING, Literal, Optional, Union +from typing import TYPE_CHECKING, Literal, Union import numpy as np import pandas as pd @@ -38,7 +38,7 @@ def apply_kcat_list( model: "EcModel", kcat_list: pd.DataFrame, *, - criteria: Optional[Literal["max", "min", "median", "mean"]] = None, + criteria: Literal["max", "min", "median", "mean"] = "max", overwrite: _OverwriteMode = True, ) -> list[str]: """Choose one kcat value per reaction and write it to ``model.ec.kcat``. @@ -81,10 +81,7 @@ def apply_kcat_list( ``criteria``. criteria How to aggregate multiple candidates per reaction: - ``"max"``, ``"min"``, ``"median"``, or ``"mean"``. ``None`` - (default) reads - ``model.adapter.params.kcat_aggregate_candidates`` if an adapter - is attached, falling back to ``"max"`` otherwise. + ``"max"`` (default), ``"min"``, ``"median"``, or ``"mean"``. overwrite ``True`` (default) overwrites unconditionally. ``False`` only fills entries currently 0 or NaN. ``"if_higher"`` only @@ -105,12 +102,6 @@ def apply_kcat_list( If ``kcat_list`` is missing the ``rxn_id``, ``kcat``, or ``source`` column. """ - if criteria is None: - adapter = getattr(model, "adapter", None) - criteria = ( - adapter.params.kcat_aggregate_candidates - if adapter is not None else "max" - ) if criteria not in _CRITERIA: raise ValueError( f"criteria must be one of {_CRITERIA}; got {criteria!r}" diff --git a/src/geckopy/kcat_tuning/evotune/mask_size.py b/src/geckopy/kcat_tuning/evotune/mask_size.py index 65cd6d7..76f0ee8 100644 --- a/src/geckopy/kcat_tuning/evotune/mask_size.py +++ b/src/geckopy/kcat_tuning/evotune/mask_size.py @@ -95,7 +95,6 @@ def sweep_tunable_mask_size( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional["EvotuneData"] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -157,7 +156,7 @@ def sweep_tunable_mask_size( screen A precomputed :func:`~.tuning.screen_kcat_leverage` report. Computed once, fresh, if not given. - adapter, params, evotune_data, okp_method, bio_rxn, make_anaerobic, + adapter, params, evotune_data, bio_rxn, make_anaerobic, change_protein_biomass, n_proc, verbose Forwarded to :func:`~.tuning.screen_kcat_leverage` and :func:`~.tuning.cmaes_kcat_tuning`; see their docstrings. @@ -167,13 +166,13 @@ def sweep_tunable_mask_size( list of :class:`MaskSizePoint`, one per share, sorted by ``target_impact_share`` ascending. """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) if screen is None: screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, n_proc=n_proc, ) @@ -187,7 +186,7 @@ def sweep_tunable_mask_size( run_model = model.copy() result = cmaes_kcat_tuning( run_model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, tunable_mask=mask, n_proc=n_proc, seed=seed, verbose=False, ) diff --git a/src/geckopy/kcat_tuning/evotune/priors.py b/src/geckopy/kcat_tuning/evotune/priors.py index 6ae356e..7c2372f 100644 --- a/src/geckopy/kcat_tuning/evotune/priors.py +++ b/src/geckopy/kcat_tuning/evotune/priors.py @@ -2,31 +2,32 @@ """ from __future__ import annotations -from typing import Optional +import logging +import warnings +from collections import Counter import numpy as np from ...adapter.params import EvotuneParams from ...gather_kcats.kcat_source_format import parse_kcat_source +logger = logging.getLogger(__name__) + #: Sentinel group name for an ``ec.source`` value matched by no #: ``source_groups`` entry -- MATLAB's ``noKcatSource``. UNLABELLED_GROUP = "unlabelled" -def classify_kcat_source( - source: str, - params: EvotuneParams, - *, - okp_method: Optional[str] = None, -) -> str: +def classify_kcat_source(source: str, params: EvotuneParams) -> str: """Map one ``ec.source`` string to a trust-tier group name. - Matching is case-insensitive, mirroring MATLAB's - ``strcmpi(ecModel.ec.source, kcatSources{i})``. A ``source_groups`` entry - matches the origin of the source (``"brenda"`` for + Matching is case-insensitive. An entry of ``sources`` without a trailing + ``*`` matches the origin of the source (``"brenda"`` for ``"brenda (wc=0, origin=1) | tuned=cmaes (prior=2)"``, so the detail and - the tuning status are ignored) and also the whole source string. + the tuning status are ignored) or the whole source string; these exact + matches are tried over all groups first. Then the entries ending in + ``*``, prefix patterns on the origin, are tried, the longest prefix first. + Entries equal in length keep the order of the groups. Parameters ---------- @@ -34,76 +35,81 @@ def classify_kcat_source( One ``ec.source`` entry. params Supplies ``source_groups``. - okp_method - The project's configured OpenKineticsPredictor method - (``OkpParams.method``), if any. A group with ``match_okp=True`` - also matches ``source`` when it equals this (case-insensitive). Returns ------- str A key of ``params.source_groups``, or :data:`UNLABELLED_GROUP` if nothing matches -- callers then fall back to - ``params.sigma0_log_default`` / ``shrink_thr_default`` / - ``force_prior_thr_default``. + ``params.sigma0_log_default``. """ source_lower = source.lower() - token = parse_kcat_source(source).origin.lower() - forms = {source_lower, token} - for name, rule in params.source_groups.items(): - if any(s.lower() in forms for s in rule.sources): - return name - if ( - rule.match_okp - and okp_method is not None - and okp_method.lower() in forms - ): + origin = parse_kcat_source(source).origin.lower() + forms = {source_lower, origin} + patterns: list[tuple[int, int, str, str]] = [] + for position, (name, rule) in enumerate(params.source_groups.items()): + for entry in rule.sources: + entry = entry.lower() + if entry.endswith("*"): + patterns.append((-(len(entry) - 1), position, name, entry[:-1])) + elif entry in forms: + return name + for _, _, name, prefix in sorted(patterns): + if origin.startswith(prefix): return name return UNLABELLED_GROUP -def classify_kcat_sources( - ec_sources: list[str], - params: EvotuneParams, - *, - okp_method: Optional[str] = None, -) -> np.ndarray: +def classify_kcat_sources(ec_sources: list[str], params: EvotuneParams) -> np.ndarray: """Vectorised :func:`classify_kcat_source` over ``ec.source``. Returns ------- numpy.ndarray of str, shape ``(len(ec_sources),)`` """ - return np.array( - [ - classify_kcat_source(s, params, okp_method=okp_method) - for s in ec_sources - ], - dtype=object, - ) + return np.array([classify_kcat_source(s, params) for s in ec_sources], dtype=object) def build_sigma0_log(groups: np.ndarray, params: EvotuneParams) -> np.ndarray: """Per-row prior std dev in log-space, from source-group membership. - Mirrors MATLAB's ``sigma0log = sigma0logDefault * ones(...); - sigma0log(uniqKcatParams) = sigma0logSource(kcatSourceIdx(...))``: - every row starts at ``params.sigma0_log_default``, then rows whose - group is a real (non-:data:`UNLABELLED_GROUP`) entry get - overridden by that group's ``sigma0_log_source`` value. + Every row starts at ``params.sigma0_log_default``, then rows whose + group is a real (non-:data:`UNLABELLED_GROUP`) entry get that group's + ``sigma0_log``, a standard deviation of the natural log of the kcat. Parameters ---------- groups Per-row group names from :func:`classify_kcat_sources`. params - Supplies ``sigma0_log_default`` and ``sigma0_log_source``. + Supplies ``sigma0_log_default`` and the groups' ``sigma0_log``. Returns ------- numpy.ndarray of float, same shape as ``groups``. """ sigma0_log = np.full(len(groups), params.sigma0_log_default, dtype=float) - for name, value in params.sigma0_log_source.items(): - sigma0_log[groups == name] = value + for name, rule in params.source_groups.items(): + sigma0_log[groups == name] = rule.sigma0_log return sigma0_log + + +def warn_unlabelled( + groups: np.ndarray, sources: list[str], params: EvotuneParams, *, max_fraction: float = 0.2, +) -> None: + """Warn when more than ``max_fraction`` of the tunable kcats are in no group. + + The kcats then take ``params.sigma0_log_default``. The warning names the + most common origins (up to five) among them, by count descending and then by name. + """ + unlabelled = [s for s, g in zip(sources, groups) if g == UNLABELLED_GROUP] + if len(groups) == 0 or len(unlabelled) <= max_fraction * len(groups): + return + counts = Counter(parse_kcat_source(s).origin for s in unlabelled) + origins = sorted(counts, key=lambda o: (-counts[o], o))[:5] + message = ( + f"{len(unlabelled)} of {len(groups)} tunable kcats match no evotune source group and get " + f"sigma0_log_default ({params.sigma0_log_default:g}); origins: {', '.join(origins)}" + ) + logger.warning(message) + warnings.warn(message, UserWarning, stacklevel=2) diff --git a/src/geckopy/kcat_tuning/evotune/tuning.py b/src/geckopy/kcat_tuning/evotune/tuning.py index a5efd9d..a7ae005 100644 --- a/src/geckopy/kcat_tuning/evotune/tuning.py +++ b/src/geckopy/kcat_tuning/evotune/tuning.py @@ -74,7 +74,7 @@ from .data import EvotuneData, load_evotune_data from .distance import evotune_distance, compute_excarbon from .parsimony import fold_change, n_changed -from .priors import build_sigma0_log, classify_kcat_sources +from .priors import build_sigma0_log, classify_kcat_sources, warn_unlabelled from .simulate import simulate_evotune_dataset from .tying import isozyme_tie_map @@ -159,7 +159,7 @@ def corrections( ) -def _resolve_context(model, adapter, params, evotune_data, bio_rxn, okp_method): +def _resolve_context(model, adapter, params, evotune_data, bio_rxn): """Resolve every optional input against ``model``'s adapter, or raise if it can't be and none was given explicitly.""" if adapter is None: @@ -186,9 +186,7 @@ def _resolve_context(model, adapter, params, evotune_data, bio_rxn, okp_method): if adapter is None: raise ValueError("bio_rxn not given and model has no adapter.") bio_rxn = adapter.params.bio_rxn - if okp_method is None and adapter is not None: - okp_method = adapter.params.okp.method - return params, evotune_data, bio_rxn, okp_method + return params, evotune_data, bio_rxn #: Termination criteria of ``cma`` that a flat objective triggers. @@ -196,7 +194,7 @@ def _resolve_context(model, adapter, params, evotune_data, bio_rxn, okp_method): def _tunable_context( - model, params, evotune_data, bio_rxn, okp_method, tunable_mask, + model, params, evotune_data, bio_rxn, tunable_mask, ): """The tunable subset plus everything derived from it: tie groups, trust weights, and the excarbon table. Shared by every function in @@ -221,7 +219,8 @@ def _tunable_context( ec_rxn_ids_tunable = [model.ec.rxns[i] for i in tunable_idx] kcat0 = model.ec.kcat[tunable_idx].astype(float).copy() sources = [model.ec.source[i] for i in tunable_idx] - groups = classify_kcat_sources(sources, params, okp_method=okp_method) + groups = classify_kcat_sources(sources, params) + warn_unlabelled(groups, sources, params) sigma0_log = build_sigma0_log(groups, params) excarbon_rxn_ids: set[str] = {bio_rxn} @@ -245,7 +244,6 @@ def screen_kcat_leverage( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional[EvotuneData] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -292,11 +290,11 @@ def screen_kcat_leverage( :func:`select_tunable_mask` needs to expand a row back into a mask; drop it before showing the table to a person. """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) (tunable_idx, ec_rxn_ids_tunable, kcat0, groups, sigma0_log, excarbon, tie_map) = _tunable_context( - model, params, evotune_data, bio_rxn, okp_method, tunable_mask) + model, params, evotune_data, bio_rxn, tunable_mask) reps = np.flatnonzero(tie_map == np.arange(len(tie_map))) members_of = {int(r): np.flatnonzero(tie_map == r) for r in reps} @@ -406,7 +404,6 @@ def cmaes_kcat_tuning( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional[EvotuneData] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -485,14 +482,14 @@ def cmaes_kcat_tuning( parameters remain after masking and tying -- too little for CMA-ES to search over. """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) if tunable_mask is None: if screen is None: screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, n_proc=n_proc, @@ -502,7 +499,7 @@ def cmaes_kcat_tuning( (tunable_idx, ec_rxn_ids_tunable, kcat0, groups, sigma0_log, excarbon, tie_map) = _tunable_context( - model, params, evotune_data, bio_rxn, okp_method, tunable_mask) + model, params, evotune_data, bio_rxn, tunable_mask) reps = np.flatnonzero(tie_map == np.arange(len(tie_map))) if len(reps) < 2: @@ -633,7 +630,6 @@ def tune_prior_penalty_weight( adapter: Optional["ModelAdapter"] = None, params: Optional["EvotuneParams"] = None, evotune_data: Optional[EvotuneData] = None, - okp_method: Optional[str] = None, bio_rxn: Optional[str] = None, make_anaerobic: Optional[Callable[["EcModel"], None]] = None, change_protein_biomass: Optional[Callable[["EcModel", float], None]] = None, @@ -687,7 +683,7 @@ def tune_prior_penalty_weight( A kcat counts as "moved" past this fold change from its prior. Matches :func:`~.parsimony.fold_change`'s convention. tunable_mask, screen, target_impact_share, popsize, n_proc, - make_anaerobic, change_protein_biomass, okp_method, bio_rxn + make_anaerobic, change_protein_biomass, bio_rxn As in :func:`cmaes_kcat_tuning`. Returns @@ -704,14 +700,14 @@ def tune_prior_penalty_weight( ``median_fold_spread`` and ``max_fold_spread`` (among movers, the largest fold-change between what the two seeds landed on). """ - params, evotune_data, bio_rxn, okp_method = _resolve_context( - model, adapter, params, evotune_data, bio_rxn, okp_method) + params, evotune_data, bio_rxn = _resolve_context( + model, adapter, params, evotune_data, bio_rxn) if tunable_mask is None: if screen is None: screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, n_proc=n_proc, @@ -738,7 +734,7 @@ def _reset(): _reset() result = cmaes_kcat_tuning( model, adapter=adapter, params=lam_params, evotune_data=evotune_data, - okp_method=okp_method, bio_rxn=bio_rxn, + bio_rxn=bio_rxn, make_anaerobic=make_anaerobic, change_protein_biomass=change_protein_biomass, tunable_mask=tunable_mask, popsize=popsize, n_proc=n_proc, diff --git a/src/geckopy/kcat_tuning/sensitivity_tuning.py b/src/geckopy/kcat_tuning/sensitivity_tuning.py index acf18ee..a74e6b1 100644 --- a/src/geckopy/kcat_tuning/sensitivity_tuning.py +++ b/src/geckopy/kcat_tuning/sensitivity_tuning.py @@ -165,7 +165,7 @@ def sensitivity_tuning( installed and a populated ``ec.kcat``. Mutated in place. desired_growth_rate Target growth rate. Defaults to - ``model.adapter.params.gr_exp``. + ``model.adapter.params.gr_exp``; raises ``ValueError`` when neither is set. fold_change Per-iteration kcat multiplier. prot_to_ignore @@ -198,18 +198,12 @@ def sensitivity_tuning( light = bool(model.ec.gecko_light) if desired_growth_rate is None: - from ..adapter import resolve_adapter - adapter = resolve_adapter( + from ..adapter import resolve_gr_exp + desired_growth_rate = resolve_gr_exp( model, purpose="sensitivity_tuning needs `desired_growth_rate` " - "(pass it explicitly, or rely on params.gr_exp from the adapter)", + "(pass it explicitly, or set gr_exp in the adapter)", ) - if adapter.params.gr_exp is None: - raise ValueError( - "desired_growth_rate not provided and " - "model.adapter.params.gr_exp is unset." - ) - desired_growth_rate = float(adapter.params.gr_exp) if prot_to_ignore is None: prot_to_ignore = [] diff --git a/src/geckopy/kcat_tuning/sigma_fitter.py b/src/geckopy/kcat_tuning/sigma_fitter.py index e2aa0b7..c47ecb5 100644 --- a/src/geckopy/kcat_tuning/sigma_fitter.py +++ b/src/geckopy/kcat_tuning/sigma_fitter.py @@ -130,7 +130,7 @@ def fit_sigma( return. growth_rate Experimental growth rate to match. Defaults to - ``model.adapter.params.gr_exp``. + ``model.adapter.params.gr_exp``; raises ``ValueError`` when neither is set. p_tot Total protein content (g/gDCW). Defaults to ``model.adapter.params.p_tot``. @@ -174,7 +174,9 @@ def fit_sigma( params = model.adapter.params if model.adapter is not None else None if growth_rate is None: - growth_rate = float(params.gr_exp) + from ..adapter import resolve_gr_exp + + growth_rate = resolve_gr_exp(model) if not growth_rate > 0: # also NaN raise ValueError(f"growth_rate is {growth_rate}; fit_sigma needs a target growth rate above 0.") if p_tot is None: diff --git a/src/geckopy/limit_proteins/flexibilize_enz_concs.py b/src/geckopy/limit_proteins/flexibilize_enz_concs.py index 9f374bc..5ee7b8d 100644 --- a/src/geckopy/limit_proteins/flexibilize_enz_concs.py +++ b/src/geckopy/limit_proteins/flexibilize_enz_concs.py @@ -150,7 +150,7 @@ def flexibilize_enz_concs( itself is left unchanged. exp_growth Target growth rate. Defaults to - ``model.adapter.params.gr_exp``. + ``model.adapter.params.gr_exp``; raises ``ValueError`` when neither is set. fold_change Controls how much the limiting enzyme's usage upper bound grows each time it is selected: on its ``k``-th selection, the @@ -188,18 +188,12 @@ def flexibilize_enz_concs( flexibilize), or if ``model.adapter`` is None and ``exp_growth`` is also None. """ - from ..adapter import resolve_param - if exp_growth is None: - exp_growth = resolve_param( - model, None, "gr_exp", - purpose="flexibilize_enz_concs needs `exp_growth` (pass it " - "explicitly, or rely on params.gr_exp from the adapter)", - ) - if exp_growth is None: - raise ValueError( - "exp_growth not provided and params.gr_exp is unset." - ) - exp_growth = float(exp_growth) + from ..adapter import resolve_gr_exp, resolve_param + exp_growth = resolve_gr_exp( + model, exp_growth, + purpose="flexibilize_enz_concs needs `exp_growth` (pass it " + "explicitly, or set gr_exp in the adapter)", + ) if iter_per_enzyme == 0: iter_per_enzyme = math.inf diff --git a/src/geckopy/limit_proteins/relax_enz_concs.py b/src/geckopy/limit_proteins/relax_enz_concs.py index 703eea8..794cd8f 100644 --- a/src/geckopy/limit_proteins/relax_enz_concs.py +++ b/src/geckopy/limit_proteins/relax_enz_concs.py @@ -60,7 +60,7 @@ def relax_enz_concs( ``ec.concs`` itself is left unchanged, as in ``flexibilize_enz_concs``. exp_growth - Target growth rate. Defaults to ``params.gr_exp``. + Target growth rate. Defaults to ``params.gr_exp``; raises ``ValueError`` when neither is set. bio_rxn Biomass reaction id. Defaults to ``params.bio_rxn``. per_complex @@ -95,15 +95,11 @@ def relax_enz_concs( """ from optlang.symbolics import add - from ..adapter import resolve_param + from ..adapter import resolve_gr_exp, resolve_param if weight not in ("relative", "absolute"): raise ValueError(f"weight must be 'relative' or 'absolute', not {weight!r}") - if exp_growth is None: - exp_growth = resolve_param(model, None, "gr_exp", purpose="relax_enz_concs needs `exp_growth`") - if exp_growth is None: - raise ValueError("exp_growth not provided and params.gr_exp is unset.") - exp_growth = float(exp_growth) + exp_growth = resolve_gr_exp(model, exp_growth, purpose="relax_enz_concs needs `exp_growth`") bio = model.reactions.get_by_id( resolve_param(model, bio_rxn, "bio_rxn", purpose="relax_enz_concs needs the biomass reaction id") ) diff --git a/tests/test_adapter.py b/tests/test_adapter.py index eae193b..ba8c4e5 100644 --- a/tests/test_adapter.py +++ b/tests/test_adapter.py @@ -70,7 +70,7 @@ def test_missing_toml_raises(tmp_path): def test_unknown_field_rejected(tmp_path): bad_toml = MINIMAL_TOML + '\nsgma = 0.3\n' # typo folder = _write_adapter_folder(tmp_path, bad_toml) - with pytest.raises(ValidationError): + with pytest.raises(ValueError, match="unknown key sgma in model_adapter.toml"): ModelAdapter.from_folder(folder) @@ -187,13 +187,11 @@ def test_spontaneous_reactions_from_toml(tmp_path): [evotune.source_groups.dlkcat] sources = ["dlkcat"] +sigma0_log = 0.4 [evotune.source_groups.brenda] sources = ["brenda"] - -[evotune.sigma0_log_source] -dlkcat = 0.4 -brenda = 0.2 +sigma0_log = 0.2 [matlab] make_model_anaerobic = "anaerobicModel_GECKO" @@ -268,7 +266,218 @@ def test_uniprot_table_lookup_reads_quoted_cells_and_first_duplicate(tmp_path): assert result == ["P12345", "P67890"] -def test_default_okp_method_is_kinform_h(): - from geckopy.adapter.params import ModelParameters +def test_okp_section_has_no_method(): + from geckopy.adapter.params import OkpParams + + assert "method" not in OkpParams.model_fields + + +@pytest.mark.parametrize("extra, key", [ + ("typo = 1\n", "typo"), + ("[kegg]\nidd = \"sce\"\n", "kegg.idd"), + ("[okp]\nmethod = \"CataPro\"\n", "okp.method"), + ("[evotune]\nmatch_okp = true\n", "evotune.match_okp"), + ("[evotune.source_groups.brenda]\nsources = [\"brenda\"]\nsigma0_log = 0.2\nmatch_okp = true\n", + "evotune.source_groups.brenda.match_okp"), + ("[evotune.sigma0_log_source]\nbrenda = 0.2\n", "evotune.sigma0_log_source"), + ("[[kcat_assignment.stage]]\nsources = [\"custom\"]\ncriterion = \"max\"\n", "kcat_assignment.stage.criterion"), + ("[kcat_sources.brenda]\naggregate = \"max\"\n", "kcat_sources.brenda.aggregate"), + ("[kcat_sources.okp_kinform_h]\nmethod = \"KinForm-H\"\n", "kcat_sources.okp_kinform_h.method"), + ("[kcat_sources.nosuchsource]\nk = 5\n", "kcat_sources.nosuchsource"), + ("[matlab]\nmake_model_anaerobicc = \"f\"\n", "matlab.make_model_anaerobicc"), + ("kcat_aggregate_brenda = \"max\"\n", "kcat_aggregate_brenda"), +]) +def test_an_unknown_key_is_an_error_naming_file_and_key(tmp_path, extra, key): + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + extra) + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == f"{folder.resolve() / 'model_adapter.toml'}: unknown key {key} in model_adapter.toml" + + +def test_known_kcat_source_keys_and_free_matlab_params_load(tmp_path): + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + """\ +[kcat_sources.brenda] +aggregation = "max" +force_wildcard_level = 1 +[kcat_sources.ec_tree] +k = 5 +[kcat_sources.homologs] +k = 5 +min_identity = 0.4 +coverage = 0.5 +[kcat_sources.okp] +methods = ["KinForm-H"] +[kcat_sources.okp_kinform_h] +[kcat_sources.custom] +[kcat_sources.dlkcat] +[matlab.params] +anything = 1 +""") + assert ModelAdapter.from_folder(folder).params.kcat_sources["homologs"]["coverage"] == 0.5 + + +SHOW_TOML = MINIMAL_TOML + """\ +gr_exp = 0.5 + +[kcat_sources.brenda] +aggregation = "max" + +[evotune.source_groups.okp] +sources = ["okp_*"] +sigma0_log = 0.25 +""" + - assert ModelParameters.model_fields["okp"].get_default(call_default_factory=True).method == "KinForm-H" +def test_show_prints_one_line_per_item_in_the_agreed_format(tmp_path, capsys): + from geckopy.adapter import show_adapter_params + + show_adapter_params(ModelAdapter.from_folder(_write_adapter_folder(tmp_path, SHOW_TOML))) + lines = capsys.readouterr().out.splitlines() + assert lines[0] == "project_format = [not set]" + for expected in [ + "org_name = Testus organismus [file]", + "gr_exp = 0.5 [file]", + "sigma = 0.5 [default]", + "uniprot.reviewed = false [default]", + "kcat_assignment.preset = brenda_ec_tree [default]", + "kcat_assignment.isozymes = true [default]", + "kcat_assignment.stage[1] = select: custom, database_top, ec_tree; top_origin_limit=1 [preset brenda_ec_tree]", + "kcat_assignment.stage[2] = combine: okp [preset brenda_ec_tree]", + "kcat_sources.brenda.aggregation = max [file]", + "kcat_sources.ec_tree.k = 5 [preset brenda_ec_tree]", + "kcat_sources.okp.methods = KinForm-H [default]", + "evotune.source_groups.okp = okp_*; sigma0_log 0.25 [file]", + ]: + assert expected in lines, expected + assert not any(ln.startswith("Adapter parameters") for ln in lines) + # Items in the order of the table; each table key is listed (stage, kcat_sources, groups expanded). + table = Path(__file__).resolve().parent.parent / "src" / "geckopy" / "adapter" / "adapterParameters.tsv" + expected_keys = [ln.split("\t")[0] for ln in table.read_text().splitlines()[1:]] + seen = [] + for ln in lines: + key = ln.split(" =")[0] + base = next((k for k in expected_keys if key == k or key.startswith(k + ".") or key.startswith(k + "[")), key) + if not seen or seen[-1] != base: + seen.append(base) + assert seen == expected_keys + + +def test_show_with_stages_and_file_settings(tmp_path, capsys): + from geckopy.adapter import show_adapter_params + + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + """[[kcat_assignment.stage]] +sources = ["custom", "database_top"] +top_origin_limit = 2 +criteria = "median" +[[kcat_assignment.stage]] +sources = ["okp"] +action = "combine" +""") + show_adapter_params(ModelAdapter.from_folder(folder)) + lines = capsys.readouterr().out.splitlines() + assert "kcat_assignment.preset = none (stages are given) [file]" in lines + assert "kcat_assignment.stage[1] = select: custom, database_top; criteria=median; top_origin_limit=2 [file]" in lines + assert "kcat_assignment.stage[2] = combine: okp [file]" in lines + + +def test_show_cli_prints_the_project_of_the_folder(tmp_path, capsys): + from geckopy.cli import main + + folder = _write_adapter_folder(tmp_path, SHOW_TOML) + assert main(["adapter", "show", str(folder)]) == 0 + out = capsys.readouterr().out + assert out.splitlines()[0] == "project_format = [not set]" + assert "gr_exp = 0.5 [file]" in out + + +def _group_toml(body: str) -> str: + return MINIMAL_TOML + "[evotune.source_groups.g]\n" + body + + +@pytest.mark.parametrize("toml, message", [ + ("[evotune.source_groups]\ng = 1\n", "evotune.source_groups.g must be a table with sources and sigma0_log"), + (_group_toml('sources = ["a"]\n'), "evotune.source_groups.g needs both sources and sigma0_log"), + (_group_toml("sigma0_log = 0.1\n"), "evotune.source_groups.g needs both sources and sigma0_log"), + (_group_toml('sources = "a"\nsigma0_log = 0.1\n'), "evotune.source_groups.g.sources must be a non-empty list of strings"), + (_group_toml('sources = []\nsigma0_log = 0.1\n'), "evotune.source_groups.g.sources must be a non-empty list of strings"), + (_group_toml('sources = ["a"]\nsigma0_log = inf\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a"]\nsigma0_log = nan\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a", 1]\nsigma0_log = 0.1\n'), "evotune.source_groups.g.sources must be a non-empty list of strings"), + (_group_toml('sources = ["a"]\nsigma0_log = "x"\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a"]\nsigma0_log = 0\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), + (_group_toml('sources = ["a"]\nsigma0_log = true\n'), "evotune.source_groups.g.sigma0_log must be a positive number"), +]) +def test_a_malformed_evotune_group_has_its_own_message(tmp_path, toml, message): + folder = _write_adapter_folder(tmp_path, toml) + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == f"{folder.resolve() / 'model_adapter.toml'}: {message}" + + +def test_okp_source_tables_are_limited_to_known_methods(tmp_path): + ok = _write_adapter_folder(tmp_path, MINIMAL_TOML + "[kcat_sources.okp_unikp]\n") + assert "okp_unikp" in ModelAdapter.from_folder(ok).params.kcat_sources + for name in ("okp_nosuchmethod", "okp_catpred"): + folder = tmp_path / name + folder.mkdir() + (folder / "model_adapter.toml").write_text(MINIMAL_TOML + f"[kcat_sources.{name}]\n") + with pytest.raises(ValueError, match=f"unknown key kcat_sources.{name} in model_adapter.toml"): + ModelAdapter.from_folder(folder) + + +@pytest.mark.parametrize("extra, message", [ + ("[kcat_sources.okp]\nmethods = \"KinForm-H\"\n", "kcat_sources.okp.methods must be a list of strings"), + ("[kcat_sources.okp]\nmethods = [1]\n", "kcat_sources.okp.methods must be a list of strings"), + ("kcat_sources.brenda = 1\n", "kcat_sources.brenda must be a table"), + ("kcat_sources.foo = 1\n", "unknown key kcat_sources.foo in model_adapter.toml"), + ("[kcat_sources.brenda]\naggregation = \"mean\"\n", "kcat_sources.brenda.aggregation must be max or median"), +]) +def test_kcat_source_values_are_checked_at_load(tmp_path, extra, message): + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + extra) + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == f"{folder.resolve() / 'model_adapter.toml'}: {message}" + + +def test_unknown_okp_methods_are_an_error_at_load_in_file_order_without_duplicates(tmp_path): + from geckopy.gather_kcats.kcat_sources import OKP_METHODS + + folder = _write_adapter_folder( + tmp_path, MINIMAL_TOML + '[kcat_sources.okp]\nmethods = ["Zed", "KinForm-H", "Alpha", "Zed", "TurNup"]\n') + with pytest.raises(ValueError) as err: + ModelAdapter.from_folder(folder) + assert str(err.value) == ( + "Unknown OKP methods in [kcat_sources.okp] methods: Zed, Alpha, TurNup; " + f"methods: {', '.join(OKP_METHODS)}.") + + +def test_show_number_formats_and_kcat_sources_order(tmp_path, capsys): + from geckopy.adapter import show_adapter_params + + folder = _write_adapter_folder(tmp_path, MINIMAL_TOML + """\ +sigma = 1e6 +[complex] +taxonomic_id = 5592920 +[kcat_sources.okp] +methods = ["UniKP"] +[kcat_sources.ec_tree] +k = 3 +[kcat_sources.brenda] +force_wildcard_level = 1 +aggregation = "median" +[kcat_assignment] +preset = "gecko_classic" +""") + show_adapter_params(ModelAdapter.from_folder(folder)) + lines = capsys.readouterr().out.splitlines() + assert "sigma = 1000000 [file]" in lines + assert "complex.taxonomic_id = 5592920 [file]" in lines + assert ("kcat_assignment.stage[1] = select: custom, database_top, dlkcat, database_bottom; " + "top_origin_limit=6 [preset gecko_classic]") in lines + ks = [ln for ln in lines if ln.startswith("kcat_sources.")] + assert ks == [ + "kcat_sources.brenda.aggregation = median [file]", + "kcat_sources.brenda.force_wildcard_level = 1 [file]", + "kcat_sources.ec_tree.k = 3 [file]", + "kcat_sources.okp.methods = UniKP [file]", + ] diff --git a/tests/test_cli.py b/tests/test_cli.py index f52c6fb..7c8fe1d 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -25,16 +25,17 @@ def test_template_includes_nested_sections(): assert "[evotune]" in text -def test_template_renders_dict_valued_evotune_fields(): - """The evotune section carries per-source tables, not just scalars.""" +def test_template_renders_source_groups_as_tables(): + """The evotune section lists the default groups as one table each.""" text = generate_template_toml() uncommented = "\n".join( - line.lstrip("# ") for line in text.splitlines() - if line.lstrip("# ").startswith(("sigma0_log_source", "source_groups")) + line[2:] for line in text.splitlines() + if line.startswith("# [evotune.source_groups.") or line.startswith("# sources =") or line.startswith("# sigma0_log =") ) - parsed = tomllib.loads(uncommented) - assert parsed["sigma0_log_source"]["brenda"] == 0.2 - assert parsed["source_groups"]["brenda"]["sources"] == ["brenda"] + parsed = tomllib.loads(uncommented)["evotune"]["source_groups"] + assert parsed["brenda"] == {"sources": ["brenda"], "sigma0_log": 0.2} + assert list(parsed) == ["custom", "brenda", "ec_tree", "combined"] + assert "sigma0_log_source" not in text and "match_okp" not in text def test_template_includes_field_descriptions_as_comments(): @@ -195,3 +196,10 @@ def fake_download(kid, path, *, gene_id_field): assert calls["args"][0] == "sce" assert calls["args"][2] == "kegg" assert out.exists() + + +def test_template_lists_the_kcat_source_keys_after_the_sections(): + text = generate_template_toml() + assert text.index("# [kcat_sources.brenda]") > text.index("# [evotune.source_groups.combined]") + for key in ("aggregation", "force_wildcard_level", "min_identity", "coverage", "methods"): + assert f"# {key} = " in text diff --git a/tests/test_evotune.py b/tests/test_evotune.py index 0b0b845..71f2fca 100644 --- a/tests/test_evotune.py +++ b/tests/test_evotune.py @@ -18,7 +18,7 @@ import cobra from geckopy import EcModel, ModelAdapter -from geckopy.adapter.params import EvotuneParams +from geckopy.adapter.params import EvotuneParams, SourceGroupRule from geckopy.databases.flux_data import FluxData from geckopy.ec_model.ec_data import EcData from geckopy.kcat_tuning.evotune.tuning import ( @@ -30,6 +30,12 @@ from geckopy.kcat_tuning.evotune.data import EvotuneData from geckopy.kcat_tuning.evotune.tuning import EvotuneResult +GROUPS = { + "dlkcat": SourceGroupRule(sources=["dlkcat"], sigma0_log=0.4), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "custom": SourceGroupRule(sources=["custom"], sigma0_log=0.1), +} + _TRUE_KCAT = 2.0 _START_KCAT = 1.0 _MW = 100.0 @@ -271,7 +277,7 @@ def test_cmaes_tuning_moves_kcats_toward_truth(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=15, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=15, rmse_threshold=1e-6) before = model.ec.kcat.copy() result = cmaes_kcat_tuning( @@ -295,7 +301,7 @@ def test_cmaes_tuning_moves_kcats_toward_truth(tmp_path): def test_cmaes_tuning_extends_the_source_of_changed_kcats(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) - params = EvotuneParams(max_generations=15, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=15, rmse_threshold=1e-6) before = model.ec.kcat.copy() origins = ["brenda", "dlkcat"] @@ -320,7 +326,7 @@ def test_cmaes_tuning_extends_the_source_of_changed_kcats(tmp_path): def test_cmaes_tuning_n_proc_matches_serial(tmp_path): - params = EvotuneParams(max_generations=5, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=5, rmse_threshold=-1.0) def _run(n_proc): adapter = _adapter(tmp_path) @@ -346,7 +352,7 @@ def test_tie_isozymes_forces_equal_values(tmp_path): adapter = _adapter(tmp_path) model = _build_tied_toy(adapter) evotune_data = _tied_evotune_data() - params = EvotuneParams(max_generations=8, rmse_threshold=-1.0, + params = EvotuneParams(source_groups=GROUPS, max_generations=8, rmse_threshold=-1.0, tie_isozymes=True) screen = screen_kcat_leverage( @@ -368,7 +374,7 @@ def test_tie_isozymes_false_keeps_groups_separate(tmp_path): adapter = _adapter(tmp_path) model = _build_tied_toy(adapter) evotune_data = _tied_evotune_data() - params = EvotuneParams(tie_isozymes=False) + params = EvotuneParams(source_groups=GROUPS, tie_isozymes=False) screen = screen_kcat_leverage( model, adapter=adapter, params=params, evotune_data=evotune_data, n_proc=1, @@ -402,7 +408,7 @@ def test_tune_prior_penalty_weight_shape_and_fit_cost(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=6, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=6, rmse_threshold=-1.0) report = tune_prior_penalty_weight( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -427,7 +433,7 @@ def test_tune_prior_penalty_weight_restores_model(tmp_path): evotune_data = _evotune_data() before = model.ec.kcat.copy() source_before = list(model.ec.source) - params = EvotuneParams(max_generations=4, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=4, rmse_threshold=-1.0) tune_prior_penalty_weight( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -444,7 +450,7 @@ def test_tune_prior_penalty_weight_single_seed_gives_nan_reproducibility(tmp_pat adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=4, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=4, rmse_threshold=-1.0) report = tune_prior_penalty_weight( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -493,7 +499,7 @@ def test_cmaes_tuning_runs_all_generations_on_a_flat_objective(tmp_path, monkeyp monkeypatch.setattr(tuning, "_score_kcat_vector", lambda *a, **k: (1.0, 1.0)) adapter = _adapter(tmp_path) model = _build_toy(adapter) - params = EvotuneParams(max_generations=6, rmse_threshold=-1.0) + params = EvotuneParams(source_groups=GROUPS, max_generations=6, rmse_threshold=-1.0) result = cmaes_kcat_tuning( model, adapter=adapter, params=params, evotune_data=_evotune_data(), n_proc=1, seed=0, verbose=False, @@ -510,3 +516,26 @@ def test_screen_warns_when_the_model_has_no_formulas(tmp_path): screen_kcat_leverage( model, adapter=adapter, evotune_data=_evotune_data(), n_proc=1, ) + + +def test_default_groups_label_kcats_by_group_name_through_the_tuning_context(tmp_path, caplog): + """With default EvotuneParams, combined and ec_tree kcats get their group's width, + the group name labels the kcat, and the unlabelled warning fires for kcats in no group.""" + from geckopy.kcat_tuning.evotune.tuning import _tunable_context + + model = _build_toy(_adapter(tmp_path)) + model.ec.source = ["combined (sd=0.10, n=2)", "ec_tree"] + with caplog.at_level("WARNING"): + out = _tunable_context(model, EvotuneParams(), _evotune_data(), "biomass", None) + groups, sigma0_log = out[3], out[4] + assert list(groups) == ["combined", "ec_tree"] + assert list(sigma0_log) == [0.4, 0.3] + assert "match no evotune source group" not in caplog.text + + model.ec.source = ["standard", "standard"] + caplog.clear() + with caplog.at_level("WARNING"): + out = _tunable_context(model, EvotuneParams(), _evotune_data(), "biomass", None) + assert list(out[3]) == ["unlabelled", "unlabelled"] + assert list(out[4]) == [0.5, 0.5] + assert "2 of 2 tunable kcats match no evotune source group and get sigma0_log_default (0.5); origins: standard" in caplog.text diff --git a/tests/test_evotune_priors.py b/tests/test_evotune_priors.py index 562cecc..d3a8330 100644 --- a/tests/test_evotune_priors.py +++ b/tests/test_evotune_priors.py @@ -8,6 +8,7 @@ build_sigma0_log, classify_kcat_source, classify_kcat_sources, + warn_unlabelled, ) @@ -15,11 +16,10 @@ def _params() -> EvotuneParams: return EvotuneParams( sigma0_log_default=0.5, source_groups={ - "dlkcat": SourceGroupRule(sources=["dlkcat"]), - "brenda": SourceGroupRule(sources=["brenda"]), - "okp": SourceGroupRule(match_okp=True), + "dlkcat": SourceGroupRule(sources=["dlkcat"], sigma0_log=0.4), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "okp": SourceGroupRule(sources=["okp_*"], sigma0_log=0.3), }, - sigma0_log_source={"dlkcat": 0.4, "brenda": 0.2, "okp": 0.3}, ) @@ -39,23 +39,61 @@ def test_classify_unmatched_source_is_unlabelled(): assert classify_kcat_source("some_other_source", params) == UNLABELLED_GROUP -def test_classify_matches_okp_method(): +def test_classify_matches_prefix_pattern_on_the_origin(): params = _params() - assert classify_kcat_source("CataPro", params, okp_method="CataPro") == "okp" - # Case-insensitive here too. - assert classify_kcat_source("catapro", params, okp_method="CataPro") == "okp" - # No okp_method configured -> falls through to unlabelled. - assert classify_kcat_source("CataPro", params, okp_method=None) == UNLABELLED_GROUP + assert classify_kcat_source("okp_kinform_h", params) == "okp" + assert classify_kcat_source("OKP_EITLEM_Kinetics (similarity=0.9)", params) == "okp" + assert classify_kcat_source("okp", params) == UNLABELLED_GROUP + assert classify_kcat_source("xokp_kinform_h", params) == UNLABELLED_GROUP + + +def test_exact_matches_win_over_patterns_and_longest_prefix_wins(): + params = EvotuneParams(source_groups={ + "wide": SourceGroupRule(sources=["okp_*"], sigma0_log=0.3), + "narrow": SourceGroupRule(sources=["okp_kin*"], sigma0_log=0.2), + "same": SourceGroupRule(sources=["okp_kin*"], sigma0_log=0.25), + "exact": SourceGroupRule(sources=["okp_kinform_h"], sigma0_log=0.1), + }) + assert classify_kcat_source("okp_kinform_h", params) == "exact" # exact over every pattern + assert classify_kcat_source("okp_kinform_l", params) == "narrow" # longest prefix; ties keep group order + assert classify_kcat_source("okp_catapro", params) == "wide" + + +def test_pattern_group_listed_first_does_not_beat_an_exact_match_of_a_later_group(): + params = EvotuneParams(source_groups={ + "all": SourceGroupRule(sources=["*"], sigma0_log=0.5), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + }) + assert classify_kcat_source("brenda", params) == "brenda" + assert classify_kcat_source("anything", params) == "all" def test_classify_kcat_sources_vectorised(): params = _params() - groups = classify_kcat_sources( - ["dlkcat", "brenda", "CataPro", "unknown"], params, okp_method="CataPro", - ) + groups = classify_kcat_sources(["dlkcat", "brenda", "okp_unikp", "unknown"], params) assert list(groups) == ["dlkcat", "brenda", "okp", UNLABELLED_GROUP] +def test_unlabelled_warning_names_count_default_and_common_origins(caplog): + params = _params() + sources = ["brenda"] * 5 + ["standard"] * 3 + ["isozymes"] * 2 + ["a"] + ["b"] + ["c"] + ["d"] + groups = classify_kcat_sources(sources, params) + with caplog.at_level("WARNING"), pytest.warns(UserWarning, match="match no evotune source group"): + warn_unlabelled(groups, sources, params) + assert ( + "9 of 14 tunable kcats match no evotune source group and get sigma0_log_default (0.5); " + "origins: standard, isozymes, a, b, c" + ) in caplog.text + + +def test_unlabelled_warning_only_above_a_fifth(caplog): + params = _params() + sources = ["brenda"] * 4 + ["standard"] # exactly 20 % + with caplog.at_level("WARNING"): + warn_unlabelled(classify_kcat_sources(sources, params), sources, params) + assert "match no evotune source group" not in caplog.text + + # --------------------------------------------------------------------------- # # build_sigma0_log # --------------------------------------------------------------------------- # @@ -71,8 +109,7 @@ def test_build_sigma0_log_uses_group_values_and_default_fallback(): def test_a_tuning_status_does_not_hide_the_source(): params = EvotuneParams( - source_groups={"brenda": SourceGroupRule(sources=["brenda"])}, - sigma0_log_source={"brenda": 0.2}, + source_groups={"brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2)}, sigma0_log_default=0.3, ) for source in ("brenda | tuned=cmaes (prior=6.55)", @@ -85,8 +122,7 @@ def test_a_fuzzy_match_detail_does_not_hide_the_source(): """``ec.source`` carries the wildcard/origin detail; the tier is the source token in front of it.""" params = EvotuneParams( - source_groups={"brenda": SourceGroupRule(sources=["brenda"])}, - sigma0_log_source={"brenda": 0.2}, + source_groups={"brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2)}, sigma0_log_default=0.3, ) assert classify_kcat_source("brenda (wc=0, origin=1)", params) == "brenda" @@ -104,9 +140,8 @@ def test_a_fuzzy_match_detail_does_not_hide_the_source(): def test_a_group_may_still_list_the_whole_source_string(): params = EvotuneParams( source_groups={ - "wildcarded": SourceGroupRule(sources=["brenda (wc=1, origin=3)"]), + "wildcarded": SourceGroupRule(sources=["brenda (wc=1, origin=3)"], sigma0_log=0.5), }, - sigma0_log_source={"wildcarded": 0.5}, ) assert classify_kcat_source( "brenda (wc=1, origin=3)", params, diff --git a/tests/test_fill_kcats.py b/tests/test_fill_kcats.py index c8d7f51..56b8102 100644 --- a/tests/test_fill_kcats.py +++ b/tests/test_fill_kcats.py @@ -272,47 +272,16 @@ def test_filled_kcats_have_isozymes_source(): # --------------------------------------------------------------------------- # -# Adapter-default aggregate (params.kcat_aggregate_isozymes) +# Aggregate choice # --------------------------------------------------------------------------- # -def _adapter_with_isozyme_aggregate( - tmp_path: Path, aggregate: str | None = None, -) -> ModelAdapter: - body = ['conv_gem = "dummy.xml"', 'org_name = "yeast"'] - if aggregate is not None: - body.append(f'kcat_aggregate_isozymes = "{aggregate}"') - (tmp_path / "model_adapter.toml").write_text("\n".join(body) + "\n") - return ModelAdapter.from_folder(tmp_path) - - -def test_aggregate_default_mean_with_adapter_default(tmp_path): - """Adapter defaults to ``kcat_aggregate_isozymes='mean'``; omitting - ``aggregate`` falls through to that default.""" - model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) - model.adapter = _adapter_with_isozyme_aggregate(tmp_path) - fill_kcats_from_isozymes(model, apply=False) - assert model.ec.kcat[3] == pytest.approx((1.0 + 2.0 + 100.0) / 3.0) - - -def test_aggregate_adapter_median_flips_default(tmp_path): - """With the adapter at ``kcat_aggregate_isozymes='median'``, an - omitted ``aggregate`` falls through to median averaging.""" - model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) - model.adapter = _adapter_with_isozyme_aggregate(tmp_path, "median") - fill_kcats_from_isozymes(model, apply=False) - assert model.ec.kcat[3] == pytest.approx(2.0) - - -def test_aggregate_default_mean_when_no_adapter(): - """Without an adapter attached, the mean default is used.""" +def test_aggregate_default_is_the_mean(): model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) fill_kcats_from_isozymes(model, apply=False) assert model.ec.kcat[3] == pytest.approx((1.0 + 2.0 + 100.0) / 3.0) -def test_explicit_aggregate_overrides_adapter(tmp_path): - """Caller-passed ``aggregate=`` wins over the adapter setting.""" +def test_explicit_aggregate_is_used(): model = _isozyme_model([1.0, 2.0, 100.0, 0.0]) - model.adapter = _adapter_with_isozyme_aggregate(tmp_path, "median") fill_kcats_from_isozymes(model, apply=False, aggregate="max") assert model.ec.kcat[3] == pytest.approx(100.0) diff --git a/tests/test_flexibilize_enz_concs.py b/tests/test_flexibilize_enz_concs.py index 0282e3d..7012090 100644 --- a/tests/test_flexibilize_enz_concs.py +++ b/tests/test_flexibilize_enz_concs.py @@ -297,3 +297,12 @@ def test_infeasible_start_returns_empty_result_when_gurobi_presolves_it(tmp_path model.solver.problem.Params.Presolve = 2 model.reactions.get_by_id("R_AB").lower_bound = 50.0 assert flexibilize_enz_concs(model).uniprot_ids == [] + + +def test_a_missing_gr_exp_is_an_error_naming_the_file_key(tmp_path): + (tmp_path / "model_adapter.toml").write_text( + 'conv_gem = "dummy.xml"\norg_name = "test"\nbio_rxn = "biomass"\n' + ) + model = _build_toy(ModelAdapter.from_folder(tmp_path)) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + flexibilize_enz_concs(model) diff --git a/tests/test_fuzzy_kcat_matching.py b/tests/test_fuzzy_kcat_matching.py index c85947c..dab5deb 100644 --- a/tests/test_fuzzy_kcat_matching.py +++ b/tests/test_fuzzy_kcat_matching.py @@ -454,6 +454,7 @@ def test_force_wildcard_level_skips_exact_match(tmp_path): ]), _phyl_dist(["yeast"]), force_wildcard_level=1, + aggregate="max", ) row = df.iloc[0] # With force_wildcard_level=1, we never try the exact 1.1.1.1 token; @@ -673,56 +674,63 @@ def test_aggregate_kcats_max_or_median(): assert _aggregate_kcats(arr, "median") == pytest.approx(2.5) -def _adapter_with_aggregate( - tmp_path: Path, org_name: str, aggregate: str, -) -> ModelAdapter: +def _adapter_with_aggregate(tmp_path: Path, org_name: str, toml_extra: str) -> ModelAdapter: (tmp_path / "model_adapter.toml").write_text( f'conv_gem = "dummy.xml"\n' f'org_name = "{org_name}"\n' - f'kcat_aggregate_brenda = "{aggregate}"\n' + f'{toml_extra}\n' ) return ModelAdapter.from_folder(tmp_path) -def test_aggregate_default_max_when_no_explicit_arg(tmp_path): - """With the adapter at its default ``kcat_aggregate_brenda='max'``, an - omitted ``aggregate`` arg picks the max of the matched BRENDA rows.""" - adapter = _adapter_with_org(tmp_path, "yeast") # adapter default = max - model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) - brenda = _brenda([ +def _three_values(): + return _brenda([ ("1.1.1.1", "alpha", "yeast", 2.0), ("1.1.1.1", "alpha", "yeast", 8.0), ("1.1.1.1", "alpha", "yeast", 100.0), ]) - df = fuzzy_kcat_matching(model, brenda, _phyl_dist(["yeast"])) - assert df["kcat"].iloc[0] == 100.0 -def test_aggregate_adapter_median_changes_default(tmp_path): - """Adapter ``kcat_aggregate_brenda='median'`` flips the default - behaviour to median without the caller passing ``aggregate=``.""" - adapter = _adapter_with_aggregate(tmp_path, "yeast", "median") +def test_aggregate_default_is_the_median_of_the_default_preset(tmp_path): + """Without an explicit ``aggregate``, the default preset brenda_ec_tree + sets ``[kcat_sources.brenda] aggregation`` to median.""" + adapter = _adapter_with_org(tmp_path, "yeast") model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) - brenda = _brenda([ - ("1.1.1.1", "alpha", "yeast", 2.0), - ("1.1.1.1", "alpha", "yeast", 8.0), - ("1.1.1.1", "alpha", "yeast", 100.0), - ]) - df = fuzzy_kcat_matching(model, brenda, _phyl_dist(["yeast"])) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) assert df["kcat"].iloc[0] == 8.0 +def test_aggregate_default_follows_the_preset(tmp_path): + """The gecko_classic preset aggregates BRENDA by the maximum.""" + adapter = _adapter_with_aggregate(tmp_path, "yeast", '[kcat_assignment]\npreset = "gecko_classic"') + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) + assert df["kcat"].iloc[0] == 100.0 + + +def test_aggregate_default_is_max_for_stages_without_a_brenda_table(tmp_path): + adapter = _adapter_with_aggregate( + tmp_path, "yeast", '[[kcat_assignment.stage]]\nsources = ["custom", "database_top"]') + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) + assert df["kcat"].iloc[0] == 100.0 + + +def test_aggregate_default_follows_kcat_sources_over_the_preset(tmp_path): + adapter = _adapter_with_aggregate( + tmp_path, "yeast", '[kcat_sources.brenda]\naggregation = "max"') + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + df = fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"])) + assert df["kcat"].iloc[0] == 100.0 + + def test_explicit_aggregate_overrides_adapter(tmp_path): """Caller-passed ``aggregate=`` wins over the adapter setting.""" - adapter = _adapter_with_aggregate(tmp_path, "yeast", "median") + adapter = _adapter_with_aggregate( + tmp_path, "yeast", '[kcat_sources.brenda]\naggregation = "median"') model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) - brenda = _brenda([ - ("1.1.1.1", "alpha", "yeast", 2.0), - ("1.1.1.1", "alpha", "yeast", 8.0), - ("1.1.1.1", "alpha", "yeast", 100.0), - ]) df = fuzzy_kcat_matching( - model, brenda, _phyl_dist(["yeast"]), aggregate="max", + model, _three_values(), _phyl_dist(["yeast"]), aggregate="max", ) assert df["kcat"].iloc[0] == 100.0 @@ -765,3 +773,10 @@ def test_kcat_is_divided_when_the_matched_substrate_is_taken_more_than_once(tmp_ _phyl_dist(["yeast"]), ) assert df.iloc[0]["kcat"] == pytest.approx(50.0) + + +def test_an_invalid_aggregation_argument_has_the_shared_text(tmp_path): + adapter = _adapter_with_org(tmp_path, "yeast") + model = _ec_model(adapter, [("r1", "1.1.1.1", [("alpha", -1.0)])]) + with pytest.raises(ValueError, match="^aggregation must be max or median$"): + fuzzy_kcat_matching(model, _three_values(), _phyl_dist(["yeast"]), aggregate="mean") diff --git a/tests/test_kcat_assignment.py b/tests/test_kcat_assignment.py index ae80beb..67bb809 100644 --- a/tests/test_kcat_assignment.py +++ b/tests/test_kcat_assignment.py @@ -269,3 +269,45 @@ def test_default_okp_methods_are_kinform_h_and_a_project_list_is_kept(tmp_path): assert okp_methods(kcat_assignment_config(adapter)) == ["KinForm-H"] adapter.params.kcat_sources["okp"] = {"methods": ["CataPro", "UniKP"]} assert okp_methods(kcat_assignment_config(adapter)) == ["CataPro", "UniKP"] + + +@pytest.mark.parametrize("method", ["CatPred", "TurNup"]) +def test_reaction_okp_methods_are_unknown_in_methods(tmp_path, method): + from geckopy.gather_kcats.kcat_assignment import kcat_assignment_config, okp_methods + + _, adapter, _ = _project(tmp_path, '[kcat_assignment]\npreset = "okp"\n') + adapter.params.kcat_sources["okp"] = {"methods": [method]} + from geckopy.gather_kcats.kcat_sources import OKP_METHODS + + with pytest.raises(ValueError) as err: + okp_methods(kcat_assignment_config(adapter)) + assert str(err.value) == ( + f"Unknown OKP methods in [kcat_sources.okp] methods: {method}; methods: {', '.join(OKP_METHODS)}.") + + +def test_okp_source_file_of_no_known_method_fails_naming_the_file(tmp_path): + model, adapter, project = _project( + tmp_path, '[[kcat_assignment.stage]]\nsources = ["okp_nosuchmethod"]\n') + report = update_kcat_sources(model, adapter=adapter).set_index("source") + assert report.loc["okp_nosuchmethod", "status"] == "failed" + from geckopy.gather_kcats.kcat_sources import OKP_METHODS, okp_file + + assert report.loc["okp_nosuchmethod", "note"] == ( + "okp_nosuchmethod is not the file of a known OKP method; the files are " + f"{', '.join(okp_file(m) for m in OKP_METHODS)}.") + + +def test_direct_kcats_from_brenda_follows_preset_or_stages(tmp_path): + """Without an aggregation: the median under a preset, the maximum under stages without a brenda table.""" + from geckopy.gather_kcats import read_kcat_source + + model, adapter, project = _project(tmp_path / "preset", '[kcat_assignment]\npreset = "brenda_ec_tree"\n') + kcats_from_brenda(model, adapter=adapter) + _, meta = read_kcat_source(project / "data" / "kcats" / "brenda.tsv") + assert '"aggregation":"median"' in meta["parameters"] + + model, adapter, project = _project( + tmp_path / "stages", '[[kcat_assignment.stage]]\nsources = ["custom", "database_top"]\n') + kcats_from_brenda(model, adapter=adapter) + _, meta = read_kcat_source(project / "data" / "kcats" / "brenda.tsv") + assert '"aggregation":"max"' in meta["parameters"] diff --git a/tests/test_kcat_sources.py b/tests/test_kcat_sources.py index f7cc2e3..6febac1 100644 --- a/tests/test_kcat_sources.py +++ b/tests/test_kcat_sources.py @@ -466,3 +466,46 @@ def test_kcats_from_dlkcat_run_is_docker(tmp_path, monkeypatch): assert done and ran == ["write", "run"] _, meta = read_kcat_source(tmp_path / "data" / "kcats" / "dlkcat.tsv") assert meta["parameters"] == '{"run":true}' + + +def test_kcats_from_okp_default_method_is_first_of_methods(tmp_path, monkeypatch): + """Without a method, kcats_from_okp uses the first entry of [kcat_sources.okp] methods + (KinForm-H when the project lists none).""" + adapter = _adapter(tmp_path) + full = _model(adapter, light=False) + (tmp_path / "data").mkdir() + submitted = [] + + def fake_submit(model, **kw): + submitted.append(kw["method"]) + return "job" + + monkeypatch.setattr(kcats_from_module, "_submit_open_kinetics_predictor", fake_submit) + monkeypatch.setattr(kcats_from_module, "_fetch_open_kinetics_predictor", lambda model, **kw: (False, None)) + assert kcats_from_okp(full, save=False) == (None, False) + adapter.params.kcat_sources["okp"] = {"methods": ["UniKP", "CataPro"]} + kcats_from_okp(full, save=False, resubmit=True) + assert submitted == ["KinForm-H", "UniKP"] + + +@pytest.mark.parametrize("method", ["CatPred", "TurNup"]) +def test_kcats_from_okp_rejects_reaction_methods(tmp_path, method): + full = _model(_adapter(tmp_path), light=False) + with pytest.raises(ValueError, match="enzyme-substrate pairs only"): + kcats_from_okp(full, method=method) + + +@pytest.mark.parametrize("method, file", [ + ("KinForm-H", "okp_kinform_h"), ("OmniESI-O2DENet", "okp_omniesi_o2denet"), ("UniKP", "okp_unikp"), +]) +def test_okp_file_names(method, file): + from geckopy.gather_kcats.kcat_sources import okp_file, okp_method_of_file + + assert okp_file(method) == file + assert okp_method_of_file(file) == method + + +def test_kcats_from_brenda_rejects_an_invalid_aggregation(tmp_path): + full = _model(_adapter(tmp_path), light=False) + with pytest.raises(ValueError, match="^aggregation must be max or median$"): + kcats_from_brenda(full, aggregation="mean") diff --git a/tests/test_mask_size.py b/tests/test_mask_size.py index 0a591e3..a7f5b8a 100644 --- a/tests/test_mask_size.py +++ b/tests/test_mask_size.py @@ -12,7 +12,7 @@ import cobra from geckopy import EcModel, ModelAdapter -from geckopy.adapter.params import EvotuneParams +from geckopy.adapter.params import EvotuneParams, SourceGroupRule from geckopy.databases.flux_data import FluxData from geckopy.ec_model.ec_data import EcData from geckopy.kcat_tuning.evotune.data import EvotuneData @@ -23,6 +23,12 @@ ) from geckopy.kcat_tuning.evotune.tuning import screen_kcat_leverage +GROUPS = { + "dlkcat": SourceGroupRule(sources=["dlkcat"], sigma0_log=0.4), + "brenda": SourceGroupRule(sources=["brenda"], sigma0_log=0.2), + "custom": SourceGroupRule(sources=["custom"], sigma0_log=0.1), +} + _TRUE_KCAT = 2.0 _START_KCAT = 1.0 _MW = 100.0 @@ -126,7 +132,7 @@ def test_sweep_reports_one_point_per_share_sorted(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) points = sweep_tunable_mask_size( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -145,7 +151,7 @@ def test_sweep_does_not_mutate_input_model(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) before = model.ec.kcat.copy() sweep_tunable_mask_size( @@ -160,7 +166,7 @@ def test_sweep_runs_every_seed_at_every_share(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) points = sweep_tunable_mask_size( model, adapter=adapter, params=params, evotune_data=evotune_data, @@ -180,7 +186,7 @@ def test_sweep_reuses_a_given_screen(tmp_path): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) screen = screen_kcat_leverage( model, adapter=adapter, evotune_data=evotune_data, n_proc=1, ) @@ -299,7 +305,7 @@ def test_sweep_uses_rmse_trace_minimum_not_last_entry(tmp_path, monkeypatch): adapter = _adapter(tmp_path) model = _build_toy(adapter) evotune_data = _evotune_data() - params = EvotuneParams(max_generations=10, rmse_threshold=1e-6) + params = EvotuneParams(source_groups=GROUPS, max_generations=10, rmse_threshold=1e-6) # A later generation can improve the (prior-penalised) objective # while its own plain RMSE is worse than an earlier generation's -- diff --git a/tests/test_params.py b/tests/test_params.py index a7b3e72..2ab664c 100644 --- a/tests/test_params.py +++ b/tests/test_params.py @@ -5,40 +5,32 @@ from geckopy.adapter.params import EvotuneParams, SourceGroupRule -def test_defaults_are_consistent(): - # The shipped defaults must satisfy the group-key validator. - bp = EvotuneParams() - assert set(bp.sigma0_log_source) == set(bp.source_groups) - - -def test_mismatched_source_dict_raises(): - with pytest.raises(ValidationError, match="sigma0_log_source"): - EvotuneParams( - source_groups={ - "dlkcat": SourceGroupRule(sources=["dlkcat"]), - "brenda": SourceGroupRule(sources=["brenda"]), - }, - sigma0_log_source={"dlkcat": 0.4}, # missing "brenda" - ) - - -def test_consistent_custom_groups_ok(): - bp = EvotuneParams( - source_groups={ - "a": SourceGroupRule(sources=["src_a"]), - "b": SourceGroupRule(sources=["src_b"], match_okp=True), - }, - sigma0_log_source={"a": 0.1, "b": 0.2}, - ) - assert set(bp.source_groups) == {"a", "b"} - assert bp.source_groups["b"].match_okp is True +def test_default_groups_are_custom_brenda_ec_tree_combined(): + groups = EvotuneParams().source_groups + assert {k: (v.sources, v.sigma0_log) for k, v in groups.items()} == { + "custom": (["custom"], 0.1), + "brenda": (["brenda"], 0.2), + "ec_tree": (["ec_tree"], 0.3), + "combined": (["combined"], 0.4), + } + assert EvotuneParams().sigma0_log_default == 0.5 -def test_dropped_fields_are_rejected(): - """target_accept and variance_cap_* were confirmed dead/cosmetic - in MATLAB and are not part of the schema; extra="forbid" should - reject them outright rather than silently ignoring them.""" +def test_groups_given_replace_the_defaults(): + bp = EvotuneParams(source_groups={"a": SourceGroupRule(sources=["src_a"], sigma0_log=0.1)}) + assert set(bp.source_groups) == {"a"} + + +@pytest.mark.parametrize("kwargs", [{"sources": ["a"]}, {"sigma0_log": 0.1}]) +def test_a_group_needs_sources_and_sigma0_log(kwargs): with pytest.raises(ValidationError): - EvotuneParams(target_accept=10.0) + SourceGroupRule(**kwargs) + + +def test_dropped_fields_are_rejected(): + """Removed settings are unknown keys, not silently ignored.""" + for kwargs in ({"target_accept": 10.0}, {"variance_cap_default": 10.0}, {"sigma0_log_source": {}}): + with pytest.raises(ValidationError): + EvotuneParams(**kwargs) with pytest.raises(ValidationError): - EvotuneParams(variance_cap_default=10.0) + SourceGroupRule(sources=["a"], sigma0_log=0.1, match_okp=True) diff --git a/tests/test_parity_fixes_g1b.py b/tests/test_parity_fixes_g1b.py index 12d21e5..8fe6cd1 100644 --- a/tests/test_parity_fixes_g1b.py +++ b/tests/test_parity_fixes_g1b.py @@ -236,3 +236,10 @@ def test_gecko_light_model_is_tuned(): assert parsed.origin == (parse_kcat_source(source_before[i]).origin or "unknown") else: assert source == source_before[i] + + +def test_sensitivity_tuning_without_gr_exp_is_an_error(tmp_path): + (tmp_path / "model_adapter.toml").write_text('conv_gem = "dummy.xml"\norg_name = "test"\nbio_rxn = "biomass"\n') + model = _chain(ModelAdapter.from_folder(tmp_path), [1.0, 1.0]) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + sensitivity_tuning(model) diff --git a/tests/test_relax_enz_concs.py b/tests/test_relax_enz_concs.py index 2b09e8f..fe8895e 100644 --- a/tests/test_relax_enz_concs.py +++ b/tests/test_relax_enz_concs.py @@ -107,3 +107,12 @@ def test_an_unknown_weighting_is_rejected(tmp_path): model = _two_routes(_adapter(tmp_path, gr_exp=8.0), {"E1": 2.0, "E2": 2.0}) with pytest.raises(ValueError, match="weight"): relax_enz_concs(model, weight="max") + + +def test_a_missing_gr_exp_is_an_error_naming_the_file_key(tmp_path): + (tmp_path / "model_adapter.toml").write_text('conv_gem = "dummy.xml"\norg_name = "test"\nbio_rxn = "biomass"\n') + model = _two_routes(ModelAdapter.from_folder(tmp_path), {"E1": 2.0, "E2": 2.0}) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + relax_enz_concs(model) + # An explicit growth rate needs no gr_exp. + relax_enz_concs(model, exp_growth=8.0) diff --git a/tests/test_select_kcat_value.py b/tests/test_select_kcat_value.py index 45b8447..a2de349 100644 --- a/tests/test_select_kcat_value.py +++ b/tests/test_select_kcat_value.py @@ -1,12 +1,10 @@ """Tests for apply_kcat_list.""" -from pathlib import Path import numpy as np import pandas as pd import pytest from scipy import sparse -from geckopy import ModelAdapter from geckopy.ec_model import EcModel from geckopy.ec_model.ec_data import EcData from geckopy.gather_kcats import apply_kcat_list @@ -395,56 +393,20 @@ def test_source_bare_lowercase_token_without_metadata(): # --------------------------------------------------------------------------- # -# Adapter-default criteria (params.kcat_aggregate_candidates) +# Criteria choice # --------------------------------------------------------------------------- # -def _adapter_with_candidates( - tmp_path: Path, criteria: str | None = None, -) -> ModelAdapter: - body = ['conv_gem = "dummy.xml"', 'org_name = "yeast"'] - if criteria is not None: - body.append(f'kcat_aggregate_candidates = "{criteria}"') - (tmp_path / "model_adapter.toml").write_text("\n".join(body) + "\n") - return ModelAdapter.from_folder(tmp_path) - - -def _ec_model_with_adapter(adapter: ModelAdapter, rxn_ids: list[str]) -> EcModel: - model = _ec_model(rxn_ids) - model.adapter = adapter - return model - - _THREE_CAND = [("r1", 1.0, "src_low"), ("r1", 4.0, "src_mid"), ("r1", 9.0, "src_high")] -def test_criteria_default_max_with_adapter_default(tmp_path): - """Adapter defaults to ``kcat_aggregate_candidates='max'``; omitting - ``criteria`` picks the largest of the candidate kcats.""" - adapter = _adapter_with_candidates(tmp_path) - model = _ec_model_with_adapter(adapter, ["r1"]) - apply_kcat_list(model, _kcat_list(_THREE_CAND)) - assert model.ec.kcat[0] == 9.0 - - -def test_criteria_adapter_median_flips_default(tmp_path): - """When the adapter sets median, an omitted ``criteria`` follows.""" - adapter = _adapter_with_candidates(tmp_path, "median") - model = _ec_model_with_adapter(adapter, ["r1"]) - apply_kcat_list(model, _kcat_list(_THREE_CAND)) - assert model.ec.kcat[0] == 4.0 - - -def test_criteria_default_max_when_no_adapter(): - """Without an adapter attached, ``criteria`` defaults to ``'max'``.""" - model = _ec_model(["r1"]) # no adapter +def test_criteria_default_is_max(): + model = _ec_model(["r1"]) apply_kcat_list(model, _kcat_list(_THREE_CAND)) assert model.ec.kcat[0] == 9.0 -def test_explicit_criteria_overrides_adapter(tmp_path): - """Caller-passed ``criteria=`` wins over the adapter setting.""" - adapter = _adapter_with_candidates(tmp_path, "median") - model = _ec_model_with_adapter(adapter, ["r1"]) +def test_explicit_criteria_is_used(): + model = _ec_model(["r1"]) apply_kcat_list(model, _kcat_list(_THREE_CAND), criteria="min") assert model.ec.kcat[0] == 1.0 diff --git a/tests/test_sigma_fitter.py b/tests/test_sigma_fitter.py index dde8130..841a04c 100644 --- a/tests/test_sigma_fitter.py +++ b/tests/test_sigma_fitter.py @@ -244,3 +244,11 @@ def test_default_method_is_the_full_grid(tmp_path): result = fit_sigma(model) # default == grid assert result.sigma == pytest.approx(0.01, abs=1e-12) assert len(result.sigma_grid) == 100 + + +def test_fit_sigma_without_gr_exp_is_an_error(tmp_path): + (tmp_path / "model_adapter.toml").write_text('conv_gem = "dummy.xml"\norg_name = "test"\np_tot = 1.0\nf = 1.0\n') + model = _build_pool_only_model(ModelAdapter.from_folder(tmp_path)) + with pytest.raises(ValueError, match="gr_exp is not set in model_adapter.toml"): + fit_sigma(model) + assert fit_sigma(model, growth_rate=10.0).sigma == pytest.approx(0.01, abs=1e-4) diff --git a/tutorials/full_ecModel/model_adapter.toml b/tutorials/full_ecModel/model_adapter.toml index 63bc8e9..973a474 100644 --- a/tutorials/full_ecModel/model_adapter.toml +++ b/tutorials/full_ecModel/model_adapter.toml @@ -27,9 +27,9 @@ reviewed = true [complex] taxonomic_id = 559292 -# Kcat assignment (assign_kcats): the stages of the gecko_median preset, -# with the BRENDA aggregation that kcats_from_brenda takes from -# kcat_aggregate_brenda (the maximum). +# Kcat assignment (assign_kcats): the stages of the gecko_median preset. There +# is no [kcat_sources.brenda] table, so the stages use the BRENDA maximum, not +# the median of the preset. [[kcat_assignment.stage]] sources = ["custom", "database_top", "dlkcat", "database_bottom"] top_origin_limit = 4 @@ -48,21 +48,19 @@ max_generations = 212 # 30 parts glucose. Same field and value as MATLAB's maxGrowthWeight. max_growth_weight = 2.0 -# Trust tiers, least to most trusted. This model labels predicted kcats -# "OpenKineticsPredictor"; match_okp additionally catches models that label -# them with the predictor method name (OkpParams.method) instead. -[evotune.source_groups.okp] -sources = ["OpenKineticsPredictor"] -match_okp = true +# Trust tiers, each with its prior width (the standard deviation of the +# natural log of the kcat; lower is more trusted). The groups match the labels +# of ec.source that the stages above give: dlkcat (DLKcat predictions), brenda +# and custom. Kcats with another label, such as standard, take +# sigma0_log_default. +[evotune.source_groups.dlkcat] +sources = ["dlkcat"] +sigma0_log = 0.3 [evotune.source_groups.brenda] sources = ["brenda"] +sigma0_log = 0.2 [evotune.source_groups.custom] sources = ["custom"] - -[evotune.sigma0_log_source] -okp = 0.25 -brenda = 0.2 -custom = 0.1 - +sigma0_log = 0.1