A research workspace for evaluating how much of a full excitation-emission matrix (EEM) is preserved when only a small number of excitation wavelengths (EX) are retained.
A full EEM contains dense fluorescence responses across excitation and emission dimensions. A future compact fluorescence sensor may only be able to measure a small number of excitation wavelengths. This repository studies a staged question:
If only K excitation slices are measured, how well can the full EEM itself be reconstructed, and how well are the inter-sample relationships defined by the full EEM preserved?
The project deliberately separates two endpoints that should not be conflated:
- Full-EEM reconstruction fidelity — preservation of the spectral object itself.
- Inter-sample structural preservation — preservation of sample-to-sample relationships induced by the full EEM.
Neither endpoint is interpreted as "percentage of all EEM information" or as a direct measure of algae-classification information.
The first pilot should keep the existing V5 wavelength-selection framework unchanged and add a transparent reconstruction endpoint.
For a selected set of K excitation wavelengths:
- retain all physically valid emission responses for each selected EX;
- reconstruct missing EX slices along the excitation axis using a fixed, non-learning baseline;
- compare the reconstructed EEM with the measured full EEM on the common valid mask.
Initial baseline:
- piecewise linear interpolation along the EX axis;
- no neural network or learned reconstruction model;
- one identical reconstruction rule for all wavelength-selection strategies;
- no interpolation into physically excluded Rayleigh/Raman mask positions;
- explicit handling of wavelengths outside the selected EX range.
Primary candidate reconstruction metrics:
- relative Frobenius error;
- NRMSE;
- full-vs-reconstructed spectral cosine similarity or spectral angle.
Retain the V5 endpoint:
- L2-normalize full and sparse EEM representations;
- compute cosine distances between samples;
- for each held-out site, compare its full-EEM distance ranking to its sparse-EX distance ranking using Spearman correlation.
This endpoint measures preservation of full-EEM-derived inter-sample relational structure, not full spectral reconstruction.
The most informative outcome may be a difference between the two endpoints. For example, a small K may preserve sample relationships well while still reconstructing the full EEM imperfectly. Such a result would imply that the excitation budget required for downstream representation may be smaller than the budget required for faithful recovery of the original high-resolution EEM.
Conversely, if reconstruction and structural preservation improve together, this would strengthen the case that a compact EX set captures broadly useful spectral variation.
This project does not assume that:
- EEM information is equivalent to algae information;
- a wavelength set that is optimal for EEM reconstruction is optimal for algae classification;
- reconstruction fidelity alone validates a specific hardware design;
- a 4–6 EX budget is a universal optimum;
- simulated finite EX windows validate a real LED spectral power distribution.
Task-specific algae classification or quantification requires separate labeled validation.
docs/scientific_question.md— precise problem definition and scope.docs/methodology_v1.md— proposed dual-endpoint methodology.docs/reconstruction_metrics.md— definitions and interpretation of reconstruction metrics.docs/relation_to_V5.md— how this pilot extends rather than replaces V5.protocols/reconstruction_pilot_protocol.md— executable analysis protocol for the next coding stage.src/README.md— implementation rules before code is added.CHANGELOG.md— methodology history.
The isolated reconstruction pilot has been executed for NAIG and V5-defined Uniform at K = 2, 3, 4, 6, and 8 using 29 verified main scans. A follow-up fixed-configuration dual-endpoint analysis evaluates the pre-defined Uniform-4/6 sets against 500 seeded feasible random combinations per K. Reproducible code is in src/, tables and 300-dpi/vector figures are in outputs/, and the interpretations are documented in reports/reconstruction_pilot_report.md and reports/dual_endpoint_configuration_report.md. The current recommendation retains V5 structural rho as primary, keeps reconstruction supplementary, and carries Uniform-4/6 in parallel for the next discretisation validation; no V5/V6 manuscript text was changed.