Add scalar image layer controls to the Rendering tab

docs/loading_data.md

@@ -64,7 +64,7 @@ To layer images:
 
 1. Load the base image.
 1. On the Data tab, click the Add Layer button on the upper layer dataset.
-1. Under the Rendering tab, an opacity slider changes the transparency of the upper layer.
+1. Under the Rendering tab, open Layers to adjust the overlay color map, display range, opacity, and transparent background.
 
 ![Add Layer](./assets/add-layer.jpg)
 

docs/rendering.md

@@ -1,17 +1,36 @@
-# Cinematic Volume Rendering
+# Rendering
 
-VolView reads the DICOM tags of your data to determine appropriate preset parameter values for cinematic volume rendering for your data, but often you will want to tweak those presets to emphasize specific details. Rendering controls apply to the currently selected/active view. We recommend the following sequence of tweaks to improve your visualizations.
+## Image layers in 2D views
+
+Open **Layers** in the Rendering tab to change scalar image color maps,
+display ranges, visibility, and overlay opacity. The base image has its own
+color map and range controls in ordinary 2D and oblique views. Overlay layers
+render in ordinary 2D views.
+
+RGB and RGBA images use their original colors in slice views. Scalar color maps
+and window/level do not recolor them, including when opening saved sessions.
+RGBA overlays multiply their embedded alpha by the layer opacity. Integer color
+channels use their type's zero-based range; floating-point channels use 0 to 1.
+
+The color bar always spans the image's full intensity range, even when the
+selected display range extends beyond it.
 
-1. Cinematic volume rendering involves two "transfer functions": one maps the recorded intensity values (e.g., CT hounsfield units) to opacity and the other maps recorded intensity values to color. Both can be adjusted using the controls at the top of the Rendering tab. The graph at the top shows a histogram of the recorded intensity values in light gray and the current opacity transfer function as an overlaid curve in black. In that graph, beneath the black curve, is a depiction of the color transfer function. ![Cinematic](./assets/16-volview-rendering.jpg)
+**Transparent background** makes overlay values at or below Min transparent,
+then fades linearly to the selected opacity over the first 10% of the display
+range above Min.
 
+## Cinematic rendering controls
+
+VolView reads the DICOM tags of your data to determine appropriate preset parameter values for cinematic volume rendering for your data, but often you will want to tweak those presets to emphasize specific details. Rendering controls apply to the currently selected/active view. We recommend the following sequence of tweaks to improve your visualizations.
+
+1. Cinematic volume rendering involves two "transfer functions": one maps the recorded intensity values (e.g., CT hounsfield units) to opacity and the other maps recorded intensity values to color. Both can be adjusted by expanding **3D Rendering** in the Rendering tab. The graph at the top shows a histogram of the recorded intensity values in light gray and the current opacity transfer function as an overlaid curve in black. In that graph, beneath the black curve, is a depiction of the color transfer function. ![Cinematic](./assets/16-volview-rendering.jpg)
    1. Begin by adjusting the opacity transfer function by a press-and-drag right/left on the top graph. ![Opacity](./assets/17-volview-opacity-notes.jpg)
    2. Then adjust the color transfer function by moving the blue dots on the colorbar beneath the graph. ![Color](./assets/17-volview-colormap-notes.jpg)
 
-2. You may decide that the colormaps and/or transfer function aren't suitable for your data. Click on the "Presets" bar to expand and show the available presets that offer alternative opacity and color transfer functions. ![Presets](./assets/18-volview-presets.jpg)
+2. You may decide that the colormaps and/or transfer function aren't suitable for your data. Expand **Color Presets** within **3D Rendering** to show the available presets that offer alternative opacity and color transfer functions. ![Presets](./assets/18-volview-presets.jpg)
    [**_Watch the video!_**](https://youtu.be/eyrGd-meg6I)
 
 3. Next, the lighting can be controlled.
-
    1. Ambient lighting is the general brightness of the scene.
    2. Diffuse lighting controls how the light reflects off the data. It is influenced by the position of the light relative to the orientation of the data's local surface.
    3. Light-follows-camera can be enabled / disabled to create shadows that highlights details within the data. When enabled, the light will be positioned in-line with the camera. When disabled, the last position of the light stays fixed relative to the data, even if the camera is moved. With light-follows-camera enabled, here is a view when the light is following the camera: ![Default Lighting](./assets/20-volview-lightfollowcamera1.jpg). By moving the camera (and thereby the light) to the front of the data and then disabling light-follows-camera, the view should look light this: ![Light Setup](./assets/20-volview-lightfollowcamera2.jpg). Then moving back to the side view will show shadows that emphasize depth and details: ![Shadows](./assets/20-volview-lightfollowcamera3.jpg).

docs/welcome_screen.md

@@ -8,15 +8,15 @@
 
 Information on the Patient data and non-DICOM data that have been loaded. The currently display image data is highlighted in blue.
 
-**Sample Data** presents a variety of DICOM data that can be used to quickly explore the capabilities of VolView. When you select a sample dataset, that data is downloaded from [http://data.kitware.com/](https://data.kitware.com/#collection/586fef9f8d777f05f44a5c86/folder/634713cf11dab81428208e1e).
+**Sample Data** presents a variety of DICOM data that can be used to quickly explore the capabilities of VolView. When you select a sample dataset, that data is downloaded from [http://data.kitware.com/](https://data.kitware.com/#collection/586fef9f8d777f05f44a5c86/folder/634713cf11dab81428208e1e). The PET/CT Chest sample opens a PET overlay on CT for trying the [layer display controls](./rendering.md).
 
 ### Annotations
 
 Lists the ruler and other measures that have been made on the currently loaded data.
 
 ### Rendering
 
-Controls for the 3D cinematic volume rendering.
+Controls for 3D cinematic rendering and 2D image layers. See [Rendering](./rendering.md) for details.
 
 ## Load / Save State
 

package.json

@@ -54,6 +54,7 @@
     "@types/express": "^5.0.5",
     "@types/file-saver": "^2.0.7",
     "@types/mocha": "^10.0.10",
+    "@types/pngjs": "^6.0.5",
     "@velipso/polybool": "^2.0.11",
     "@vitejs/plugin-vue": "^6.0.2",
     "@vue/test-utils": "^2.4.6",
@@ -89,6 +90,7 @@
     "nanoid": "^5.1.6",
     "patch-package": "^8.0.1",
     "pinia": "^3.0.4",
+    "pngjs": "^7.0.0",
     "prettier": "3.7.2",
     "rollup-plugin-visualizer": "^6.0.5",
     "sass": "^1.94.2",

src/actions/loadUserFiles.ts

@@ -270,6 +270,25 @@ export type DataSourceImporter = (
   sources: DataSource[]
 ) => Promise<ImportDataSourcesResult[]>;
 
+export function showImportedData(results: ImportResult[]) {
+  const loadDataStore = useLoadDataStore();
+  const primaryDataSource = findBaseDataSource(
+    results,
+    loadDataStore.segmentationExtension,
+    loadDataStore.layerExtension
+  );
+  if (!primaryDataSource || !isVolumeResult(primaryDataSource)) return;
+
+  useViewStore().setDataForAllViews(toDataSelection(primaryDataSource));
+  autoLayerDicoms(primaryDataSource, results);
+  autoLayerByName(primaryDataSource, results, loadDataStore.layerExtension);
+  loadSegmentations(
+    primaryDataSource,
+    results,
+    loadDataStore.segmentationExtension
+  );
+}
+
 type LoadDataSourcesOutcome = {
   datasetIds: string[];
   hadErrors: boolean;
@@ -305,27 +324,7 @@ function loadDataSourcesWithOutcome(
       .every((view) => !view.dataID);
 
     if (succeeded.length && shouldShowData) {
-      const primaryDataSource = findBaseDataSource(
-        succeeded,
-        loadDataStore.segmentationExtension,
-        loadDataStore.layerExtension
-      );
-
-      if (isVolumeResult(primaryDataSource)) {
-        const selection = toDataSelection(primaryDataSource);
-        viewStore.setDataForAllViews(selection);
-        autoLayerDicoms(primaryDataSource, succeeded);
-        autoLayerByName(
-          primaryDataSource,
-          succeeded,
-          loadDataStore.layerExtension
-        );
-        loadSegmentations(
-          primaryDataSource,
-          succeeded,
-          loadDataStore.segmentationExtension
-        );
-      } // else must be primaryDataSource.type === 'model', which are not dealt with here yet
+      showImportedData(succeeded);
     }
 
     // Every error importDataSources returns is unreported by contract
@@ -410,7 +409,10 @@ export async function loadUserPromptedFiles() {
   return loadFiles(files);
 }
 
-function urlsToDataSources(urls: string[], names: string[] = []): DataSource[] {
+export function urlsToDataSources(
+  urls: string[],
+  names: string[] = []
+): DataSource[] {
   return urls.map((url, idx) => {
     const defaultName =
       basename(parseUrl(url, window.location.href).pathname) || url;

src/components/AboutBox.vue

@@ -197,6 +197,24 @@
             >tomovision.com</a
           >
         </li>
+        <li>
+          <strong>PET/CT Chest:</strong> Li, P., Wang, S., Li, T., Lu, J.,
+          HuangFu, Y., and Wang, D. (2020). "A Large-Scale CT and PET/CT Dataset
+          for Lung Cancer Diagnosis (Lung-PET-CT-Dx)." The Cancer Imaging
+          Archive.
+          <a
+            rel="noopener noreferrer"
+            target="_blank"
+            href="https://doi.org/10.7937/TCIA.2020.NNC2-0461"
+            >DOI: 10.7937/TCIA.2020.NNC2-0461</a
+          >.
+          <a
+            rel="noopener noreferrer"
+            target="_blank"
+            href="https://creativecommons.org/licenses/by/4.0/"
+            >CC BY 4.0</a
+          >. Ten original DICOM instances selected from each series.
+        </li>
       </ul>
     </v-card-text>
   </v-card>

src/components/BaseImageProperties.vue

@@ -0,0 +1,141 @@
+<script setup lang="ts">
+import { computed } from 'vue';
+import { useImage } from '@/src/composables/useCurrentImage';
+import { isCineImage } from '@/src/core/cine/isCineImage';
+import { NO_NAME } from '@/src/constants';
+import { useImageStatsStore } from '@/src/store/image-stats';
+import { useViewStore } from '@/src/store/views';
+import { useWindowingStore } from '@/src/store/view-configs/windowing';
+import '@/src/vtk/ColorMaps';
+import ImageDisplayControls from './ImageDisplayControls.vue';
+import { resolveBaseColorPreset } from '@/src/vtk/LayerColorPresets';
+import WindowLevelControls from './tools/windowing/WindowLevelControls.vue';
+
+const props = defineProps<{ imageId: string }>();
+const { imageData, metadata } = useImage(computed(() => props.imageId));
+const viewStore = useViewStore();
+const windowingStore = useWindowingStore();
+const statsStore = useImageStatsStore();
+const imageName = computed(() => metadata.value.name ?? NO_NAME);
+const scalars = computed(() => imageData.value?.getPointData().getScalars());
+const matchingViews = computed(() =>
+  viewStore.visibleViews.filter((view) => view.dataID === props.imageId)
+);
+const sliceView = computed(() => {
+  const views = matchingViews.value.filter(
+    ({ type }) =>
+      isCineImage(props.imageId) || type === '2D' || type === 'Oblique'
+  );
+  return views.find(({ id }) => id === viewStore.activeView) ?? views[0];
+});
+const referenceView = computed(
+  () =>
+    sliceView.value ??
+    matchingViews.value.find(({ id }) => id === viewStore.activeView) ??
+    matchingViews.value[0]
+);
+const windowing = computed(() =>
+  referenceView.value
+    ? windowingStore.getConfig(referenceView.value.id, props.imageId)
+    : undefined
+);
+const visibility = computed(() => windowing.value?.visibility ?? true);
+const visibilityUnavailableReason = computed(() => {
+  if (!imageData.value || !scalars.value)
+    return 'The image must finish loading first.';
+  if (!sliceView.value)
+    return 'Open a 2D or oblique view showing this image to change visibility.';
+  return '';
+});
+function toggleVisibility() {
+  if (visibilityUnavailableReason.value || !sliceView.value) return;
+  windowingStore.updateConfig(
+    sliceView.value.id,
+    props.imageId,
+    { visibility: !visibility.value },
+    true
+  );
+}
+const preset = computed(() => resolveBaseColorPreset(windowing.value?.preset));
+function setPreset(value: string) {
+  if (unavailableReason.value || !sliceView.value) return;
+  windowingStore.updateConfig(
+    sliceView.value.id,
+    props.imageId,
+    {
+      preset: resolveBaseColorPreset(value),
+    },
+    true
+  );
+}
+const mappingRange = computed(() => {
+  const width = windowing.value?.width ?? 1;
+  const level = windowing.value?.level ?? 0.5;
+  return [level - width / 2, level + width / 2] as const;
+});
+const imageRange = computed(() => {
+  const stats = statsStore.stats[props.imageId];
+  const range = scalars.value?.getRange() ?? [0, 1];
+  return [stats?.scalarMin ?? range[0], stats?.scalarMax ?? range[1]] as const;
+});
+const unavailableReason = computed(() => {
+  if (!imageData.value || !scalars.value)
+    return 'The image must finish loading first.';
+  if (isCineImage(props.imageId))
+    return 'Cine images already contain display-encoded pixels.';
+  if (scalars.value.getNumberOfComponents() !== 1)
+    return 'Window/level editing requires a single-component image.';
+  if (!sliceView.value)
+    return 'Open a 2D or oblique view showing this image to adjust window/level.';
+  return '';
+});
+function setRange([min, max]: [number, number]) {
+  const view = sliceView.value;
+  const width = max - min;
+  const level = min + width / 2;
+  if (
+    unavailableReason.value ||
+    !view ||
+    ![min, max, width, level].every(Number.isFinite) ||
+    width <= 0 ||
+    ![level - width / 2, level + width / 2].every(Number.isFinite) ||
+    level - width / 2 >= level + width / 2
+  )
+    return;
+  windowingStore.updateConfig(view.id, props.imageId, { width, level }, true);
+}
+</script>
+
+<template>
+  <ImageDisplayControls
+    :key="imageId"
+    :image-name="imageName"
+    :preset="preset"
+    :mapping-range="mappingRange"
+    :image-range="imageRange"
+    :visibility="visibility"
+    :unavailable-reason="unavailableReason"
+    :visibility-unavailable-reason="visibilityUnavailableReason"
+    :close-on-content-click="false"
+    window-level
+    test-id-prefix="base-image"
+    data-testid="base-image-properties"
+    @toggle-visibility="toggleVisibility"
+    @update:preset="setPreset"
+    @update:mapping-range="setRange"
+  >
+    <template #label>
+      <span class="text-caption text-medium-emphasis text-no-wrap">
+        Base image
+      </span>
+    </template>
+    <template #contrast="{ resetRangeDrafts }">
+      <WindowLevelControls
+        v-if="sliceView"
+        :view-id="sliceView.id"
+        :image-id="imageId"
+        @change="resetRangeDrafts"
+      />
+    </template>
+  </ImageDisplayControls>
+</template>