X2HDR: HDR Image Generation in a Perceptually Uniform Space

Ronghuan Wu¹, Wanchao Su², Kede Ma¹, Jing Liao¹, Rafał K. Mantiuk³

^{1   2   3}

Table of Contents

• Setup
• Download Pretrained Models
• HDR Visualization
• Inference: Text-to-HDR Image Generation
• Inference: RAW-to-HDR Image Reconstruction
• Inference: LDR-to-HDR Image Reconstruction
• Training
• Test
• Evaluation (RAW-to-HDR)
• HDR Video Generation

Setup

conda create -n x2hdr python=3.10
conda activate x2hdr

pip3 install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt

Download Pretrained Models

mkdir models

# Download FLUX.1-dev model
hf download --local-dir ./models/Flux black-forest-labs/FLUX.1-dev

# Text-to-HDR, RAW-to-HDR, and LDR-to-HDR LoRA
hf download --local-dir ./models x2hdr/HDR

• Evaluation: The paper reports results for text2hdr_lora, raw2hdr_lora, and ldr2hdr_lora.
• Preview: The preview checkpoints are not comprehensively evaluated.
  • text2hdr_lora_preview is a preview checkpoint trained for 2,000 steps on an internal dataset.
  • raw2hdr_lora_preview is a preview checkpoint obtained by continuing finetuning from raw2hdr_lora on higher-resolution images.

HDR Visualization

To visualize HDR images, you can use tev (downloadable HDR viewer) or OpenHDR (web-based HDR viewer).

Inference: Text-to-HDR Image Generation

The following command will read prompts from test/text2hdr.txt (100 text prompts used for comparison) to generate HDR images:

python infer_text2hdr.py --batch_prompts test/text2hdr.txt

CLIP LoRA weights are disabled by default. Add --load_clip_lora to enable them.

Generation parameters are set in the prompt file (--d overrides --seed):

• --w: width
• --h: height
• --d: seed
• --g: guidance scale
• --s: number of inference steps

If batch_prompts is not specified, you can specify a single prompt to generate:

python infer_text2hdr.py \
    --prompt "masterpiece, 4K, sharp and detailed, high resolution, best quality, A grand, dimly lit hall with a single candle in the foreground" \
    --width 1024 \
    --height 1024 \
    --seed 2026 \
    --guidance_scale 3.5 \
    --num_inference_steps 40

The inference script will output both an HDR image (in EXR format) and an LDR image (in PNG format). The LDR image is provided only for quick visualization, which is helpful when viewing image content on remote servers where direct inspection of EXR files is inconvenient.

Note

• Memory Requirements: The inference script requires approximately 45GB of GPU memory for 1024×1024 images and 33GB of GPU memory for 512×512 images.

• Output Luminance: The generated HDR images are scaled so that their 99.5th percentile luminance equals the target_luminance (default: 16.0).

• Dynamic Range: We observe the dynamic range of the generated HDR images is related to the number of inference steps and different random seeds.

• Banding Effect: We observe banding artifacts in some of the test prompts (i.e., prompt 33), particularly in smooth, dark areas. While switching to float32 can resolve this problem, it increases memory usage and inference time. Therefore, we continue to use bfloat16 by default, as it produces realistic HDR images for most cases while remaining efficient.

Inference: RAW-to-HDR Image Reconstruction

To reconstruct an HDR image from a RAW image, run:

python infer_raw2hdr.py --raw_image test/test_raw.CR2 --width 720 --height 480

Optionally, you can provide a text prompt to guide the reconstruction of over-saturated regions:

python infer_raw2hdr.py --raw_image test/test_raw.CR2 --width 720 --height 480 --prompt "A serene riverside scene with purple sky featuring historic industrial buildings, with their reflections mirrored perfectly on the calm water."

Note

• RAW Image I/O: We use HDRutils for RAW image I/O, which can convert RAW images to .exr format. Therefore, our infer_raw2hdr.py script supports both RAW image files and .exr files as input.

• Downsampling: RAW images typically have high resolution. Since FLUX cannot process such high resolutions, the input image is first downsampled to width × height.

• Text-Guided Hallucination: For RAW-to-HDR without text prompt, the defaults are guidance_scale=3.5 and num_inference_steps=30. When using a text prompt to guide the hallucination, you can increase these values to get results that better match the text prompt. You can also change the default seed=42 to get different results.

Inference: LDR-to-HDR Image Reconstruction

To reconstruct an HDR image from an LDR image without a text prompt, run:

python infer_ldr2hdr.py --ldr_image test/test_ldr.jpg --width 512 --height 512

Training

Please refer to the training README for more details.

Test

Please refer to the test README for more details.

Evaluation (RAW-to-HDR)

Please refer to the evaluation README for more details.

HDR Video Generation

Our training paradigm can be extended to HDR video generation. We finetune Wan2.2-TI2V-5B on an internal HDR video dataset for 15,000 steps using PQ encoding. The finetuned model supports text-to-video generation. We present 8 example HDR videos generated by the model. Due to limited computational resources, we adopt a relatively small T2V model as the baseline, leading to suboptimal video quality.

water.mp4	village.mp4	rocket.mp4	lantern.mp4
fire.mp4	desert.mp4	cup.mp4	candle.mp4

This paradigm can also be extended to HDR video reconstruction.