feat: Valve extension rgb encode input

AGENTS.md

@@ -117,6 +117,17 @@ checks the output against a context pixelforge created itself. It needs no
 ffmpeg. Anything that reaches a core 1.2 or 1.3 entry point panics there, so
 run it after touching submission or barrier code.
 
+The RGB input path (`EncodeConfig::with_rgb_input`) needs
+`VK_VALVE_video_encode_rgb_conversion`, which today only RADV has.
+`tests/rgb_encode.rs` compares it against the colour converter, checks that
+`supports_rgb_encode_input` agrees with encoder creation, and skips what the
+device cannot do, so a green run elsewhere says little about it. Under
+validation it draws two messages that are the layer's, not pixelforge's: the
+layer does not model that extension, so it rejects the RGB profile struct in a
+query pool's profile chain (the spec requires that chain to match the
+session's) and checks the RGB input view against the plain format features
+rather than the video profile's.
+
 Make sure there are no Vulkan validation layer errors during execution. Enable
 them with `PIXELFORGE_VALIDATION=1`; the layer's messages are routed through
 `tracing`, so pair it with `RUST_LOG=warn` (or `debug` for the layer's own

CHANGELOG.md

@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add RGB input converted by the encoder itself through
+  `VK_VALVE_video_encode_rgb_conversion`: `EncodeConfig::with_rgb_input`.
+- Add `VideoContext::supports_rgb_encode_input` and
+  `ColorConverterConfig::rgb_encode_input`, which say when a conversion can be
+  left to the encoder.
+- Add `VideoContext::has_video_encode_rgb_conversion` and
+  `DeviceFeatures::video_encode_rgb_conversion`.
+
 ## [0.10.0] - 19-10-2026
 
 ### Added

README.md

@@ -14,7 +14,8 @@ H.265 and AV1 encode, and H.264 decode.
 - **Asynchronous pipelines**: both directions submit without waiting.
   Encoding hands back an [`EncodeFuture`]; decoding delivers frames through a
   [`DecodeSource`] as the GPU finishes with them.
-- **GPU color conversion**: RGB/BGR → YUV via Vulkan compute shaders (BT.709, BT.2020, sRGB→BT.2020+PQ, scRGB-linear→BT.2020+PQ).
+- **GPU color conversion**: RGB/BGR → YUV via Vulkan compute shaders (BT.709, BT.2020, sRGB→BT.2020+PQ, scRGB-linear→BT.2020+PQ),
+  or in the encoder itself where `VK_VALVE_video_encode_rgb_conversion` is available.
 - **Shared devices**: encode and decode on a Vulkan device your application
   created, from Vulkan 1.1 up, on queues you choose, ordered against your own
   work with timeline semaphores.
@@ -250,6 +251,23 @@ let mut encoder = Encoder::new(context, encode_config)?;
 // converter.convert(input_image, layout, encoder.input_image())?;
 ```
 
+Where the conversion is nothing but the YUV matrix, as for ordinary SDR
+content, some encoders can do it themselves and the shader need not run at
+all. [`ColorConverterConfig::rgb_encode_input`] says whether that works for
+a given conversion and codec on this device. When it does, build the
+encoder with [`EncodeConfig::with_rgb_input`] and hand it the RGB frames.
+
+```rust
+let encode_config = EncodeConfig::h264(1920, 1080)
+    .with_color_description(config.color_description().expect("SDR is encodable"));
+let encoder = match config.rgb_encode_input(&context, Codec::H264) {
+    // The encoder converts: give it the RGB images as they are.
+    Some(format) => Encoder::new(context, encode_config.with_rgb_input(format))?,
+    // The shader converts into the encoder's input image, as above.
+    None => Encoder::new(context, encode_config)?,
+};
+```
+
 ### Encoding on your own device
 
 An application that already renders with Vulkan can give pixelforge its own

src/converter/mod.rs

@@ -8,7 +8,7 @@
 
 mod pipeline;
 
-use crate::encoder::ColorDescription;
+use crate::encoder::{Codec, ColorDescription};
 use crate::error::{PixelForgeError, Result};
 use crate::sync::TimelinePoint;
 use crate::video::TimelineChain;
@@ -325,6 +325,38 @@ impl ColorConverterConfig {
         )
     }
 
+    /// The RGB input to give a `codec` encoder instead of running this
+    /// conversion, when the encoder can do it itself.
+    ///
+    /// The hardware applies only the YUV matrix, so this is `Some` only for a
+    /// conversion that is nothing more: source and target are the same space,
+    /// so no transfer function or gamut change is involved, and the output is
+    /// 4:2:0 at the input's bit depth, NV12 from 8-bit RGB or P010 from
+    /// `ABGR2101010`. The device is then asked, with
+    /// [`VideoContext::supports_rgb_encode_input`], whether it does that
+    /// matrix and range for `codec` and takes this format.
+    ///
+    /// When `Some`, build the encoder with
+    /// [`EncodeConfig::with_rgb_input`](crate::EncodeConfig::with_rgb_input)
+    /// and this format, and with [`Self::color_description`], then hand it the
+    /// RGB images directly and skip the converter.
+    pub fn rgb_encode_input(&self, context: &VideoContext, codec: Codec) -> Option<InputFormat> {
+        if self.source != self.target {
+            return None;
+        }
+        let shape_matches = match self.output_format {
+            OutputFormat::NV12 => matches!(
+                self.input_format,
+                InputFormat::BGRx | InputFormat::BGRA | InputFormat::RGBx | InputFormat::RGBA
+            ),
+            OutputFormat::P010 => self.input_format == InputFormat::ABGR2101010,
+            _ => false,
+        };
+        let desc = self.color_description()?;
+        (shape_matches && context.supports_rgb_encode_input(codec, self.input_format, &desc))
+            .then_some(self.input_format)
+    }
+
     /// Whether the shader can get from this source to this target.
     ///
     /// Anything that would need tone mapping or a forward gamma encode is not

src/encoder/av1/init.rs

@@ -4,6 +4,7 @@ use crate::encoder::codec::{
     CodecEncoder, CommonInitRequest, build_encoder_common, query_video_caps,
 };
 use crate::encoder::profile::av1_profile;
+use crate::encoder::rgb::{RgbConversionCaps, rgb_conversion_profile};
 use crate::encoder::{ColorDescription, EncodeConfig};
 use crate::error::{PixelForgeError, Result};
 use crate::vulkan::VideoContext;
@@ -44,13 +45,17 @@ impl Av1 {
             .tuning_mode(encoder_tuning_mode);
 
         let mut av1_profile_info = vk::VideoEncodeAV1ProfileInfoKHR::default().std_profile(profile);
-        let profile_info = vk::VideoProfileInfoKHR::default()
+        let mut rgb_profile = rgb_conversion_profile();
+        let mut profile_info = vk::VideoProfileInfoKHR::default()
             .video_codec_operation(vk::VideoCodecOperationFlagsKHR::ENCODE_AV1)
             .chroma_subsampling(chroma_subsampling)
             .luma_bit_depth(bit_depth)
             .chroma_bit_depth(bit_depth)
             .push(&mut av1_profile_info)
             .push(&mut video_encode_usage_info);
+        if config.rgb_input.is_some() {
+            profile_info = profile_info.push(&mut rgb_profile);
+        }
 
         let bitstream_buffer_size = MIN_BITSTREAM_BUFFER_SIZE
             .max(config.dimensions.width as usize * config.dimensions.height as usize);
@@ -62,7 +67,12 @@ impl Av1 {
         let mut capabilities = vk::VideoCapabilitiesKHR::default()
             .push(&mut encode_caps)
             .push(&mut av1_caps);
+        let mut rgb_caps = vk::VideoEncodeRgbConversionCapabilitiesVALVE::default();
+        if config.rgb_input.is_some() {
+            capabilities = capabilities.push(&mut rgb_caps);
+        }
         let caps = query_video_caps(&context, &profile_info, &mut capabilities)?;
+        let rgb_caps = config.rgb_input.map(|_| RgbConversionCaps::from(&rgb_caps));
 
         let init = build_encoder_common(&CommonInitRequest {
             context: &context,
@@ -81,6 +91,7 @@ impl Av1 {
             // when the driver does not support separate reference images
             // (required for AMD RADV).
             allow_layered_dpb: true,
+            rgb_caps,
         })?;
         let active_reference_count = init.active_reference_count;
         let common = init.common;

src/encoder/codec.rs

@@ -13,6 +13,7 @@
 //! scaffolding is here.
 
 use ash::vk;
+use ash::vk::TaggedStructure;
 
 use crate::encoder::dpb::MAX_DPB_SLOTS;
 use crate::encoder::gop::{GopFrameType, GopPosition, GopStructure};
@@ -22,6 +23,7 @@ use crate::encoder::resources::{
     create_dpb_images, destroy_encoder_resources, get_video_format, lcm,
     query_supported_video_formats, upload_image_to_input,
 };
+use crate::encoder::rgb::RgbConversionCaps;
 use crate::encoder::{ColorDescription, EncodeConfig, FrameType, RateControlMode};
 use crate::error::{PixelForgeError, Result};
 use crate::sync::TimelinePoint;
@@ -115,6 +117,7 @@ impl EncoderCommon {
             height: self.config.dimensions.height,
             pixel_format: self.config.pixel_format,
             input_image_layout,
+            rgb: self.config.rgb_input.is_some(),
             upload_queue: self.context.transfer_queue(),
             waits: &waits,
         };
@@ -447,6 +450,10 @@ pub(crate) struct CommonInitRequest<'a> {
     /// Whether the codec can use a layered DPB image when the driver lacks
     /// `SEPARATE_REFERENCE_IMAGES` (H.264/H.265 yes; AV1 no).
     pub allow_layered_dpb: bool,
+    /// What the hardware RGB conversion supports for this profile, queried
+    /// alongside the codec capabilities when [`EncodeConfig::rgb_input`] is
+    /// set.
+    pub rgb_caps: Option<RgbConversionCaps>,
 }
 
 /// Result of [`build_encoder_common`]: the assembled common state plus the
@@ -574,6 +581,27 @@ pub(crate) fn build_encoder_common(req: &CommonInitRequest) -> Result<CommonInit
         capabilities.max_coded_extent.height
     );
 
+    // With RGB input the encoder converts, so the session needs its settings
+    // and the input image is RGB; the DPB stays YUV either way.
+    // `Encoder::new` has already checked the configuration itself.
+    let rgb_session = match config.rgb_input {
+        Some(_) => {
+            let caps = req.rgb_caps.ok_or_else(|| {
+                PixelForgeError::NoSuitableDevice(
+                    "RGB input: the driver reported no conversion capabilities".to_string(),
+                )
+            })?;
+            Some(
+                caps.session_info(
+                    &config
+                        .color_description
+                        .unwrap_or_else(ColorDescription::bt709),
+                )?,
+            )
+        }
+        None => None,
+    };
+
     // Pick input (SRC) and reference (DPB) formats.
     let preferred_src_format = get_video_format(config.pixel_format, config.bit_depth);
     let supported_src_formats = query_supported_video_formats(
@@ -596,20 +624,22 @@ pub(crate) fn build_encoder_common(req: &CommonInitRequest) -> Result<CommonInit
             "No supported Vulkan Video DPB formats for this profile".to_string(),
         ));
     }
+    let picture_format = config
+        .rgb_input
+        .map_or(preferred_src_format, |format| format.vk_format());
     if !supported_src_formats
         .iter()
-        .any(|f| f.format == preferred_src_format)
+        .any(|f| f.format == picture_format)
     {
         return Err(PixelForgeError::NoSuitableDevice(format!(
-            "Preferred input format {:?} is not supported for VIDEO_ENCODE_SRC_KHR. Supported: {:?}",
-            preferred_src_format, supported_src_formats
+            "Input format {:?} is not supported for VIDEO_ENCODE_SRC_KHR. Supported: {:?}",
+            picture_format, supported_src_formats
         )));
     }
-    let picture_format = preferred_src_format;
     let reference_picture_format = supported_dpb_formats
         .iter()
         .map(|f| f.format)
-        .find(|f| *f == picture_format)
+        .find(|f| *f == preferred_src_format)
         .unwrap_or(supported_dpb_formats[0].format);
 
     // Negotiate DPB slots and active references.
@@ -640,7 +670,8 @@ pub(crate) fn build_encoder_common(req: &CommonInitRequest) -> Result<CommonInit
 
     // Use the driver-reported std header version for this profile.
     let std_header_version = capabilities.std_header_version;
-    let session_create_info = vk::VideoSessionCreateInfoKHR::default()
+    let mut rgb_session_info = rgb_session.unwrap_or_default();
+    let mut session_create_info = vk::VideoSessionCreateInfoKHR::default()
         .queue_family_index(encode_queue_family)
         .flags(vk::VideoSessionCreateFlagsKHR::empty())
         .video_profile(req.profile_info)
@@ -653,6 +684,9 @@ pub(crate) fn build_encoder_common(req: &CommonInitRequest) -> Result<CommonInit
         .max_dpb_slots(dpb_slot_count as u32)
         .max_active_reference_pictures(max_active_reference_pictures as u32)
         .std_header_version(&std_header_version);
+    if rgb_session.is_some() {
+        session_create_info = session_create_info.push(&mut rgb_session_info);
+    }
 
     let mut session = vk::VideoSessionKHR::null();
     let result = unsafe {
@@ -699,6 +733,7 @@ pub(crate) fn build_encoder_common(req: &CommonInitRequest) -> Result<CommonInit
         aligned_width,
         aligned_height,
         picture_format,
+        rgb_input: config.rgb_input.is_some(),
         pixel_format: config.pixel_format,
         bit_depth: config.bit_depth,
         bitstream_buffer_size: req.bitstream_buffer_size,

src/encoder/h264/init.rs

@@ -6,6 +6,7 @@ use crate::encoder::codec::{
 use crate::encoder::dpb::{DecodedPictureBuffer, DecodedPictureBufferTrait, DpbConfig};
 use crate::encoder::profile::h264_profile_idc;
 use crate::encoder::resources::MIN_BITSTREAM_BUFFER_SIZE;
+use crate::encoder::rgb::{RgbConversionCaps, rgb_conversion_profile};
 use crate::encoder::{ColorDescription, EncodeConfig};
 use crate::error::{PixelForgeError, Result};
 use crate::vulkan::VideoContext;
@@ -49,13 +50,17 @@ impl H264 {
 
         let mut h264_profile_info =
             vk::VideoEncodeH264ProfileInfoKHR::default().std_profile_idc(profile_idc);
-        let profile_info = vk::VideoProfileInfoKHR::default()
+        let mut rgb_profile = rgb_conversion_profile();
+        let mut profile_info = vk::VideoProfileInfoKHR::default()
             .video_codec_operation(vk::VideoCodecOperationFlagsKHR::ENCODE_H264)
             .chroma_subsampling(chroma_subsampling)
             .luma_bit_depth(bit_depth)
             .chroma_bit_depth(bit_depth)
             .push(&mut h264_profile_info)
             .push(&mut video_encode_usage_info);
+        if config.rgb_input.is_some() {
+            profile_info = profile_info.push(&mut rgb_profile);
+        }
 
         // The driver's preferred entropy mode is H.264-specific (some require
         // CAVLC for High 4:4:4 Predictive).
@@ -68,7 +73,12 @@ impl H264 {
         let mut capabilities = vk::VideoCapabilitiesKHR::default()
             .push(&mut encode_caps)
             .push(&mut h264_caps);
+        let mut rgb_caps = vk::VideoEncodeRgbConversionCapabilitiesVALVE::default();
+        if config.rgb_input.is_some() {
+            capabilities = capabilities.push(&mut rgb_caps);
+        }
         let caps = query_video_caps(&context, &profile_info, &mut capabilities)?;
+        let rgb_caps = config.rgb_input.map(|_| RgbConversionCaps::from(&rgb_caps));
 
         // Scale the bitstream buffer by resolution so a single frame can't overflow (matches AV1).
         let bitstream_buffer_size = MIN_BITSTREAM_BUFFER_SIZE
@@ -83,6 +93,7 @@ impl H264 {
             max_active_refs_cap: 32,
             bitstream_buffer_size,
             allow_layered_dpb: true,
+            rgb_caps,
         })?;
         let active_reference_count = init.active_reference_count;
         let common = init.common;

src/encoder/h265/init.rs

@@ -6,6 +6,7 @@ use crate::encoder::codec::{
 use crate::encoder::dpb::{DecodedPictureBuffer, DecodedPictureBufferTrait, DpbConfig};
 use crate::encoder::profile::h265_profile_idc;
 use crate::encoder::resources::MIN_BITSTREAM_BUFFER_SIZE;
+use crate::encoder::rgb::{RgbConversionCaps, rgb_conversion_profile};
 use crate::encoder::{ColorDescription, EncodeConfig};
 use crate::error::{PixelForgeError, Result};
 use crate::vulkan::VideoContext;
@@ -53,13 +54,17 @@ impl H265 {
 
         let mut h265_profile_info =
             vk::VideoEncodeH265ProfileInfoKHR::default().std_profile_idc(profile_idc);
-        let profile_info = vk::VideoProfileInfoKHR::default()
+        let mut rgb_profile = rgb_conversion_profile();
+        let mut profile_info = vk::VideoProfileInfoKHR::default()
             .video_codec_operation(vk::VideoCodecOperationFlagsKHR::ENCODE_H265)
             .chroma_subsampling(chroma_subsampling)
             .luma_bit_depth(bit_depth)
             .chroma_bit_depth(bit_depth)
             .push(&mut h265_profile_info)
             .push(&mut video_encode_usage_info);
+        if config.rgb_input.is_some() {
+            profile_info = profile_info.push(&mut rgb_profile);
+        }
 
         // Query device capabilities with the H.265-specific capability struct
         // chained in (required by the driver).
@@ -68,7 +73,12 @@ impl H265 {
         let mut capabilities = vk::VideoCapabilitiesKHR::default()
             .push(&mut encode_caps)
             .push(&mut h265_caps);
+        let mut rgb_caps = vk::VideoEncodeRgbConversionCapabilitiesVALVE::default();
+        if config.rgb_input.is_some() {
+            capabilities = capabilities.push(&mut rgb_caps);
+        }
         let caps = query_video_caps(&context, &profile_info, &mut capabilities)?;
+        let rgb_caps = config.rgb_input.map(|_| RgbConversionCaps::from(&rgb_caps));
 
         // Scale the bitstream buffer by resolution so a single frame can't overflow (matches AV1).
         let bitstream_buffer_size = MIN_BITSTREAM_BUFFER_SIZE
@@ -83,6 +93,7 @@ impl H265 {
             max_active_refs_cap: 15,
             bitstream_buffer_size,
             allow_layered_dpb: true,
+            rgb_caps,
         })?;
         let active_reference_count = init.active_reference_count;
         let common = init.common;