diff --git a/README.md b/README.md index 4d2182f..2fb7050 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ The wrapper talks to the [`opentelemetry`](https://docs.rs/opentelemetry) API di - **Spans on every operation.** Every `sqlx::Executor` method emits a `SpanKind::Client` span carrying the OTel database-spans recommended set: `db.system.name`, `db.namespace`, `server.address`/`port`, `network.peer.address`/`port`, `network.protocol.name`, `network.transport`, `db.client.connection.pool.name`, `db.query.text`, returned/affected row counts, and SQLSTATE / `error.type` on failure. - **Span/metric attribute parity.** The `db.client.operation.duration` histogram carries the same bounded attribute set as spans – every annotation, every error-path attribute, every connection-level attribute – so dashboards can slice query latency by operation verb, target collection, error class, or pool name. `db.query.text` is the only span attribute deliberately excluded from metrics for cardinality. -- **Caller-supplied annotations.** The library does not parse SQL. Per-query attributes (`db.operation.name`, `db.collection.name`, `db.query.summary`, `db.stored_procedure.name`) come from a small annotation API. +- **Caller-supplied annotations.** Per-query attributes (`db.operation.name`, `db.collection.name`, `db.query.summary`, `db.stored_procedure.name`) come from a small annotation API. Optional automatic query summaries provide a best-effort operation/target label when annotations are absent; SQL extraction is off by default. - **Three backends, one API.** Postgres, SQLite, and MySQL behind feature flags; the wrapper API is identical across all three. - **Drop-in.** `&Pool` implements `sqlx::Executor`, so existing call sites keep working unchanged. @@ -87,9 +87,44 @@ let pool = PoolBuilder::from(raw_pool) `network.protocol.name` defaults to the backend's wire protocol (`"postgresql"` for Postgres, `"mysql"` for MySQL, absent for SQLite); override only when the connection is tunnelled through a different application-layer protocol. `network.transport` is not inferred from the connect string – callers who want this attribute on spans / metrics must declare it explicitly so the value reflects the deployment configuration rather than a guess. +## Automatic query summaries + +Automatic summaries are **off by default**. Enable best-effort `db.query.summary` +generation when a short operation name is useful for grouping: + +```rust +use sqlx_otel::{PoolBuilder, QuerySummaryMode}; + +let pool = PoolBuilder::from(raw_pool) + .with_query_summary_mode(QuerySummaryMode::Auto) + .build(); +``` + +The summary contains only the outer operation and primary target, such as `SELECT users` or +`INSERT orders`. Common table expressions are resolved to their primary physical target +when possible. Recognized literals are scrubbed before extraction; supported forms omit +predicates, selected columns, and bind parameters. The generated summary names the span and +is attached to operation metrics, independently of `QueryTextMode` (including `Off`). It does not infer +`db.operation.name` or `db.collection.name`. + +Any explicit per-query annotations take precedence and disable inference for that query, +even if they do not set a summary. This is a lightweight extractor, not a complete SQL +parser: unsupported forms may yield operation-only or inaccurate summaries, or keep the +generic span name. This is not a general SQL redaction boundary; leave it off for sensitive +SQL. Summaries are capped at 255 bytes, falling back to the operation rather than +truncating an identifier. Table/schema names still contribute to metric cardinality; +use explicit summaries or keep this mode off for dynamically named or sensitive objects. + +The shared scrubber treats backslashes as escapes in single-quoted literals and always +treats `"` as an identifier quote. It does not model PostgreSQL `standard_conforming_strings`, +SQLite literal backslashes, or MySQL `NO_BACKSLASH_ESCAPES`/default double-quoted strings; +in those forms literal text can surface as the summary target. Use explicit annotations +or keep automatic summaries off for queries that depend on those semantics. + ## Per-query annotations -Because the library does not parse SQL, per-query attributes are the caller's responsibility: +Operation, collection, stored-procedure, and custom summary annotations can be supplied by +the caller: ```rust use sqlx_otel::{QueryAnnotateExt, QueryAnnotations}; @@ -133,6 +168,9 @@ See [`QueryAnnotations`](https://docs.rs/sqlx-otel/latest/sqlx_otel/struct.Query | `Obfuscated` | Replace literal values (string, numeric, hex, boolean, dollar-quoted) with `?` in `db.query.text`. | | `Off` | Do not capture `db.query.text`. | +`QueryTextMode::Off` does not disable automatic summaries. Keep `QuerySummaryMode::Off` +as well when table or schema names are sensitive. + `Obfuscated` is useful when SQL is constructed via string interpolation rather than bind parameters – the structure of the query is preserved while sensitive literal values are redacted. Comments, identifiers (quoted or otherwise), operators, and `NULL` are kept verbatim. Both `Full` and `Obfuscated` collapse inter-token whitespace runs to a single space and trim leading/trailing whitespace before emitting `db.query.text`, so multi-line SQL written for source-level readability renders cleanly in OTel exports. Whitespace **inside** string literals, quoted identifiers, dollar-quoted bodies, and comments is preserved verbatim. For example, an UPSERT split across multiple lines for readability: @@ -171,7 +209,7 @@ Set on every `Executor` method (`execute`, `fetch`, `fetch_all`, `fetch_one`, `f | `db.query.text` | The SQL query string with inter-token whitespace collapsed | Unless `QueryTextMode::Off` | | `db.operation.name` | Database operation (e.g. `SELECT`) | When annotated | | `db.collection.name` | Target table or collection | When annotated | -| `db.query.summary` | Low-cardinality query summary | When annotated | +| `db.query.summary` | Generated operation/target or caller-supplied summary | When generated or annotated | | `db.stored_procedure.name` | Stored procedure name | When annotated | | `db.response.returned_rows` | Row count | On `fetch*` methods | | `db.response.affected_rows` | Rows affected (`rows_affected()`) | On `execute` | @@ -190,7 +228,7 @@ On error, the span status is set to `Error` and an `exception` event is added wi | `db.client.response.returned_rows` | Histogram | | Number of rows returned per `fetch*` call | | `db.client.response.affected_rows` | Histogram | | Rows affected per `execute` call (custom) | -These mirror the bounded portion of the span attribute set: connection-level attributes (`db.system.name`, `db.namespace`, `server.address`/`port`, `network.peer.address`/`port`, `network.protocol.name`, `network.transport`, `db.client.connection.pool.name` – wherever set), plus annotation-derived attributes (`db.operation.name`, `db.collection.name`, `db.query.summary`, `db.stored_procedure.name`) when present, plus error-path attributes (`error.type`, plus `db.response.status_code` for `sqlx::Error::Database`) on the error path. `db.query.text` is deliberately excluded for cardinality; `db.query.summary` is caller-controlled and inherits its cardinality cost from the span side. +These mirror the bounded portion of the span attribute set: connection-level attributes (`db.system.name`, `db.namespace`, `server.address`/`port`, `network.peer.address`/`port`, `network.protocol.name`, `network.transport`, `db.client.connection.pool.name` – wherever set), plus annotation-derived attributes (`db.operation.name`, `db.collection.name`, `db.query.summary`, `db.stored_procedure.name`) when present, plus error-path attributes (`error.type`, plus `db.response.status_code` for `sqlx::Error::Database`) on the error path. `db.query.text` is deliberately excluded for cardinality; `db.query.summary` may be generated or caller-supplied, and dynamically named targets or explicit summaries contribute to metric cardinality. `db.client.response.affected_rows` is not part of the OpenTelemetry semantic conventions – we ship it for the same reason as the matching span attribute: backends report a useful database-confirmed count that's worth slicing alongside duration. It is recorded only on `execute()` calls (where `QueryResult::rows_affected()` is meaningful) and is not recorded for `execute_many` (deprecated upstream). diff --git a/src/annotations.rs b/src/annotations.rs index fe1ac78..4f68edc 100644 --- a/src/annotations.rs +++ b/src/annotations.rs @@ -46,7 +46,8 @@ macro_rules! impl_with_annotations_mut { } /// Per-query annotation values that enrich OpenTelemetry spans with semantic-convention -/// attributes the library cannot derive automatically (because it does not parse SQL). +/// attributes supplied by the caller. Explicit annotations suppress automatic query +/// summaries, including when no custom summary is supplied. /// /// Use the builder methods to set whichever attributes apply to a given query, then pass /// the result through one of the equivalent annotation surfaces: @@ -265,7 +266,7 @@ mod tests { fn test_state() -> SharedState { use std::sync::Arc; - use crate::attributes::{ConnectionAttributes, QueryTextMode}; + use crate::attributes::{ConnectionAttributes, QuerySummaryMode, QueryTextMode}; use crate::metrics::Metrics; SharedState { @@ -280,6 +281,7 @@ mod tests { network_transport: None, pool_name: None, query_text_mode: QueryTextMode::Off, + query_summary_mode: QuerySummaryMode::Off, }), metrics: Arc::new(Metrics::new()), } diff --git a/src/attributes.rs b/src/attributes.rs index d80d3d2..1726081 100644 --- a/src/attributes.rs +++ b/src/attributes.rs @@ -27,7 +27,8 @@ use opentelemetry_semantic_conventions::attribute; /// otherwise), operators, and `NULL` are kept verbatim. /// - **[`Off`](Self::Off)** – appropriate when the query text is itself sensitive /// (proprietary schemas, query shapes that reveal business logic) or when query-text -/// cardinality must be eliminated entirely. +/// cardinality must be eliminated entirely. Keep [`QuerySummaryMode::Off`] as well +/// when object names are sensitive: automatic summaries can still emit those names. /// /// `db.query.parameter.` capture is **not supported** – `SQLx`'s `Execute` trait does /// not expose bind values, and reverse-engineering them from the encoded buffer would tie @@ -50,6 +51,46 @@ pub enum QueryTextMode { Off, } +/// Controls whether `sqlx-otel` generates a low-cardinality `db.query.summary`. +/// +/// Automatic summaries contain only the outer SQL operation and its primary target, such +/// as `SELECT users` or `INSERT orders`. For a query whose outer target is a common table +/// expression, the summarizer resolves the CTE to its primary physical target when +/// possible. Recognized literals are scrubbed before extraction; supported forms omit +/// predicates, selected columns, and bind parameters. Generated summaries name the span +/// and appear on operation metrics, independently of [`QueryTextMode`] (including +/// [`QueryTextMode::Off`]). +/// They do not infer `db.operation.name` or `db.collection.name`. +/// +/// This mode is off by default. Any explicit [`QueryAnnotations`](crate::QueryAnnotations) +/// disable automatic summaries for that query, even when they do not supply a summary. +/// This is a best-effort extractor, not a complete SQL parser: unsupported forms may +/// yield an operation-only or inaccurate summary, or keep the generic span name. This is +/// not a general SQL redaction boundary; leave it off for sensitive SQL. Summaries are +/// capped at 255 bytes, falling back to the operation rather than truncating an identifier. +/// Dynamic object names still contribute to metric cardinality. Keep this mode off or +/// supply explicit annotations if object names are sensitive or unbounded. +/// +/// The shared scrubber treats backslashes as escapes in single-quoted literals and always +/// treats `"` as an identifier quote. It does not model `PostgreSQL` +/// `standard_conforming_strings`, `SQLite` literal backslashes, or `MySQL` +/// `NO_BACKSLASH_ESCAPES`/default double-quoted strings; in those forms literal text can +/// surface as the summary target. Use explicit annotations or keep this mode off for +/// queries that depend on those semantics. +/// +/// Configured via +/// [`PoolBuilder::with_query_summary_mode`](crate::PoolBuilder::with_query_summary_mode). +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub enum QuerySummaryMode { + /// Do not generate `db.query.summary`. Explicit [`QueryAnnotations`](crate::QueryAnnotations) + /// are unaffected. + #[default] + Off, + /// Generate a best-effort low-cardinality summary when no explicit query annotations + /// were supplied. + Auto, +} + /// Immutable, connection-level OpenTelemetry attributes shared by every span and metric /// recording from a single pool. /// @@ -84,6 +125,8 @@ pub(crate) struct ConnectionAttributes { pub pool_name: Option, /// Controls `db.query.text` capture. pub query_text_mode: QueryTextMode, + /// Controls automatic `db.query.summary` generation. + pub query_summary_mode: QuerySummaryMode, } impl ConnectionAttributes { @@ -257,6 +300,7 @@ mod tests { network_transport: Some("tcp".into()), pool_name: Some("primary".into()), query_text_mode: QueryTextMode::Full, + query_summary_mode: QuerySummaryMode::Off, }; let kvs = attrs.base_key_values(); assert_eq!(kvs.len(), 9); @@ -284,6 +328,7 @@ mod tests { network_transport: None, pool_name: None, query_text_mode: QueryTextMode::Off, + query_summary_mode: QuerySummaryMode::Off, }; let kvs = attrs.base_key_values(); assert_eq!(kvs.len(), 1); @@ -395,6 +440,7 @@ mod tests { network_transport: network_transport.clone(), pool_name: pool_name.clone(), query_text_mode: QueryTextMode::Off, + query_summary_mode: QuerySummaryMode::Off, }; let kvs = attrs.base_key_values(); let expected = 1 diff --git a/src/executor.rs b/src/executor.rs index 9fbc235..7fecd6a 100644 --- a/src/executor.rs +++ b/src/executor.rs @@ -10,7 +10,7 @@ use opentelemetry::{Context as OtelContext, KeyValue}; use opentelemetry_semantic_conventions::attribute; use crate::annotations::QueryAnnotations; -use crate::attributes::{self, ConnectionAttributes, QueryTextMode}; +use crate::attributes::{self, ConnectionAttributes, QuerySummaryMode, QueryTextMode}; use crate::database::Database; use crate::metrics::Metrics; @@ -97,13 +97,24 @@ fn start_span(name: &str, span_attrs: Vec) -> (OtelContext, Instant) { /// The returned `metric_attrs` mirror the bounded portion of the span attribute set: connection /// attributes plus the four annotation-derived attributes when present, plus error-path attributes /// (`error.type`, `db.response.status_code`) appended later by `record_error`. The unbounded -/// `db.query.text` attribute is deliberately excluded; `db.query.summary` is caller-controlled and -/// can be unbounded – that cardinality cost is inherited from the span side. +/// `db.query.text` attribute is deliberately excluded. Automatically generated +/// `db.query.summary` values contain an operation/target pair; dynamically named targets +/// still contribute to cardinality. Explicit summaries remain caller-controlled. fn begin_query_span( attrs: &ConnectionAttributes, sql: Option<&str>, annotations: Option<&QueryAnnotations>, ) -> (OtelContext, Instant, Vec) { + let generated_annotations; + let annotations = if annotations.is_none() && attrs.query_summary_mode == QuerySummaryMode::Auto + { + generated_annotations = sql + .and_then(|sql| crate::summary::summarize(sql, attrs.system)) + .map(|summary| QueryAnnotations::new().query_summary(summary)); + generated_annotations.as_ref() + } else { + annotations + }; let (op, coll, summary) = annotations.map_or((None, None, None), |a| { ( a.operation.as_deref(), @@ -836,6 +847,7 @@ mod tests { network_transport: None, pool_name: None, query_text_mode: QueryTextMode::Full, + query_summary_mode: QuerySummaryMode::Off, } } @@ -881,6 +893,67 @@ mod tests { ); } + #[test] + fn begin_query_span_generates_low_cardinality_summary_in_auto_mode() { + let mut attrs = test_attrs(); + attrs.query_text_mode = QueryTextMode::Obfuscated; + attrs.query_summary_mode = QuerySummaryMode::Auto; + let (_, _, metric_attrs) = begin_query_span( + &attrs, + Some( + "WITH source_rows AS (SELECT id FROM events WHERE token = 'secret') \ + SELECT COUNT(*) FROM source_rows", + ), + None, + ); + let summary = metric_attrs + .iter() + .find(|kv| kv.key.as_str() == "db.query.summary") + .map(|kv| kv.value.clone()); + assert_eq!( + summary, + Some(opentelemetry::Value::String("SELECT events".into())) + ); + } + + #[test] + fn begin_query_span_does_not_generate_summary_when_disabled() { + let attrs = test_attrs(); + let (_, _, metric_attrs) = begin_query_span(&attrs, Some("SELECT * FROM users"), None); + assert!( + !metric_attrs + .iter() + .any(|kv| kv.key.as_str() == "db.query.summary") + ); + } + + #[test] + fn begin_query_span_prefers_explicit_annotations_over_auto_summary() { + let mut attrs = test_attrs(); + attrs.query_summary_mode = QuerySummaryMode::Auto; + let annotations = QueryAnnotations::new() + .operation("LOOKUP") + .collection("logical_users"); + let (_, _, metric_attrs) = begin_query_span( + &attrs, + Some("SELECT * FROM physical_users"), + Some(&annotations), + ); + assert!( + !metric_attrs + .iter() + .any(|kv| kv.key.as_str() == "db.query.summary") + ); + assert!(metric_attrs.iter().any(|kv| { + kv.key.as_str() == "db.operation.name" + && kv.value == opentelemetry::Value::String("LOOKUP".into()) + })); + assert!(metric_attrs.iter().any(|kv| { + kv.key.as_str() == "db.collection.name" + && kv.value == opentelemetry::Value::String("logical_users".into()) + })); + } + // =========================================================================== // annotations // =========================================================================== @@ -1051,6 +1124,7 @@ mod tests { network_transport: None, pool_name: None, query_text_mode, + query_summary_mode: QuerySummaryMode::Off, } } @@ -1263,6 +1337,18 @@ mod tests { ); } + /// The same supported marked literal shapes must not enter automatically + /// generated summaries for any backend. + #[test] + fn automatic_summary_does_not_expose_marked_literals(s in chain_fragment_marked()) { + let sql = format!("SELECT {s} FROM known_target WHERE token = '{CHAIN_SENTINEL}'"); + for system in ["postgresql", "mysql", "sqlite"] { + let summary = crate::summary::summarize(&sql, system); + prop_assert!(summary.as_ref().is_none_or(|value| !value.contains(CHAIN_SENTINEL)), + "sentinel leaked into summary: input={sql:?} output={summary:?}"); + } + } + /// Trim invariant on the emitted `db.query.text`: for both `Full` and /// `Obfuscated` modes, the captured value never starts or ends with `' '`. /// Trailing `'\n'` is permitted (line-comment terminator); only `' '` is diff --git a/src/lib.rs b/src/lib.rs index d9a0f53..4c7fc94 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -76,7 +76,7 @@ //! | `db.query.text` | The SQL query string with inter-token whitespace collapsed | Unless [`QueryTextMode::Off`] | //! | `db.operation.name` | Database operation (e.g. `SELECT`) | When [annotated] | //! | `db.collection.name` | Target table or collection | When [annotated] | -//! | `db.query.summary` | Low-cardinality query summary | When [annotated] | +//! | `db.query.summary` | Generated or caller-supplied query summary | When generated or [annotated] | //! | `db.stored_procedure.name` | Stored procedure name | When [annotated] | //! | `db.response.returned_rows` | Row count | On `fetch*` methods | //! | `db.response.affected_rows` | Rows affected (`QueryResult::rows_affected()`) | On `execute` ([note](#a-note-on-dbresponseaffected_rows)) | @@ -153,10 +153,19 @@ //! //! # Per-query annotations //! -//! `sqlx-otel` does **not** parse SQL. The four per-query semantic-convention attributes – -//! `db.operation.name`, `db.collection.name`, `db.query.summary`, and -//! `db.stored_procedure.name` – are the caller's responsibility, supplied through the -//! annotation API. There are two equivalent surfaces depending on whether you prefer the +//! Automatic summaries are off by default. Enable [`QuerySummaryMode::Auto`] to generate +//! a best-effort `db.query.summary` containing the operation and primary target (for +//! example `SELECT users`), resolving common table expressions when possible. This works +//! regardless of [`QueryTextMode`], including [`QueryTextMode::Off`], and supplies both the +//! span name and an operation metric attribute. Summaries are capped at 255 bytes; +//! dynamically named objects still contribute to cardinality. This extractor is not a +//! general SQL redaction boundary; unsupported forms may be inaccurate. Leave automatic +//! summaries off for sensitive SQL. The other per-query +//! semantic-convention attributes – `db.operation.name`, +//! `db.collection.name`, and `db.stored_procedure.name` – remain caller-supplied through +//! the annotation API. Any explicit annotations disable automatic summaries for that +//! query, even when they do not supply a summary. +//! There are two equivalent annotation surfaces depending on whether you prefer the //! annotation to live next to the executor or next to the query: //! //! **Executor-side** ([`Pool::with_annotations`], [`PoolConnection::with_annotations`], @@ -263,10 +272,11 @@ mod pool_metrics; mod query_ext; mod rebuilt_query; mod runtime; +mod summary; mod transaction; pub use annotations::{Annotated, AnnotatedMut, QueryAnnotations}; -pub use attributes::QueryTextMode; +pub use attributes::{QuerySummaryMode, QueryTextMode}; pub use connection::PoolConnection; pub use database::Database; pub use pool::{Pool, PoolBuilder}; diff --git a/src/pool.rs b/src/pool.rs index a57d4c3..4eac67a 100644 --- a/src/pool.rs +++ b/src/pool.rs @@ -4,7 +4,7 @@ use std::time::Duration; use opentelemetry_semantic_conventions::metric as semconv_metric; use crate::annotations::{Annotated, QueryAnnotations}; -use crate::attributes::{ConnectionAttributes, QueryTextMode}; +use crate::attributes::{ConnectionAttributes, QuerySummaryMode, QueryTextMode}; use crate::connection::PoolConnection; use crate::database::Database; use crate::metrics::Metrics; @@ -31,13 +31,14 @@ pub(crate) struct SharedState { /// ```no_run /// # #[cfg(feature = "sqlite")] /// # async fn _doc() -> Result<(), sqlx::Error> { -/// use sqlx_otel::{PoolBuilder, QueryTextMode}; +/// use sqlx_otel::{PoolBuilder, QuerySummaryMode, QueryTextMode}; /// use std::time::Duration; /// /// let raw = sqlx::SqlitePool::connect(":memory:").await?; /// let pool = PoolBuilder::from(raw) /// .with_database("my_db") /// .with_query_text_mode(QueryTextMode::Obfuscated) +/// .with_query_summary_mode(QuerySummaryMode::Auto) /// .with_pool_name("my-service-db") /// .with_pool_metrics_interval(Duration::from_secs(5)) /// .build(); @@ -56,6 +57,7 @@ pub struct PoolBuilder { network_protocol_name: Option, network_transport: Option, query_text_mode: QueryTextMode, + query_summary_mode: QuerySummaryMode, pool_name: Option, pool_metrics_interval: Duration, } @@ -78,6 +80,7 @@ impl From> for PoolBuilder { network_protocol_name: DB::DEFAULT_NETWORK_PROTOCOL_NAME.map(String::from), network_transport: None, query_text_mode: QueryTextMode::default(), + query_summary_mode: QuerySummaryMode::default(), pool_name: None, pool_metrics_interval: Duration::from_secs(10), } @@ -149,6 +152,19 @@ impl PoolBuilder { self } + /// Configure automatic low-cardinality `db.query.summary` generation. Defaults to + /// [`QuerySummaryMode::Off`]. + /// + /// [`QuerySummaryMode::Auto`] extracts only the outer SQL operation and primary + /// target, independently of [`QueryTextMode`]. Any explicit per-query + /// [`QueryAnnotations`] disable inference, even without a custom summary. See + /// [`QuerySummaryMode`] for limits and cardinality considerations. + #[must_use] + pub fn with_query_summary_mode(mut self, mode: QuerySummaryMode) -> Self { + self.query_summary_mode = mode; + self + } + /// Set the `db.client.connection.pool.name` attribute and enable the /// `db.client.connection.count` polling task. /// @@ -199,6 +215,7 @@ impl PoolBuilder { network_transport: self.network_transport, pool_name: self.pool_name, query_text_mode: self.query_text_mode, + query_summary_mode: self.query_summary_mode, }); let metrics = Arc::new(Metrics::new()); let meter = opentelemetry::global::meter("sqlx-otel"); diff --git a/src/summary.rs b/src/summary.rs new file mode 100644 index 0000000..36977cd --- /dev/null +++ b/src/summary.rs @@ -0,0 +1,912 @@ +//! Best-effort low-cardinality SQL query summaries. +//! +//! The OpenTelemetry database semantic conventions recommend using +//! `db.query.summary` as the span name when query-level instrumentation can derive one. +//! This module intentionally extracts only the outer operation and its primary target. +//! Recognized literals are scrubbed before extraction; supported forms omit predicates, +//! selected columns, and bind parameters. This is not a complete SQL parser or a general +//! redaction boundary for arbitrary dialect extensions. + +const MAX_SUMMARY_BYTES: usize = 255; +const MAX_CTE_RESOLUTION_DEPTH: usize = 16; + +#[derive(Clone, Debug, PartialEq, Eq)] +enum Token { + Word(String), + QuotedIdentifier(String), + Dot, + LeftParen, + RightParen, + Comma, + Semicolon, +} + +#[derive(Clone, Debug)] +struct Cte { + key: String, + body: Vec, +} + +#[derive(Clone, Debug)] +struct Target { + display: String, + key: Option, +} + +/// Return a short summary such as `SELECT users` or `INSERT orders`. +pub(crate) fn summarize(sql: &str, system: &str) -> Option { + // Reuse QueryTextMode::Obfuscated's literal handling before extraction. Both passes + // are best effort; callers with sensitive SQL should leave automatic summaries off. + let sanitized = crate::obfuscate::obfuscate(sql); + let tokens = tokenize(&sanitized, system == "mysql"); + let (operation, target) = summarize_tokens(&tokens, &[], &mut Vec::new(), 0)?; + let Some(target) = target else { + return Some(operation); + }; + let candidate = format!("{operation} {}", target.display); + if candidate.len() <= MAX_SUMMARY_BYTES { + Some(candidate) + } else { + // Never truncate inside an identifier. Falling back to the operation preserves a + // useful, bounded grouping key. + Some(operation) + } +} + +fn summarize_tokens( + tokens: &[Token], + inherited_cte_scopes: &[&[Cte]], + visited_ctes: &mut Vec, + resolution_depth: usize, +) -> Option<(String, Option)> { + let (local_ctes, statement_start) = parse_ctes(tokens); + let (operation, operation_index) = find_operation(tokens, statement_start)?; + let target = find_target(tokens, operation_index, &operation).map(|target| { + // A write target names the physical object even when a CTE shadows its name. + if !operation.eq_ignore_ascii_case("SELECT") { + return target; + } + resolve_cte_target( + target, + &local_ctes, + inherited_cte_scopes, + visited_ctes, + resolution_depth, + ) + }); + Some((operation, target)) +} + +fn resolve_cte_target( + target: Target, + local_ctes: &[Cte], + inherited_cte_scopes: &[&[Cte]], + visited_ctes: &mut Vec, + resolution_depth: usize, +) -> Target { + if resolution_depth >= MAX_CTE_RESOLUTION_DEPTH { + return target; + } + let Some(ref key) = target.key else { + return target; + }; + if visited_ctes.iter().any(|visited| visited == key) { + return target; + } + let cte = local_ctes.iter().find(|cte| cte.key == *key).or_else(|| { + inherited_cte_scopes + .iter() + .find_map(|scope| scope.iter().find(|cte| cte.key == *key)) + }); + let Some(cte) = cte else { + return target; + }; + + visited_ctes.push(key.clone()); + let mut scopes = Vec::with_capacity(inherited_cte_scopes.len() + 1); + scopes.push(local_ctes); + scopes.extend_from_slice(inherited_cte_scopes); + let resolved = summarize_tokens(&cte.body, &scopes, visited_ctes, resolution_depth + 1) + .and_then(|(operation, nested_target)| { + if operation.eq_ignore_ascii_case("SELECT") { + nested_target + } else { + None + } + }) + .unwrap_or(target); + visited_ctes.pop(); + resolved +} + +fn parse_ctes(tokens: &[Token]) -> (Vec, usize) { + if !token_is_keyword(tokens.first(), "WITH") { + return (Vec::new(), 0); + } + + let mut ctes = Vec::new(); + let mut index = 1; + if token_is_keyword(tokens.get(index), "RECURSIVE") { + index += 1; + } + + loop { + let Some(alias) = tokens.get(index).and_then(identifier_key) else { + return (ctes, index); + }; + index += 1; + + // Optional CTE column list: name(col_a, col_b) AS (...). + if matches!(tokens.get(index), Some(Token::LeftParen)) { + let Some(close) = matching_right_paren(tokens, index) else { + return (ctes, index); + }; + index = close + 1; + } + if !token_is_keyword(tokens.get(index), "AS") { + return (ctes, index); + } + index += 1; + if token_is_keyword(tokens.get(index), "NOT") { + index += 1; + } + if token_is_keyword(tokens.get(index), "MATERIALIZED") { + index += 1; + } + if !matches!(tokens.get(index), Some(Token::LeftParen)) { + return (ctes, index); + } + let Some(close) = matching_right_paren(tokens, index) else { + return (ctes, index); + }; + ctes.push(Cte { + key: alias, + body: tokens[index + 1..close].to_vec(), + }); + index = close + 1; + if matches!(tokens.get(index), Some(Token::Comma)) { + index += 1; + continue; + } + return (ctes, index); + } +} + +fn matching_right_paren(tokens: &[Token], open: usize) -> Option { + let mut depth = 0_u32; + for (index, token) in tokens.iter().enumerate().skip(open) { + match token { + Token::LeftParen => depth = depth.saturating_add(1), + Token::RightParen if depth == 1 => return Some(index), + Token::RightParen => depth = depth.saturating_sub(1), + _ => {} + } + } + None +} + +fn find_operation(tokens: &[Token], start: usize) -> Option<(String, usize)> { + let Token::Word(word) = tokens.get(start)? else { + return None; + }; + is_operation(word).then(|| (word.clone(), start)) +} + +fn find_target(tokens: &[Token], operation_index: usize, operation: &str) -> Option { + if operation.eq_ignore_ascii_case("SELECT") { + let from = find_keyword_at_depth(tokens, operation_index + 1, "FROM")?; + return parse_target(tokens, from + 1); + } + if operation.eq_ignore_ascii_case("INSERT") || operation.eq_ignore_ascii_case("MERGE") { + let into = find_keyword_at_depth(tokens, operation_index + 1, "INTO")?; + return parse_target(tokens, into + 1); + } + if operation.eq_ignore_ascii_case("DELETE") { + let from = find_keyword_at_depth(tokens, operation_index + 1, "FROM")?; + return parse_target(tokens, from + 1); + } + if operation.eq_ignore_ascii_case("UPDATE") + || operation.eq_ignore_ascii_case("COPY") + || operation.eq_ignore_ascii_case("CALL") + || operation.eq_ignore_ascii_case("EXECUTE") + { + return parse_target(tokens, operation_index + 1); + } + if operation.eq_ignore_ascii_case("TRUNCATE") { + let table = find_keyword_at_depth(tokens, operation_index + 1, "TABLE"); + return parse_target(tokens, table.map_or(operation_index + 1, |index| index + 1)); + } + if operation.eq_ignore_ascii_case("CREATE") + || operation.eq_ignore_ascii_case("ALTER") + || operation.eq_ignore_ascii_case("DROP") + { + let object_kind = find_any_keyword_at_depth( + tokens, + operation_index + 1, + &["TABLE", "VIEW", "INDEX", "SCHEMA", "DATABASE"], + )?; + return parse_target(tokens, object_kind + 1); + } + None +} + +fn find_keyword_at_depth(tokens: &[Token], start: usize, keyword: &str) -> Option { + find_any_keyword_at_depth(tokens, start, &[keyword]) +} + +fn find_any_keyword_at_depth(tokens: &[Token], start: usize, keywords: &[&str]) -> Option { + let mut depth = 0_u32; + for (index, token) in tokens.iter().enumerate().skip(start) { + match token { + Token::LeftParen => depth = depth.saturating_add(1), + Token::RightParen => depth = depth.saturating_sub(1), + Token::Semicolon if depth == 0 => return None, + Token::Word(word) + if depth == 0 + && keywords + .iter() + .any(|keyword| word.eq_ignore_ascii_case(keyword)) => + { + return Some(index); + } + _ => {} + } + } + None +} + +fn parse_target(tokens: &[Token], mut index: usize) -> Option { + while let Some(Token::Word(word)) = tokens.get(index) { + if !is_target_modifier(word) { + break; + } + index += 1; + } + + let first = tokens.get(index)?; + let first_display = identifier_display(first)?; + if matches!(first, Token::Word(word) if is_clause_keyword(word)) { + return None; + } + let mut display = first_display.to_owned(); + let mut key = identifier_key(first); + index += 1; + + while matches!(tokens.get(index), Some(Token::Dot)) { + let next = tokens.get(index + 1)?; + let next_display = identifier_display(next)?; + if matches!(next, Token::Word(word) if is_clause_keyword(word)) { + break; + } + display.push('.'); + display.push_str(next_display); + key = None; + index += 2; + } + + Some(Target { display, key }) +} + +fn token_is_keyword(token: Option<&Token>, keyword: &str) -> bool { + matches!(token, Some(Token::Word(word)) if word.eq_ignore_ascii_case(keyword)) +} + +fn identifier_display(token: &Token) -> Option<&str> { + match token { + Token::Word(word) | Token::QuotedIdentifier(word) => Some(word), + _ => None, + } +} + +fn identifier_key(token: &Token) -> Option { + match token { + Token::Word(word) if !is_clause_keyword(word) => Some(word.to_ascii_lowercase()), + Token::QuotedIdentifier(identifier) => Some(identifier.clone()), + _ => None, + } +} + +fn is_operation(word: &str) -> bool { + [ + "SELECT", "INSERT", "UPDATE", "DELETE", "MERGE", "COPY", "CALL", "EXECUTE", "CREATE", + "ALTER", "DROP", "TRUNCATE", "BEGIN", "COMMIT", "ROLLBACK", "EXPLAIN", + ] + .iter() + .any(|operation| word.eq_ignore_ascii_case(operation)) +} + +fn is_target_modifier(word: &str) -> bool { + [ + "ONLY", + "LATERAL", + "IF", + "NOT", + "EXISTS", + "OR", + "REPLACE", + "IGNORE", + "LOW_PRIORITY", + "HIGH_PRIORITY", + "DELAYED", + "TEMP", + "TEMPORARY", + "UNLOGGED", + "CONCURRENTLY", + ] + .iter() + .any(|modifier| word.eq_ignore_ascii_case(modifier)) +} + +fn is_clause_keyword(word: &str) -> bool { + [ + "SELECT", + "FROM", + "WHERE", + "GROUP", + "ORDER", + "HAVING", + "LIMIT", + "OFFSET", + "FETCH", + "JOIN", + "LEFT", + "RIGHT", + "FULL", + "INNER", + "OUTER", + "CROSS", + "ON", + "USING", + "UNION", + "INTERSECT", + "EXCEPT", + "RETURNING", + "VALUES", + "SET", + "AS", + "INTO", + "UPDATE", + "DELETE", + "INSERT", + "MERGE", + ] + .iter() + .any(|keyword| word.eq_ignore_ascii_case(keyword)) +} + +fn tokenize(sql: &str, mysql_comments: bool) -> Vec { + let bytes = sql.as_bytes(); + let mut tokens = Vec::new(); + let mut index = 0; + + while index < bytes.len() { + match bytes[index] { + byte if byte.is_ascii_whitespace() => index += 1, + b'-' if bytes.get(index + 1) == Some(&b'-') => { + index += 2; + while index < bytes.len() && bytes[index] != b'\n' { + index += 1; + } + } + b'#' if mysql_comments => { + // PostgreSQL also permits '#' in operators; this is MySQL-only. + while index < bytes.len() && bytes[index] != b'\n' { + index += 1; + } + } + b'/' if bytes.get(index + 1) == Some(&b'*') => { + index = skip_block_comment(bytes, index); + } + b'\'' => { + // A dialect-specific comment may have desynchronized the shared scrubber. + // Skip any literal that survives rather than treating its contents as SQL. + index = scan_doubled_quote(bytes, index, b'\''); + } + b'"' | b'`' => { + let quote = bytes[index]; + let end = scan_doubled_quote(bytes, index, quote); + tokens.push(Token::QuotedIdentifier(sql[index..end].to_owned())); + index = end; + } + b'[' => { + let end = scan_bracket_identifier(bytes, index); + tokens.push(Token::QuotedIdentifier(sql[index..end].to_owned())); + index = end; + } + b'(' => { + tokens.push(Token::LeftParen); + index += 1; + } + b')' => { + tokens.push(Token::RightParen); + index += 1; + } + b',' => { + tokens.push(Token::Comma); + index += 1; + } + b'.' => { + tokens.push(Token::Dot); + index += 1; + } + b';' => { + tokens.push(Token::Semicolon); + index += 1; + } + byte if is_word_start(byte) => { + let start = index; + index += 1; + while index < bytes.len() && is_word_continue(bytes[index]) { + index += 1; + } + tokens.push(Token::Word(sql[start..index].to_owned())); + } + _ => index += 1, + } + } + + tokens +} + +fn skip_block_comment(bytes: &[u8], mut index: usize) -> usize { + let mut depth = 0_u32; + while index < bytes.len() { + if bytes.get(index..index + 2) == Some(b"/*") { + depth = depth.saturating_add(1); + index += 2; + } else if bytes.get(index..index + 2) == Some(b"*/") { + index += 2; + depth = depth.saturating_sub(1); + if depth == 0 { + break; + } + } else { + index += 1; + } + } + index +} + +fn scan_doubled_quote(bytes: &[u8], start: usize, quote: u8) -> usize { + let mut index = start + 1; + while index < bytes.len() { + if quote == b'\'' && bytes[index] == b'\\' { + index += if index + 1 < bytes.len() { 2 } else { 1 }; + continue; + } + if bytes[index] == quote { + if bytes.get(index + 1) == Some("e) { + index += 2; + continue; + } + return index + 1; + } + index += 1; + } + bytes.len() +} + +fn scan_bracket_identifier(bytes: &[u8], start: usize) -> usize { + let mut index = start + 1; + while index < bytes.len() { + if bytes[index] == b']' { + if bytes.get(index + 1) == Some(&b']') { + index += 2; + continue; + } + return index + 1; + } + index += 1; + } + bytes.len() +} + +fn is_word_start(byte: u8) -> bool { + byte.is_ascii_alphabetic() || byte == b'_' || byte >= 0x80 +} + +fn is_word_continue(byte: u8) -> bool { + is_word_start(byte) || byte.is_ascii_digit() || byte == b'$' +} + +#[cfg(test)] +mod tests { + use proptest::prelude::*; + + fn summarize(sql: &str) -> Option { + super::summarize(sql, "postgresql") + } + + #[test] + fn summarizes_simple_statements() { + let cases = [ + ("SELECT * FROM users WHERE id = $1", "SELECT users"), + ("INSERT INTO orders(id) VALUES ($1)", "INSERT orders"), + ("UPDATE orders SET status = $1", "UPDATE orders"), + ("DELETE FROM orders WHERE id = $1", "DELETE orders"), + ( + "MERGE INTO inventory USING incoming ON true", + "MERGE inventory", + ), + ("CALL refresh_inventory($1)", "CALL refresh_inventory"), + ("TRUNCATE TABLE audit_log", "TRUNCATE audit_log"), + ("CREATE TABLE users(id bigint)", "CREATE users"), + ]; + for (sql, expected) in cases { + assert_eq!(summarize(sql).as_deref(), Some(expected), "sql: {sql}"); + } + } + + #[test] + fn resolves_outer_cte_to_primary_physical_target() { + let sql = r" + WITH source_rows AS ( + SELECT event.id + FROM events event + JOIN leagues league ON league.id = event.league_id + ) + SELECT COUNT(*) FROM source_rows + "; + assert_eq!(summarize(sql).as_deref(), Some("SELECT events")); + } + + #[test] + fn resolves_chained_ctes() { + let sql = r" + WITH baseline AS ( + SELECT id FROM events + ), + source_rows AS ( + SELECT id FROM baseline + ), + evaluated AS ( + SELECT id FROM source_rows + ) + SELECT COUNT(*) FROM evaluated + "; + assert_eq!(summarize(sql).as_deref(), Some("SELECT events")); + } + + #[test] + fn insert_with_ctes_keeps_outer_write_target() { + let sql = r" + WITH baseline AS (SELECT id FROM events) + INSERT INTO event_reports(event_id) + SELECT id FROM baseline + "; + assert_eq!(summarize(sql).as_deref(), Some("INSERT event_reports")); + } + + #[test] + fn write_targets_are_not_resolved_through_cte_aliases() { + for statement in [ + "INSERT INTO reports SELECT * FROM reports", + "UPDATE reports SET id = 1", + "DELETE FROM reports", + ] { + let sql = format!("WITH reports AS (SELECT id FROM source) {statement}"); + let operation = statement.split_whitespace().next().unwrap(); + assert_eq!(summarize(&sql), Some(format!("{operation} reports"))); + } + } + + #[test] + fn unsupported_outer_operations_do_not_expose_nested_operations() { + for sql in [ + "REVOKE SELECT ON accounts FROM app_role", + "GRANT SELECT ON accounts TO app_role", + "SHOW CREATE TABLE accounts", + "REPLACE INTO reports SELECT * FROM source", + "DECLARE cursor_name CURSOR FOR SELECT * FROM accounts", + "PREPARE query_name AS SELECT * FROM accounts", + ] { + assert_eq!(summarize(sql), None, "sql: {sql}"); + } + } + + #[test] + fn read_cte_does_not_resolve_to_a_nested_write_target() { + let sql = "WITH deleted AS (DELETE FROM jobs RETURNING *) SELECT * FROM deleted"; + assert_eq!(summarize(sql).as_deref(), Some("SELECT deleted")); + } + + #[test] + fn mysql_hash_comments_cannot_supply_targets() { + for (sql, expected) in [ + ("SELECT NOW() # FROM tenant_acme", "SELECT"), + ( + "SELECT NOW() # FROM tenant_acme\n FROM users", + "SELECT users", + ), + ( + "# SELECT * FROM tenant_acme\n SELECT * FROM users", + "SELECT users", + ), + ("SELECT * FROM `tenant#users`", "SELECT `tenant#users`"), + ("SELECT '# FROM tenant_acme' FROM users", "SELECT users"), + ] { + assert_eq!(super::summarize(sql, "mysql").as_deref(), Some(expected)); + } + } + + #[test] + fn mysql_hash_comment_quotes_cannot_expose_following_literals() { + for sql in [ + "SELECT id, # 3\" pipe\n 'moved from tenant_acme' AS note FROM jobs", + "SELECT id, # 3` pipe\n 'moved from tenant_acme' AS note FROM jobs", + "SELECT id, # 3\" pipe\n 'it''s moved from tenant_acme' AS note FROM jobs", + r"SELECT id, # 3` pipe + 'it\'s moved from tenant_acme' AS note FROM jobs", + ] { + assert_eq!( + super::summarize(sql, "mysql").as_deref(), + Some("SELECT jobs") + ); + } + } + + #[test] + fn postgres_hash_operator_does_not_hide_the_target() { + assert_eq!( + summarize("SELECT flags # 1 FROM users").as_deref(), + Some("SELECT users") + ); + } + + #[test] + fn backslash_escaped_literals_cannot_supply_targets() { + let sql = r"SELECT 'it\'s' AS a, 'x FROM hidden' AS b FROM users"; + assert_eq!( + super::summarize(sql, "mysql").as_deref(), + Some("SELECT users") + ); + let sql = r"SELECT E'it\'s FROM hidden' FROM users"; + assert_eq!(summarize(sql).as_deref(), Some("SELECT users")); + } + + #[test] + fn preserves_qualified_and_quoted_targets() { + assert_eq!( + summarize(r#"SELECT * FROM "reporting"."daily orders" WHERE id = 42"#).as_deref(), + Some(r#"SELECT "reporting"."daily orders""#) + ); + assert_eq!( + summarize("SELECT * FROM `daily orders`").as_deref(), + Some("SELECT `daily orders`") + ); + } + + #[test] + fn ignores_comments_and_literal_contents() { + let sql = r" + /* SELECT secrets FROM credentials */ + SELECT '-- FROM hidden' FROM public.events + WHERE token = 'sensitive' + "; + assert_eq!(summarize(sql).as_deref(), Some("SELECT public.events")); + } + + #[test] + fn select_without_a_physical_target_uses_operation_only() { + assert_eq!(summarize("SELECT 1").as_deref(), Some("SELECT")); + assert_eq!( + summarize("SELECT * FROM (SELECT * FROM users) nested").as_deref(), + Some("SELECT") + ); + } + + #[test] + fn preserves_operation_casing() { + assert_eq!( + summarize("select * from users").as_deref(), + Some("select users") + ); + } + + #[test] + fn malformed_and_non_query_input_fails_closed() { + assert_eq!(summarize(""), None); + assert_eq!(summarize("this is not sql"), None); + assert_eq!(summarize("WITH broken AS ("), None); + } + + #[test] + fn never_truncates_inside_a_target() { + let target = format!("table_{}", "x".repeat(300)); + let sql = format!("SELECT * FROM {target}"); + assert_eq!(summarize(&sql).as_deref(), Some("SELECT")); + } + + #[test] + fn recursive_cte_does_not_recurse_forever() { + let sql = "WITH RECURSIVE loop AS (SELECT * FROM loop) SELECT * FROM loop"; + assert_eq!(summarize(sql).as_deref(), Some("SELECT loop")); + } + + #[test] + fn cte_resolution_stops_at_the_depth_limit() { + // An acyclic chain distinguishes the depth guard from cycle detection. + let ctes = (0..18) + .map(|index| { + let target = if index == 17 { + "physical_rows".to_owned() + } else { + format!("cte_{}", index + 1) + }; + format!("cte_{index} AS (SELECT * FROM {target})") + }) + .collect::>() + .join(", "); + assert_eq!( + summarize(&format!("WITH {ctes} SELECT * FROM cte_0")).as_deref(), + Some("SELECT cte_16") + ); + assert_eq!( + summarize(&format!("WITH {ctes} SELECT * FROM cte_2")).as_deref(), + Some("SELECT physical_rows") + ); + } + + #[test] + fn cte_columns_and_materialization_preserve_the_source() { + for hint in ["", "MATERIALIZED", "NOT MATERIALIZED"] { + let sql = format!( + "WITH selected(id, label) AS {hint} \ + (SELECT id, COALESCE(label, 'anonymous') FROM users) \ + SELECT * FROM selected" + ); + assert_eq!(summarize(&sql).as_deref(), Some("SELECT users")); + } + } + + #[test] + fn nested_ctes_use_the_nearest_scope() { + let sql = "WITH source AS (SELECT * FROM outer_rows), \ + selected AS (WITH source AS (SELECT * FROM inner_rows) \ + SELECT * FROM source) \ + SELECT * FROM selected"; + assert_eq!(summarize(sql).as_deref(), Some("SELECT inner_rows")); + } + + #[test] + fn incomplete_cte_prefixes_have_no_summary() { + for sql in [ + "WITH", + "WITH , SELECT * FROM users", + "WITH selected(id", + "WITH selected(id)", + "WITH selected AS", + "WITH selected AS MATERIALIZED", + "WITH selected AS (SELECT * FROM users", + ] { + assert_eq!(summarize(sql), None, "sql: {sql}"); + } + } + + #[test] + fn ddl_and_target_modifiers_preserve_object_names() { + for (sql, expected) in [ + ( + "ALTER TABLE ONLY users ADD COLUMN active boolean", + "ALTER users", + ), + ("DROP TABLE IF EXISTS users", "DROP users"), + ( + "CREATE INDEX CONCURRENTLY IF NOT EXISTS users_idx ON users(id)", + "CREATE users_idx", + ), + ("TRUNCATE users", "TRUNCATE users"), + ("SELECT * FROM ONLY users", "SELECT users"), + ("UPDATE ONLY users SET active = true", "UPDATE users"), + ("COPY users TO STDOUT", "COPY users"), + ("EXECUTE refresh_users", "EXECUTE refresh_users"), + ] { + assert_eq!(summarize(sql).as_deref(), Some(expected), "sql: {sql}"); + } + } + + #[test] + fn commands_without_recognized_targets_use_only_the_operation() { + for (sql, expected) in [ + ("CREATE EXTENSION example_extension", "CREATE"), + ("BEGIN", "BEGIN"), + ("COMMIT", "COMMIT"), + ("ROLLBACK", "ROLLBACK"), + ("EXPLAIN SELECT * FROM users", "EXPLAIN"), + ("SELECT * FROM WHERE secret = 'value'", "SELECT"), + ("SELECT * FROM users.", "SELECT"), + ] { + assert_eq!(summarize(sql).as_deref(), Some(expected), "sql: {sql}"); + } + // Malformed qualification keeps the recognized prefix, not the clause/value. + assert_eq!( + summarize("SELECT * FROM users.WHERE secret = 'value'").as_deref(), + Some("SELECT users") + ); + } + + #[test] + fn comments_cannot_supply_a_target_or_a_second_statement() { + for sql in [ + "-- SELECT * FROM secrets\nSELECT * FROM users", + "SELECT -- FROM secrets\n * FROM users", + "SELECT * FROM users -- FROM secrets", + "/* outer /* FROM secrets */ still a comment */ SELECT * FROM users", + ] { + assert_eq!( + summarize(sql).as_deref(), + Some("SELECT users"), + "sql: {sql}" + ); + } + assert_eq!( + summarize("SELECT 1; SELECT * FROM secrets").as_deref(), + Some("SELECT") + ); + } + + #[test] + fn sqlite_bracket_identifiers_keep_delimiters() { + for (sql, expected) in [ + ("SELECT * FROM [daily orders]", "SELECT [daily orders]"), + ( + "SELECT * FROM [reporting].[daily orders]", + "SELECT [reporting].[daily orders]", + ), + ] { + assert_eq!( + super::summarize(sql, "sqlite").as_deref(), + Some(expected), + "sql: {sql}" + ); + } + } + + #[test] + fn bracket_scanner_does_not_end_at_a_doubled_close() { + // Scanner characterization only: SQLite itself ends [...] at the first ]. + assert_eq!(super::scan_bracket_identifier(b"[a]]b] tail", 0), 6); + } + + #[test] + fn summary_limit_counts_utf8_bytes_and_preserves_complete_names() { + for target in ["x".repeat(248), format!("{}xx", "界".repeat(82))] { + let expected = format!("SELECT {target}"); + assert_eq!(expected.len(), 255); + assert_eq!( + summarize(&format!("SELECT * FROM {target}")), + Some(expected) + ); + assert_eq!( + summarize(&format!("SELECT * FROM {target}x")).as_deref(), + Some("SELECT") + ); + } + } + + proptest! { + #[test] + fn generated_quoted_selects_summarize_to_operation_and_target( + target in "[a-z]{1,40}", + value in any::(), + ) { + for (system, quote) in [("postgresql", '"'), ("mysql", '`'), ("sqlite", '"')] { + let sql = format!("SELECT * FROM {quote}{target}{quote} WHERE id = {value}"); + let expected = format!("SELECT {quote}{target}{quote}"); + let summary = super::summarize(&sql, system); + prop_assert_eq!(summary.as_deref(), Some(expected.as_str())); + } + } + + #[test] + fn arbitrary_and_structured_inputs_never_panic_or_exceed_summary_cap(sql in prop_oneof![ + any::(), + "[a-z_][a-z_0-9]{0,300}".prop_map(|target| format!("SELECT * FROM {target}")), + ]) { + for system in ["postgresql", "mysql", "sqlite"] { + let summary = super::summarize(&sql, system); + prop_assert!(summary.as_ref().is_none_or(|value| value.len() <= super::MAX_SUMMARY_BYTES)); + } + } + } +} diff --git a/tests/sqlite.rs b/tests/sqlite.rs index 17fdac6..02aa37b 100644 --- a/tests/sqlite.rs +++ b/tests/sqlite.rs @@ -522,6 +522,110 @@ async fn query_summary_drives_span_name() { test_query_summary_drives_span_name!(test_pool().await, common::SQLITE_DIALECT); } +#[tokio::test] +#[serial] +async fn automatic_summaries_are_opt_in_and_group_without_literal_values() { + use sqlx_otel::{QueryAnnotations, QuerySummaryMode, QueryTextMode}; + + let tel = common::TestTelemetry::install(); + let default_pool = PoolBuilder::from(raw_pool().await) + .with_query_text_mode(QueryTextMode::Off) + .build(); + default_pool + .fetch_all("SELECT name FROM sqlite_master") + .await + .unwrap(); + let spans = tel.spans(); + assert_eq!(spans.len(), 1); + assert_eq!(common::attr(&spans[0], "db.query.summary"), None); + tel.reset(); + + let pool = PoolBuilder::from(raw_pool().await) + .with_query_text_mode(QueryTextMode::Off) + .with_query_summary_mode(QuerySummaryMode::Auto) + .build(); + for sql in [ + "SELECT name FROM sqlite_master WHERE name = 'private_a'", + "SELECT name FROM sqlite_master WHERE name = 'private_b'", + ] { + sqlx::query(sql).fetch_all(&pool).await.unwrap(); + } + let spans = tel.spans(); + assert_eq!(spans.len(), 2); + for span in &spans { + assert_eq!(span.name, "SELECT sqlite_master"); + assert_eq!( + common::attr(span, "db.query.summary"), + Some("SELECT sqlite_master".into()) + ); + assert_eq!(common::attr(span, "db.query.text"), None); + assert_eq!(common::attr(span, "db.operation.name"), None); + assert_eq!(common::attr(span, "db.collection.name"), None); + } + let metrics = tel.metrics(); + let point = common::find_histogram_data_point_with( + &metrics, + "db.client.operation.duration", + &[("db.query.summary", "SELECT sqlite_master")], + ) + .unwrap(); + assert_eq!(point.count(), 2); + tel.reset(); + + sqlx::query("SELECT name FROM sqlite_master") + .with_annotations(QueryAnnotations::new().query_summary("schema lookup")) + .fetch_all(&pool) + .await + .unwrap(); + let spans = tel.spans(); + assert_eq!(spans.len(), 1); + assert_eq!(spans[0].name, "schema lookup"); + assert!( + common::find_histogram_data_point_with( + &tel.metrics(), + "db.client.operation.duration", + &[("db.query.summary", "schema lookup")], + ) + .is_some() + ); +} + +#[tokio::test] +#[serial] +async fn explicit_annotations_without_summary_suppress_auto_inference() { + use sqlx_otel::{QueryAnnotations, QuerySummaryMode, QueryTextMode}; + + let tel = common::TestTelemetry::install(); + let pool = PoolBuilder::from(raw_pool().await) + .with_query_text_mode(QueryTextMode::Off) + .with_query_summary_mode(QuerySummaryMode::Auto) + .build(); + + pool.with_operation("LOOKUP", "logical_users") + .fetch_all("SELECT name FROM sqlite_master") + .await + .unwrap(); + let spans = tel.spans(); + assert_eq!(spans.len(), 1); + assert_eq!(spans[0].name, "LOOKUP logical_users"); + assert_eq!(common::attr(&spans[0], "db.query.summary"), None); + assert_eq!(common::attr(&spans[0], "db.query.text"), None); + tel.reset(); + + sqlx::query("SELECT name FROM sqlite_master") + .with_annotations(QueryAnnotations::new()) + .fetch_all(&pool) + .await + .unwrap(); + let spans = tel.spans(); + assert_eq!(spans.len(), 1); + assert_eq!(spans[0].name, "sqlite"); + assert_eq!(common::attr(&spans[0], "db.query.summary"), None); + assert_eq!(common::attr(&spans[0], "db.operation.name"), None); + assert_eq!(common::attr(&spans[0], "db.collection.name"), None); + assert_eq!(common::attr(&spans[0], "db.query.text"), None); +} + // =========================================================================== // query-side annotations: sqlx::query(...).with_annotations(...).execute(&pool) // ===========================================================================