fix: harden search contracts and semantic resource controls

.env.example

@@ -8,6 +8,8 @@ SearchPostFilterCacheMaxEntries=100
 SearchPostFilterColdSearchConcurrency=10
 SearchPostFilterColdSearchMaxQueueDepth=500
 SearchPostFilterColdSearchMaxWaitMs=10000
+SearchInFlightMaxWaitersPerOperation=500
+SearchPostFilterEvaluationChunkSize=250
 SearchOpenSearchRequestTimeoutMs=20000
 SearchRequestDeadlineMs=30000
 SearchReadinessTimeoutMs=5000

.gitattributes

@@ -0,0 +1,2 @@
+package-lock.json linguist-generated=true
+api-docs/openapi.json linguist-generated=true

README.md

@@ -111,6 +111,18 @@ String inputs are bounded before query construction:
 
 Requests that exceed a limit return HTTP 400 before contacting OpenSearch. These bounds limit input size and query complexity without changing the documented CPE matching semantics.
 
+CVSS vector filters accept full or partial vectors containing at least one unique `KEY:VALUE` metric token. An optional vector header must identify a version supported by the selected parameter. Requested metric tokens can appear in any order, but all tokens must occur in the same stored vector string. CVSS and CWE filters search CNA-provided data only; ADP-provided enrichment is not currently included.
+
+Publication and last-modified date filters support one-sided ranges. When both values are supplied, `pubStartDate` must be less than or equal to `pubEndDate`, and `lastModStartDate` must be less than or equal to `lastModEndDate`.
+
+### Search Ordering and Pagination
+
+`POST /search` uses deterministic ordering so records do not move arbitrarily between offset-based pages. Searches with scored query terms are ordered by relevance descending and then CVE ID ascending. Filter-only searches are ordered by CVE ID ascending.
+
+`pageSize` reports the number of CVE records actually returned in the current response. `page` is the requested one-based page, except a search with no matches reports page `0`. An accepted page beyond the available matches returns an empty `hits` array and page size `0`.
+
+Pagination is limited to complete pages inside the first 10,000 results: `from + resultsPerPage` must not exceed 10,000. `totalPages` reports the smaller of the pages implied by the match count and the complete pages accessible inside that window. For example, a page size of 30 allows at most 333 pages. The match total itself is not reduced by this pagination calculation. Point-in-time search and deep pagination remain deferred.
+
 Use `cveIds` to request up to 100 CVE records by comma-separated CVE ID.
 
 ```bash
@@ -127,11 +139,11 @@ curl -X POST "http://localhost:3000/search" \
   -d '{"noRejected":true}'
 ```
 
-When filtering with `cpeName`, provide a CPE 2.3 name with non-wildcard `part`, `vendor`, `product`, and `version` components. Invalid `cpeName` values return HTTP 404. The API compares the provided CPE Name against CPE Match Criteria, including exact match strings and version range match strings when the indexed CVE data includes range fields. Use `virtualMatchString` when you intentionally need to search for partial CPE Match String values without those required components.
+When filtering with `cpeName`, provide a CPE 2.3 name with part `a`, `h`, or `o` and non-wildcard `vendor`, `product`, and `version` components. Invalid `cpeName` values return HTTP 404. The API compares the provided CPE Name against CPE Match Criteria, including exact match strings and version range match strings when the indexed CVE data includes range fields. Use `virtualMatchString` when you intentionally need to search for partial CPE Match String values without those required components.
 
 ### CPE Name Search
 
-Use `cpeName` when a user has a concrete asset CPE Name and needs CVE records whose CPE Match Criteria apply to that asset version. The `part`, `vendor`, `product`, and `version` components must all be concrete values, not `*`.
+Use `cpeName` when a user has a concrete asset CPE Name and needs CVE records whose CPE Match Criteria apply to that asset version. The part must be `a`, `h`, or `o`; the `vendor`, `product`, and `version` components must all be concrete values, not `*`.
 
 Valid `cpeName` example:
 
@@ -157,7 +169,7 @@ Set `isVulnerable` to `true` with `cpeName` to require the matching CPE Match Cr
 
 ### Virtual Match String Search
 
-Use `virtualMatchString` when a user needs CPE Match Criteria compatibility matching instead of concrete asset matching. String literal components are compared without regard to lexical case. Omitted trailing components and `*` components act as broad matches.
+Use `virtualMatchString` when a user needs CPE Match Criteria compatibility matching instead of concrete asset matching. String literal components are compared without regard to lexical case. When a part is supplied, it must be `a`, `h`, `o`, `*`, or `-`. Omitted trailing components and `*` components act as broad matches.
 
 Partial product example:
 
@@ -247,9 +259,9 @@ Every request receives an `X-Request-ID` response header. A caller-provided `X-R
 
 Access logs omit query strings and request bodies. By default, the API retains 10% of successful non-health-check access logs, suppresses successful `/api/health-check` and `/api/ready-check` access logs, and retains every HTTP 4xx or 5xx access log. Set `RequestLogSuccessSampleRate` from `0` through `1` to change successful-request sampling. Set `RequestLogHealthChecks=true` to retain successful health-check logs.
 
-The API always emits a sanitized JSON `search.performance` event for `/search` queue saturation, queue wait timeout, search timeout, client disconnect, OpenSearch connection errors, and upstream search errors. These events include the request ID, outcome, HTTP status, strategy, cache state, total elapsed time, cancellation reason, and queue state when applicable. Search terms, CPE values, request bodies, credentials, and upstream error details are never included.
+The API always emits a sanitized JSON `search.performance` event for `/search` queue saturation, shared-operation waiter saturation, queue wait timeout, search timeout, client disconnect, OpenSearch connection errors, and upstream search errors. These events include the request ID, outcome, HTTP status, strategy, cache state, total elapsed time, cancellation reason, and bounded queue or waiter state when applicable. Search terms, CPE values, request bodies, credentials, and upstream error details are never included.
 
-Set `SearchLogPerformanceTimings=true` on the API deployment when diagnosing slow `/search` requests. Successful requests then emit the same structured event with additional redacted timing details, including candidate counts, filtered counts, returned count, OpenSearch operation types, maximum observed active OpenSearch requests in the process, OpenSearch elapsed time, OpenSearch `took`, post-filter elapsed time, cold-search queue wait time, page fetch elapsed or wait time, and total handler elapsed time. Leave it disabled during normal operation unless the deployment logging volume is acceptable.
+Set `SearchLogPerformanceTimings=true` on the API deployment when diagnosing slow `/search` requests. Successful requests then emit the same structured event with additional redacted timing details, including candidate counts, filtered counts, returned count, OpenSearch operation types, maximum observed active OpenSearch requests in the process, shared waiter counts, OpenSearch elapsed time, OpenSearch `took`, post-filter elapsed time, cold-search queue wait time, page fetch elapsed or wait time, and total handler elapsed time. Leave it disabled during normal operation unless the deployment logging volume is acceptable.
 
 Each `/search` request owns a lifecycle cancellation signal. When its overall deadline expires, the API aborts the active OpenSearch transport request before returning HTTP 504. When the caller disconnects, the API aborts work that is needed only by that caller and does not attempt to write a response. This prevents timed-out or abandoned standard searches from continuing until the transport timeout.
 
@@ -277,13 +289,17 @@ The cache is intentionally small and short-lived:
 - Cold-search concurrency: cold CPE candidate queries are limited to 10 per process before reaching OpenSearch
 - Cold-search queue depth: up to 500 unique cold searches can wait per process behind the concurrency limit
 - Cold-search queue wait: up to 10 seconds before a waiting search returns HTTP 429
+- Shared operation waiters: up to 500 callers can wait on one candidate or page-fetch operation per process
+- Post-filter evaluation chunk: 250 candidates between cancellation checks and event-loop yields
 - OpenSearch request timeout: 20 seconds for each operation
 - Overall `/search` deadline: 30 seconds
 - Eviction: oldest entries are removed when the cache exceeds its max size
 
-When the cold-search queue is full, new unique CPE post-filter searches fail closed with HTTP 429 and a `SEARCH_QUEUE_FULL` response. Identical requests for a search that is already in flight still wait on the shared in-flight result and do not consume additional queue slots.
+The admission queue applies only to cold CPE candidate searches. Standard `/search` requests and page fetches are not placed in this queue. When the cold-search queue is full, new unique CPE post-filter searches fail closed with HTTP 429 and a `SEARCH_QUEUE_FULL` response. Identical requests for a search that is already in flight still wait on the shared in-flight result and do not consume additional queue slots, up to the per-operation waiter limit. Additional callers fail closed with HTTP 429 and `SEARCH_IN_FLIGHT_WAITER_LIMIT`.
 
-When a cold search waits in the queue for more than 10 seconds, it fails closed with HTTP 429 and a `SEARCH_QUEUE_TIMEOUT` response. OpenSearch operations use one attempt and return a sanitized HTTP 504 `SEARCH_TIMEOUT` response when they exceed 20 seconds. The same response is returned when a `/search` request exceeds the 30-second overall processing deadline, and the active OpenSearch request is aborted.
+When a cold search waits in the queue for more than 10 seconds, it fails closed with HTTP 429 and a `SEARCH_QUEUE_TIMEOUT` response. OpenSearch operations use one attempt and return a sanitized HTTP 504 `SEARCH_TIMEOUT` response when they exceed 20 seconds. Standard searches use the shorter of that timeout and the request's remaining overall deadline. Shared candidate and page-fetch operations retain the independent OpenSearch timeout so one caller's remaining time does not shorten work still needed by other callers. Every caller retains its own 30-second overall deadline.
+
+Semantic filtering runs in bounded chunks and yields to the Node.js event loop between chunks. This lets disconnect and deadline cancellation signals interrupt a large candidate evaluation instead of waiting for the full set to finish synchronously.
 
 Healthy dev timing logs showed the impact for repeated identical `virtualMatchString` range searches:
 
@@ -298,9 +314,11 @@ Healthy dev timing logs showed the impact for repeated identical `virtualMatchSt
 | Avg candidates scanned        |      10000 | cached from prior result |
 | Avg filtered CVEs             |          4 |                        4 |
 
-This helps most when multiple users submit the same expensive CPE search within the TTL. The first request is a cold cache miss; later identical requests on the same running task are warm cache hits. If identical cold requests arrive concurrently before the first one finishes, the later requests wait on the same candidate query instead of starting duplicate 10,000-candidate OpenSearch requests. Disconnecting one waiter does not cancel work still needed by another waiter; the shared operation is aborted when its final waiter disconnects or times out. Each active request still fetches its requested page of full CVE records after the shared candidate result is available.
+This helps most when multiple users submit the same expensive CPE search within the TTL. The first request is a cold cache miss; later equivalent requests on the same running task are warm cache hits. Equivalent requests share keys across JSON property order, CVE ID order and duplicates, CVSS metric token order, and case differences in case-insensitive CPE values. If identical cold requests arrive concurrently before the first one finishes, the later requests wait on the same candidate query instead of starting duplicate 10,000-candidate OpenSearch requests. Disconnecting one waiter does not cancel work still needed by another waiter; the shared operation is aborted when its final waiter disconnects or times out. Each active request still fetches its requested page of full CVE records after the shared candidate result is available.
+
+The cache and cold-search queue are a temporary per-process proof of concept. They do not share state across ECS tasks, so each task warms and admits work independently depending on load balancer routing. Track cache-hit and in-flight-share rates, cold-search queue waits and rejections, cold-versus-warm latency, and OpenSearch time to decide their long-term value. If the benefit is low, remove this layer; if it is material, replace it with an AWS-managed shared cache and admission-control design.
 
-The cache does not share entries across ECS tasks. With multiple tasks, each task warms independently depending on load balancer routing. Steady repeated traffic can warm every active task, but unique one-off searches still pay the cold-cache cost. The request lifecycle and in-flight waiter registry are separate from this result cache, so the cache can later be replaced with Redis or ElastiCache without changing cancellation behavior. The per-process TTL, entry limit, cold-search concurrency, queue depth, queue wait, OpenSearch timeout, and overall request deadline can be tuned with `SearchPostFilterCacheTtlMs`, `SearchPostFilterCacheMaxEntries`, `SearchPostFilterColdSearchConcurrency`, `SearchPostFilterColdSearchMaxQueueDepth`, `SearchPostFilterColdSearchMaxWaitMs`, `SearchOpenSearchRequestTimeoutMs`, and `SearchRequestDeadlineMs`.
+The request lifecycle and in-flight waiter registry are separate from the result cache, so a later shared-cache replacement does not need to change caller cancellation behavior. The per-process TTL, entry limit, cold-search concurrency, queue depth, queue wait, shared waiter limit, evaluation chunk size, OpenSearch timeout, and overall request deadline can be tuned with `SearchPostFilterCacheTtlMs`, `SearchPostFilterCacheMaxEntries`, `SearchPostFilterColdSearchConcurrency`, `SearchPostFilterColdSearchMaxQueueDepth`, `SearchPostFilterColdSearchMaxWaitMs`, `SearchInFlightMaxWaitersPerOperation`, `SearchPostFilterEvaluationChunkSize`, `SearchOpenSearchRequestTimeoutMs`, and `SearchRequestDeadlineMs`.
 
 Docker runtime smoke checks can be run after building the image. The default `Dockerfile` avoids BuildKit `RUN --mount` syntax for AWS builders that do not support it: