Add default-on PIT traversal for semantic searches

[afoote-mitre]

PIT-only review footprint: Compared with the prerequisite af/remove-cve-core branch, 2,605 of 3,094 changed lines (84.2%) are tests, documentation, or configuration; application implementation accounts for 489 changed lines. Counts include additions and deletions, count generated artifacts once, and classify Swagger-only edits in routes/index.js and routes/swagger.js as documentation. The GitHub diff against dev also includes the prerequisite PRs until they are merged.

Summary

Add default-on, internal point-in-time (PIT) traversal for semantic POST /search requests. Instead of stopping after the first 10,000 candidates, the API evaluates successive batches through one snapshot and returns exact totals only after traversal completes.

This applies to cpeName, virtualMatchString, CVSS vector filters, SSVC filters, and positive knownExploited searches with date boundaries. Existing filter matching and scope rules are reused. Direct searches and /webSearch do not use PIT.

Why These Changes Were Needed

Change	Why it was needed	Primary files
Batched semantic traversal using internal search_after	The bounded path can omit qualifying records after candidate 10,000. Scan until a valid empty batch confirms completion, count all matches, and retain only the current batch and requested page IDs. Filling the page does not end the scan because exact totals require evaluating the remaining candidates.	utils/searchPitTraversal.js:44, utils/searchQueryBuilder.js:87, controllers/searchController.js:138
Snapshot-consistent page retrieval	Candidate evaluation and full-record retrieval must see the same record versions. Fetch through the same PIT, restore the selected ID order, and reject missing, duplicate, or unexpected records.	utils/searchPitTraversal.js:19, utils/searchProvider.js:216
Explicit PIT lifecycle using the direct client	Every semantic request creates a server-side resource that must be released after success, failure, or cancellation. Create/search/delete use the shared official OpenSearch client from #163, not private cve-core fields. Cleanup has an independent timeout of at most five seconds; late creation IDs are cleaned up, and cleanup failures produce a fixed warning without replacing the search result.	utils/searchProvider.js:144, utils/searchProvider.js:177, utils/searchProvider.js:245
Complete-response and continuation checks	Partial shards, timeouts, malformed batches, invalid ordering, and expired contexts must not be mistaken for complete results. Validate every batch, including the final empty batch, and require advancing CVE-ID sort values matching source IDs. Errors do not trigger a fresh snapshot or silent bounded fallback.	utils/searchResponseValidator.js:101, utils/searchResponseValidator.js:122, controllers/searchController.js
Remaining-budget deadlines and cooperative evaluation	Multi-batch requests must not receive a fresh overall time budget per operation. Preserve current monotonic elapsed-time and timer-overflow protections, clamp PIT transport timeouts to the remaining budget, and yield between batches and every 100 candidates. Handle cancellation during transport creation without leaking late PIT responses.	utils/searchDeadline.js:36, utils/searchProvider.js:169, utils/searchPitTraversal.js
Default-on configuration with explicit rollback	Enable the approved behavior while retaining an operational escape hatch. Explicit SearchPitEnabled=false restores the bounded path; invalid configured boolean values still disable PIT. Batch size and lifetime use bounded numeric configuration.	utils/featureFlags.js:24, utils/constants.js, config/default.jsonc, config/custom-environment-variables.jsonc, .env.example
Mode-aware warnings and documentation	Distinguish exact PIT totals from bounded totals and explain that complete candidate traversal does not remove the public page window. Broad virtual-match warnings describe processing cost in PIT mode. Postman assertions follow the expected server mode, while current environment-controlled fault-test guards remain intact.	utils/searchResponseBuilder.js:39, routes/index.js, routes/swagger.js, api-docs/openapi.json, README.md:349, test/postman/buildCollection.js, test/postman/CVE-Search-API.postman_collection.json, test/postman/README.md
Regression coverage for both modes and the direct-client integration	Protect candidate continuation, exact totals, snapshot consistency, filter/scope behavior, error mapping, cancellation, cleanup, and rollback after removing the upstream client wrapper.	utils/searchPitTraversal.test.js, utils/searchProvider.test.js, utils/searchResponseValidator.test.js, utils/searchQueryBuilder.test.js, utils/searchDeadline.test.js, utils/searchResponseBuilder.test.js, utils/featureFlags.test.js, controllers/searchController.test.js, routes/search.integration.test.js, config/runtime-config.test.js, api-docs/openapi.test.js

Compatibility and Rollout

• Requested pages must still fit within the first 10,000 matching results. totalResults and totalPages can exceed the accessible page window.
• Each semantic HTTP request opens its own snapshot. Separate page requests can observe intervening index changes.
• Failed traversal returns sanitized errors instead of partial success: timeouts use 504 SEARCH_TIMEOUT, missing/expired contexts use 503 SEARCH_UNAVAILABLE, and malformed/incomplete responses use 502 UPSTREAM_SEARCH_ERROR. Cleanup has its own bounded wait.
• Defaults are SearchPitEnabled=true, SearchPitBatchSize=1000, and SearchPitKeepAliveMs=60000. Existing explicit overrides remain honored. Configuration changes require restart or redeployment.
• Rollback is SearchPitEnabled=false followed by restart or redeployment. This restores the 10,000-candidate cap and warnings; there is no automatic fallback after a PIT error.
• Deployment prerequisites include PIT create/search/delete access, compatible cluster limits, and one searchable document per CVE ID across the resolved indices. Readiness does not establish PIT permissions or global sort uniqueness.
• Complete traversal can increase latency and OpenSearch work for broad searches. Managed rollout and capacity assessment remain separate work; this PR does not merge, deploy, modify mappings, or add application caching, queueing, rate limiting, or aggregate telemetry.