Harden Search API execution, telemetry, endpoint controls, and quality gates

[afoote-mitre]

Summary

This PR removes Artillery test ownership from the Search API repository, adds an automated pull-request quality gate, bounds expensive search execution, adds production-oriented request correlation and search telemetry, prepares deprecated /webSearch for retirement, adds configurable fallback rate limiting for /search, and remediates Search API-owned production dependency vulnerabilities.

Changes

Remove Artillery Ownership

• Removed Artillery profiles, configuration, commands, dependencies, and documentation after migration to the centralized performance-testing repository.
• Keeps live performance testing separate from the Search API code-quality workflow.

Add Pull Request Quality Checks

• Added .github/workflows/quality.yml for pull requests.
• Uses Node.js 24.14.1, installs with npm ci, and runs npm run quality.
• Covers ESLint, Prettier, Jest unit and integration tests, and coverage thresholds.
• Uses read-only repository permissions and does not contact live environments.

Bound Search Execution

• Added a 10-second maximum wait for queued cold CPE searches.
• Added a 20-second timeout and zero retries for individual OpenSearch operations.
• Added a 30-second overall /search deadline.
• Returns sanitized 429 SEARCH_QUEUE_TIMEOUT or 504 SEARCH_TIMEOUT responses.
• Preserves immediate queue-full rejection and Retry-After behavior.

Add Operational Telemetry

• Adds validated X-Request-ID correlation to every response, access log, and structured /search event.
• Samples successful access logs at 10% by default, suppresses successful health checks, and retains all 4xx/5xx access logs.
• Always emits sanitized structured events for queue-full, queue-timeout, search-timeout, rate-limit, OpenSearch connection, and upstream failures.
• Distinguishes OpenSearch timeouts from the overall request deadline.
• Includes strategy, cache status, queue state, elapsed time, and correlation ID without logging search terms, CPE values, bodies, credentials, or upstream error details.
• Keeps detailed successful timing logs behind SearchLogPerformanceTimings.

Gate Deprecated /webSearch

• Marks POST /webSearch deprecated and identifies POST /search as its replacement.
• Adds the default-enabled WebSearchEnabled feature flag for controlled consumer migration.
• Returns 410 WEB_SEARCH_DISABLED before validation or OpenSearch work when disabled.
• Fails closed when an explicitly configured feature-flag value is invalid.

Add Fallback /search Rate Limiting

• Adds express-rate-limit as a proven route-limiting implementation.
• Applies limiting only to POST /search; /webSearch, health checks, Swagger, OpenAPI, and static assets are excluded.
• Keeps limiting disabled by default until a per-task production threshold is approved.
• Uses one process-wide in-memory counter and does not trust forwarded client-IP headers.
• Requires a positive SearchRateLimitMaxRequests whenever limiting is enabled and rejects invalid startup configuration.
• Returns 429 SEARCH_RATE_LIMITED with Retry-After, RateLimit, and RateLimit-Policy headers.
• Emits sanitized rate_limited search telemetry.

Remediate Owned Dependency Vulnerabilities

• Updates morgan from 1.10.1 to 1.11.0 to address the log-forging advisory.
• Updates Express's compatible body-parser resolution from 1.20.5 to 1.20.6.
• Updates the brace-expansion override from 5.0.6 to 5.0.8.
• Reduces the production audit from six reported entries to two entries representing the inherited adm-zip vulnerability and its cve-core parent.
• Leaves cve-core unchanged and avoids forcing adm-zip@0.6.0 outside cve-core's declared dependency range.

Documentation and Configuration

• Documents request correlation, access-log sampling, queue/deadline behavior, endpoint retirement, rate limiting, and runtime variables.
• Documents X-Request-ID and 429 response headers in Swagger/OpenAPI.
• Adds focused controller, queue, logging, server, feature-flag, rate-limit, configuration, and OpenAPI coverage.