Skip to content

[SVLS-10109] Prototype: cache feature flag config in the Extension for the real SDK - #1527

Draft
leoromanovsky wants to merge 1 commit into
DataDog:mainfrom
leoromanovsky:leo.romanovsky/ffe-lambda-config-cache-alt
Draft

leoromanovsky wants to merge 1 commit into
DataDog:mainfrom
leoromanovsky:leo.romanovsky/ffe-lambda-config-cache-alt

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Nov 17, 2026 •

Copy link
Copy Markdown

Jira: exploratory follow-up to SVLS-10109. Second, independent prototype; #1526 explores the same idea with a separate cache process. Not for merge as is.

Problem

Lambda customers cannot control how often their functions fetch feature flag configuration, or how long an invocation waits for it. Every tracer poll that goes through the Extension's flag config route (#1510) reaches the CDN. Each warm environment repeats that fetch on the tracer's poll interval.

flowchart LR
  SDK[tracer flag provider] -- "every poll" --> R[Extension route]
  R -- "every poll" --> CDN[(Datadog CDN)]
  R -. "no cache, no freshness policy" .- R
Loading

Cause

The route forwards each request as it arrives and keeps nothing.

This PR

An opt-in cache on the same route (DD_FEATURE_FLAGS_EXTENSION_CACHE_ENABLED, off by default, so #1510's behavior is unchanged). The Extension owns the refresh from the CDN. The tracer reads over localhost and makes no CDN requests.

flowchart LR
  SDK[dd-trace flag provider<br/>no upstream poller] -- "local read, If-None-Match" --> C[Extension cache<br/>per execution environment]
  C -- "only when older than the TTL" --> CDN[(Datadog CDN)]
  B[(bootstrap file, optional)] -.-> C
Loading

Decisions:

  • Freshness is checked at read time against the wall clock, with no background timer. Lambda freezes the environment between invocations, so a timer can't refresh during a freeze. The first read after a thaw sees the elapsed time and refreshes. The AppConfig extension handles this the same way, checking elapsed time on each invocation.
  • Customers set the policy: TTL, maximum stale age, startup wait and refresh wait. A failed or invalid refresh keeps the accepted configuration until the maximum stale age, then the route answers 5xx, and the tracer keeps its last snapshot. Datadog's 4xx (a bad key) passes through. After a failure, the cache waits 5 s before it retries Datadog.
  • Concurrent reads share one refresh, which runs in its own task, so a disconnecting tracer does not cancel it.
  • Cache entries are keyed by API key and query (dd_env, flag selection), so one environment's or key's configuration never reaches another scope. The key is stored only as a hash.
  • The cache lives only as long as the execution environment. Each new environment fetches its own configuration. A bootstrap file counts as fresh only while its createdAt is within the TTL; otherwise it is only a fallback. To avoid per-environment fetches, something has to produce fresh snapshots outside the environment, which this PR does not do.
  • The Extension starts fetching the default scope at startup, so the fetch overlaps runtime init.

examples/feature-flags-config-cache is a Node handler using the real dd-trace flag provider (dd-trace@6.25.0). It reloads from the Extension at the start of each invocation. This needs tracer.openfeature.setConfiguration, which exists at runtime but is not public dd-trace API. That gap has to be closed before this can be a supported integration. The README covers both modes and the gap. The test-mode binary reads DD_TESTMODE_FEATURE_FLAGS_CDN_URL so local tests can stand in for the CDN. Release builds ignore it.

Core code is about 420 lines, over the usual 300, because this is a prototype.

Testing

Tested commit Validation Result
leoromanovsky/datadog-lambda-extension@477efff4caa10b1e2a01686c7e88b3b7284f8e2e (this PR's head) Unit: cache rules with a fake clock and paused Tokio time, plus config parsing 12 + 2 passed
same Unit: route with cache, against a mock CDN 3 new passed; #1510's 9 still pass
same Local process: test-mode Extension binary, fake CDN, real dd-trace in one Node process per environment 9 passed
same cargo nextest run --workspace, clippy (default, test-mode, fips), fmt Passed
— AWS Lambda Not run

Test 1: Cache rules (unit)

Steps

  1. cargo nextest run --lib flag_config_cache

Result

All pass. Covered: 100 reads with 1 fetch; 32 concurrent reads with 1 fetch; a simulated freeze shorter than the TTL makes no fetch, and a longer one refreshes on the first read; invalid bodies and outages keep the accepted configuration until max stale, then fail; startup and refresh waits stop at their limits and the refresh still finishes; 304 renews the configuration and sends its ETag; scopes stay apart; 16 new environments make 16 fetches; a fresh bootstrap file makes 0 fetches, while an old one is only a fallback for its own scope.

Test 2: Real SDK against the real Extension (local process)

Steps

  1. cargo build --features test-mode --bin bottlecap-test-mode
  2. In examples/feature-flags-config-cache: npm ci && npm test

Result

9 of 9 pass. In the reload mode, dd-trace evaluates kill-switch from Extension-supplied configuration (US user on, others off). It makes no request to its agentless endpoint (a trap server) and no Remote Config request. The fake CDN gets one request, carrying the Extension's key and dd_env. 50 warm invocations make 1 CDN request (local 304s). 32 concurrent reads make 1. After the TTL, a CDN update changes the SDK's result. Corrupt and failed refreshes keep the old result. 3 fresh environments make 3 CDN requests, and a fresh bootstrap file makes 0. Exposures still reach the trace agent. In the zero-code mode, the SDK polls the Extension and the CDN still gets one request.

Limits: the freeze/thaw elapsed-time rule is tested only with a fake clock, and in the process test only by idling past the TTL (the Extension has no timer, so for the cache an idle and a frozen environment look the same). Nothing here measures Lambda's frozen clock behavior, billed duration, or memory.

🤖 Generated with Claude Code

Opt-in cache (DD_FEATURE_FLAGS_EXTENSION_CACHE_ENABLED) on the feature
flag config route. The Extension owns the CDN refresh, checks freshness
against the wall clock at read time so a thawed environment refreshes on
its first read, consolidates concurrent refreshes, keeps the accepted
config through invalid or failed refreshes up to a max stale age, bounds
startup and refresh waits, and keys entries by API key and query.

Adds a Node example that evaluates the cache with the real dd-trace flag
provider, with process-level tests against the test-mode binary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant