Skip to content

[SVLS-10070] fix(proxy): remove finished tasks from the Runtime API proxy's JoinSet - #1509

Open
lym953 wants to merge 3 commits into
yiming.luo/listener-joinset-leakfrom
yiming.luo/proxy-joinset-leak
Open

lym953 wants to merge 3 commits into
yiming.luo/listener-joinset-leakfrom
yiming.luo/proxy-joinset-leak

Conversation

@lym953

@lym953 lym953 commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The extension can run a proxy in front of the Lambda Runtime API. It starts the proxy when the function uses Lambda Web Adapter or the Datadog wrapper, for example with App and API Protection. For each invocation, the proxy starts two tasks in a Tokio JoinSet: one for the event and one for the response. A JoinSet keeps each finished task until something joins it, and the proxy joins only at shutdown. So the set grows by two tasks per invocation for the life of the execution environment.

This is the same pattern that #1508 fixes in the listener for the tracer's start and end calls. That leak made the extension's memory grow about 14 KB per invocation in #1496. I have not measured the memory growth from the proxy.

This PR

Before each new task, the proxy removes the tasks that have finished, with JoinSet::try_join_next(). This is the same change as #1508. Each module keeps its own copy of the helper, as each already keeps its own shutdown function for its set. The proxy still joins the tasks that are running at shutdown.

This PR is based on #1508. Merge #1508 first.

Testing

Steps

  1. Run cargo test --lib proxy::interceptor.
  2. Remove the two calls before each spawn, and run the new test again.
  3. Run the new test 20 times with the fix.

Result

The new test starts a fake Lambda Runtime API and sends 100 next and response cycles through the proxy's routes. It reads each body to the end, because the proxy starts its work for a body only when the whole body passes through. Then it counts the tasks left in the set and runs the proxy's shutdown.

Check Before After
Tasks in the set after 100 cycles 200 (test fails) 2 or fewer (test passes)
Shutdown finishes in 5 seconds and empties the set not reached passes

With the fix, the new test passed in 20 of 20 runs. I did not run the proxy on Lambda. The test uses neither App and API Protection nor Lambda Web Adapter, so the tasks do no processing after the body arrives.

🤖 Generated with Claude Code

…roxy's JoinSet

The proxy in front of the Lambda Runtime API spawns two tasks per invocation
into a JoinSet that is joined only at shutdown, so the set grows by two finished
tasks per invocation. Remove finished tasks before each spawn, as the lifecycle
listener does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The helper-only test would pass even without production task reaping, so a process-level regression is needed to check bounded memory growth after warmup on Linux and clean shutdown.

1 open issue
What changed in this PR

This PR prevents completed tasks from accumulating in the Lambda Runtime API proxy, applying the lifecycle listener’s cleanup pattern from #1508.

Changes:

  • Drain completed tasks before spawning invocation-event and response tasks.
  • Preserve shutdown handling for tasks still running.
  • Add a unit test verifying that the cleanup helper removes completed tasks.
File Description
bottlecap/​src/​proxy/​interceptor.rs Adds completed-task cleanup at both spawn sites and a helper unit test.

Review effort: Balanced

Comment thread bottlecap/src/proxy/interceptor.rs Resolved
@datadog-official

datadog-official Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 263b912 | Docs | Give us feedback!

lym953 and others added 2 commits November 19, 2026 16:21
…on cycles

Replace the helper-only test with one that sends 100 next/response cycles
through the proxy's router to a fake Runtime API, then checks that the set
holds at most two tasks and that shutdown finishes. Without the calls before
each spawn, the set holds 200 tasks and the test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the proxy task-cleanup change.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@lym953
lym953 marked this pull request as ready for review November 19, 2026 21:37
@lym953
lym953 requested a review from a team as a code owner November 19, 2026 21:37
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T21:38:22.727354Z 263b912 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 263b912a95

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +588 to +589
#[tokio::test]
async fn test_invocation_cycles_keep_join_set_bounded() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add the required process-level regression test

This memory-leak reliability fix only adds an in-process router test with a no-op invocation processor; it never launches the extension binary. The root AGENTS.md requires each reliability fix to include a process-level test using the harness described in bottlecap/tests/HARNESS.md. Add a harness regression that exercises repeated Runtime API proxy invocations through the real binary and checks externally observable memory growth and shutdown behavior; retain this unit test for the precise task-count assertion.

AGENTS.md reference: AGENTS.md:L21-L21

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants