[justinwlin]

Note

Companion PR: runpod/flash#394. This PR documents runpod-flash 1.21.0 as released, including the 90-second first-run failure and the broken flash deploy curl example. #394 fixes both in the SDK. The pages were tested in the same live runs. Review them together, and revisit the 90-second guidance once #394 ships.

Brings the Flash quickstart and the SDXL and text-generation tutorials in line with what actually happens on runpod-flash 1.21.0. Every page was run end to end against live endpoints, and the code, outputs, and timings come from those runs.

What changed

• Deploy step: the quickstart and both tutorials now run flash deploy before the script. Since 1.15.0, calling an @Endpoint function looks up a deployed endpoint, so without it the first run fails with endpoint ... not found. Each page shows the real deploy output.
• Model caching: both tutorials load the model once per worker (module-level cache plus global) and report model_cached. Warm SDXL calls went from about 25s to 7s end to end. Both have a "Complete file" accordion that matches the steps.
• Build size: accelerate is gone from both tutorials. It pulls torch and several GB of CUDA wheels into the local build on Linux, which is the "Disk quota exceeded" failure. SDXL builds to 50 MB and GPT-2 to 38 MB.
• First run past 90 seconds: documents what 1.21.0 does when a cold worker takes longer than 90s: timed out after 90s ... may not be deployed, or KeyError on the output. The fix is to run the script again. FLASH_SENTINEL_TIMEOUT is no longer recommended because above 90 it turns the timeout into the KeyError. Also covers the /health check (throttled means waiting for a GPU).
• Call from any language: new quickstart Step 9 with /run and /status curl calls (including exporting RUNPOD_API_KEY, which flash login doesn't set), and a warning that {"input": {}} (the example flash deploy prints in 1.21.0) fails after about 40s.
• Quickstart code: the matrix multiply now runs on the GPU with torch. Before, it used numpy on the worker's CPU and only read the GPU name.
• Corrections: measured timings replace guessed ones (SDXL 30 steps is about 4-5s on an RTX 4090, not 20-40s). Also fixed:
  • max_new_tokens instead of max_length
  • the flash init layout (gpu_worker.py, not workers/gpu/endpoint.py)
  • stale model IDs (stabilityai/stable-diffusion-2-1 returns 401)
  • SDXL Turbo needs a redeploy
  • the network volume example now sets HF_HOME, so models are actually cached
  • the real auth error text
  • the endpoint defined in multiple files error
  • redeploying during a request
  • warm workers can serve the previous version for a few minutes after a redeploy
  • shared troubleshooting no longer says scripts provision endpoints on their own
• Python versions: local 3.10 to 3.14 works. The worker version comes from --python-version, then python_version on a resource config such as LiveServerless, then 3.12. A conflict fails the build. @Endpoint has no python_version parameter.

Verification

• Quickstart: deployed the exact page code with 1.21.0 and ran Steps 5, 6, 8, and 9 plus the cleanup commands. Cold first run took 45-49s, and once failed at 90s, which is why the page warns about it. Warm runs took 1-3s. The build is 0.2 MB.
• SDXL: deployed the exact "Complete file". Cold first run took 74-82s (load 18s, generate 4.9s), and warm runs took 7s (generate 3.9s). Three prompts in parallel finished in 18s on one warm worker.
• Text generation: deployed the exact "Complete file". The first two runs failed at 90s while the worker was throttled waiting for a GPU. The third took 51s, and warm runs took 1.4s. Step 8 returned three completions in 3.3s.
• Errors: reproduced KeyError with FLASH_SENTINEL_TIMEOUT=300, the 401 without credentials, {"input": {}} failing after 40s, and defined in multiple files. All test endpoints and apps were deleted afterwards.
• Checks: mintlify broken-links found none. The only parse error is in README.md, which was already there.
• Review: five Codex passes against the 1.21.0 source, plus a fresh pass on the full PR that found 4 more issues (all fixed). The last pass found no defects.

Related Flash PRs

runpod-flash stops getting fixes on 2026-10-29, so the runpod/flash changes need to merge and release before then.

PR	What it does	Status (2026-10-27)	Requires before merge
runpod-workers/flash#113	Installs torchvision in the GPU load-balancer image	Approved	Nothing. Must be merged and released before runpod/flash#390.
runpod/flash#390	Keeps torch's dependency tree out of flash build archives	Approved	runpod-workers/flash#113 merged and released.
runpod/flash#394	Waits for cold-start jobs instead of failing at 90s, and prints a working curl example after flash deploy	Draft	Nothing.
#923	Matches the Flash quickstart and tutorials to a live run on 1.21.0	Ready for review	Nothing. Documents 1.21.0 as released. Revisit the 90-second guidance once runpod/flash#394 ships.
runpod/flash#369	Reads runpodctl's top-level apikey as a credentials fallback	Draft	Nothing.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
runpod-docs	🟢 Ready	View Preview	Oct 27, 2026, 5:00 PM

💡 Tip: Enable Automations to automatically generate PRs for you.