Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions flash/apps/deploy-apps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -468,16 +468,16 @@ You can also override the environment:
FLASH_APP=my-app FLASH_ENV=production python call_inference.py
```

### Error without context
### Endpoint not found

If Flash cannot resolve the app context and you haven't set the environment variables, it raises an error:
If no endpoint with that name is deployed in the resolved app and environment, Flash raises an error:

```text
RuntimeError: no flash context for endpoint 'inference'. either:
- use 'flash dev' for local development
- set FLASH_APP and FLASH_ENV to target a deployed environment
RuntimeError: endpoint 'inference' not found in app 'my-app' environment 'production'. deploy it first with 'flash deploy'.
```

Deploy the app with `flash deploy`, or set `FLASH_APP` and `FLASH_ENV` to target an app you've already deployed.

### Automatic context in deployed workers

When Flash deploys your app, it automatically sets `FLASH_APP` and `FLASH_ENV` environment variables on each worker. This enables cross-endpoint communication within your deployed application without additional configuration.
Expand Down
32 changes: 14 additions & 18 deletions flash/cli/build.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ Comma-separated list of packages to exclude from the build (e.g., `torch,torchvi
</ResponseField>

<ResponseField name="--python-version" type="string">
Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Overrides per-resource `python_version` declarations and local interpreter detection.
Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Defaults to the `python_version` declared on your resource configs, or 3.12 if none declares one. The build fails if the flag conflicts with a declared `python_version`.
</ResponseField>

## What happens during build

1. **Python version resolution**: Resolves the target Python version from CLI flag, resource configs, or your local interpreter.
2. **Function discovery**: Finds all `@Endpoint` decorated functions.
3. **Grouping**: Groups functions by their endpoint configuration.
1. **Function discovery**: Finds all `@Endpoint` decorated functions.
2. **Grouping**: Groups functions by their endpoint configuration.
3. **Python version resolution**: Resolves the target Python version from the CLI flag, resource configs, or the 3.12 default.
4. **Manifest generation**: Creates `.flash/flash_manifest.json` with endpoint definitions.
5. **Source fingerprinting**: Computes a SHA-256 fingerprint of your source files to detect code changes between deployments.
6. **Dependency installation**: Installs Python packages for Linux x86_64.
Expand Down Expand Up @@ -116,36 +116,32 @@ Flash automatically handles cross-platform builds:

### Python version in deployed workers

Flash workers support Python 3.10, 3.11, 3.12, and 3.13. The target version is determined by:
Flash workers support Python 3.10, 3.11, 3.12, and 3.13. Your local Python version doesn't change the worker version. Flash picks the worker version in this order:

1. **CLI flag:** The `--python-version` flag takes precedence.
2. **Resource config:** The `python_version` parameter on your endpoint configs.
3. **Local interpreter:** Your local Python version (from `sys.version_info`) when neither is specified.
1. **CLI flag:** `--python-version`.
2. **Resource config:** the `python_version` field on a resource config class such as `LiveServerless`. The `@Endpoint` decorator has no `python_version` parameter.
3. **Default:** Python 3.12.

All resources in a Flash app must use the same Python version because Flash ships a single tarball for the entire app. If resources declare conflicting versions, the build fails.

<Warning>
**Breaking change in Flash 0.x:** Flash now matches your local Python version by default instead of always defaulting to Python 3.12. If you upgrade Flash and your local Python differs from 3.12, your first deploy will trigger a rolling release. To maintain consistent behavior across team members, declare `python_version` explicitly on your resource configs or use `--python-version` on the CLI.
</Warning>
All endpoints in an app use the same version because Flash ships a single tarball for the entire app. The build fails if resources declare different versions, or if `--python-version` conflicts with a version a resource declares.

| Version | Cold start | Notes |
|---------|------------|-------|
| 3.12 | No overhead | PyTorch pre-installed in base image |
| 3.13 | +~7 GB on GPU | Alternative Python install required |
| 3.11 | +~7 GB on GPU | Alternative Python install required |
| 3.13 | +~7 GB on GPU | PyTorch reinstalled for this interpreter |
| 3.11 | +~7 GB on GPU | PyTorch reinstalled for this interpreter |
| 3.10 | +~7 GB on GPU | EOL 2026-10-31; consider migrating to 3.11+ |

If your local Python version is not supported (for example, 3.9 or 3.14), the build fails with an actionable error message listing the supported versions.
You can run `flash build` from any local Python version from 3.10 to 3.14. Flash downloads dependency wheels for the worker's Python version, not your local one.

Image tags follow the pattern `py{version}-{tag}` (for example, `runpod/flash:py3.12-latest`).

## Managing deployment size

Runpod Serverless has a **1.5GB deployment limit**. Flash automatically excludes packages that are pre-installed in the base image:
Runpod Serverless has a **1.5GB deployment limit**. Flash automatically leaves these packages out of the archive because the GPU worker images provide them:

- `torch`, `torchvision`, `torchaudio`, `triton`

These packages are excluded at archive time, so you don't need to specify them manually.
You don't need to exclude them manually. CPU worker images don't include them, so code on CPU workers can't import these packages.

### Manual exclusions

Expand Down
4 changes: 2 additions & 2 deletions flash/cli/deploy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Build and launch a local Docker-based preview environment instead of deploying t
</ResponseField>

<ResponseField name="--python-version" type="string">
Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Overrides per-resource `python_version` declarations and local interpreter detection.
Target Python version for worker images (3.10, 3.11, 3.12, or 3.13). Defaults to the `python_version` declared on your resource configs, or 3.12 if none declares one. The build fails if the flag conflicts with a declared `python_version`.
</ResponseField>

## What happens during deployment
Expand Down Expand Up @@ -222,7 +222,7 @@ This builds your project and runs it in Docker containers locally:

## Managing deployment size

Runpod Serverless has a **1.5GB limit**. Flash automatically excludes packages that are pre-installed in the base image (`torch`, `torchvision`, `torchaudio`, `numpy`, `triton`).
Runpod Serverless has a **1.5GB limit**. Flash automatically leaves `torch`, `torchvision`, `torchaudio`, and `triton` out of the upload because the GPU worker images provide them. CPU workers don't include them, so CPU code can't import these packages.

If the deployment is still too large, use `--exclude` to skip additional packages:

Expand Down
44 changes: 8 additions & 36 deletions flash/configuration/parameters.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ This page provides a complete reference for all parameters available on the `End
| `scaler_value` | `int` | Scaling threshold | `4` |
| `template` | `PodTemplate` | Pod template overrides | `None` |
| `min_cuda_version` | `str` or `CudaVersion` | Minimum CUDA version for GPU host selection | `"12.8"` (GPU) / `None` (CPU) |
| `python_version` | `str` | Python version for the worker image | Local Python |

## Parameter details

Expand Down Expand Up @@ -603,52 +602,25 @@ This parameter has no effect on CPU endpoints.
Valid CUDA versions: `CudaVersion.V11_1`, `V11_4`, `V11_7`, `V11_8`, `V12_0`, `V12_1`, `V12_2`, `V12_3`, `V12_4`, `V12_6`, `V12_8` (or equivalent strings like `"12.4"`). Invalid values raise a `ValueError`.
</Note>

### python_version
### Worker Python version

**Type**: `str`
**Default**: Local Python version

Sets the Python version for the worker image. Supported values: `"3.10"`, `"3.11"`, `"3.12"`, and `"3.13"`.

When you don't specify a Python version, Flash matches your local interpreter (the Python version you run Flash from). The resolution order is:

1. `--python-version` CLI flag (highest priority)
2. `python_version` declared on resource configs
3. Your local Python version (`sys.version_info`)

```python
from runpod_flash import Endpoint, GpuGroup

# Explicitly set Python 3.11 for this endpoint
@Endpoint(
name="legacy-model",
gpu=GpuGroup.ANY,
python_version="3.11"
)
async def process(data): ...
`Endpoint` doesn't take a `python_version` parameter. Workers run Python 3.12 by default, whatever your local version. To target another version, pass `--python-version` to `flash build` or `flash deploy`:

# Uses your local Python version (e.g., 3.12 if that's what you're running)
@Endpoint(name="modern-model", gpu=GpuGroup.ANY)
async def infer(data): ...
```bash
flash deploy --python-version 3.11
```

All resources in a Flash app must use the same Python version because Flash ships a single tarball for the entire app. If resources declare conflicting versions, the build fails.
Supported values are `3.10`, `3.11`, `3.12`, and `3.13`. The flag applies to every endpoint in the app because Flash ships a single tarball for the entire app.

<Warning>
**Breaking change:** Flash now matches your local Python version by default instead of always defaulting to Python 3.12. If your local Python differs from 3.12, your first deploy after upgrading Flash will trigger a rolling release. For consistent behavior across team members, declare `python_version` explicitly or use the `--python-version` CLI flag.
</Warning>

<Warning>
Python 3.10, 3.11, and 3.13 workers incur approximately 7 GB of additional cold-start overhead on GPU endpoints because the alternative Python interpreter must be installed alongside the base image's PyTorch environment.
Python 3.10, 3.11, and 3.13 workers incur approximately 7 GB of additional cold-start overhead on GPU endpoints because the GPU image's PyTorch is built for Python 3.12, so PyTorch and its CUDA packages are installed again for the selected interpreter.
</Warning>

<Note>
Python 3.10 reaches end-of-life on 2026-10-31. Consider migrating to Python 3.11 or later.
</Note>

If your local Python version is not supported (for example, 3.9 or 3.14), the build fails with an actionable error message listing the supported versions.

The `--python-version` CLI flag on `flash build` and `flash deploy` overrides both per-resource declarations and local interpreter detection.
You can run Flash from any local Python version from 3.10 to 3.14.

## EndpointJob

Expand Down Expand Up @@ -690,7 +662,7 @@ These changes restart all workers:
- Datacenter (`datacenter`)
- Flashboot setting (`flashboot`)
- CUDA version requirement (`min_cuda_version`)
- Python version (`python_version`)
- Worker Python version (`--python-version`)

Workers are temporarily unavailable during recreation (typically 30-90 seconds).

Expand Down
2 changes: 1 addition & 1 deletion flash/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Flash requires a Runpod account with a verified email address.
### Install Flash

<Note>
Flash requires [Python 3.10, 3.11, 3.12, or 3.13](https://www.python.org/downloads/) and runs natively on macOS and Linux. Windows users can run Flash through [WSL2](/flash/windows-wsl2).
Flash requires [Python 3.10 to 3.14](https://www.python.org/downloads/) and runs natively on macOS and Linux. Windows users can run Flash through [WSL2](/flash/windows-wsl2).
</Note>

Install Flash using `pip` or `uv`:
Expand Down
Loading
Loading