diff --git a/docs/develop/dotnet/workers/serverless-workers/cloud-run.mdx b/docs/develop/dotnet/workers/serverless-workers/cloud-run.mdx index 0137c3a0b4..7b96de07f9 100644 --- a/docs/develop/dotnet/workers/serverless-workers/cloud-run.mdx +++ b/docs/develop/dotnet/workers/serverless-workers/cloud-run.mdx @@ -21,7 +21,7 @@ import { ReleaseNoteHeader } from '@site/src/components'; On a [GCP Cloud Run worker pool](https://cloud.google.com/run/docs/resource-model#worker-pools), you run a standard long-lived Temporal Worker. Register Workflows and Activities the same way you would with any other .NET Worker, and Temporal Cloud scales the pool up and down as work arrives and drains. -A Cloud Run Worker needs no Cloud Run-specific package. +A Cloud Run Worker needs no Cloud Run-specific runtime or handler. The one addition to a standard Worker is [Worker Versioning](/worker-versioning), which is required for Serverless Workers. For the end-to-end deployment guide covering the Worker Pool, IAM, and compute configuration, see [Deploy a Serverless Worker on GCP Cloud Run](/production-deployment/worker-deployments/serverless-workers/cloud-run). @@ -133,7 +133,50 @@ public static string Process(IReadOnlyList items) For how scale-in decisions are made, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run#lifecycle). -## Add observability {/* #add-observability */} +## Configure OpenTelemetry {/* #opentelemetry */} -A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else. -For how to configure metrics export and OpenTelemetry tracing interceptors, see [Observability - .NET SDK](/develop/dotnet/platform/observability) and the [SDK metrics reference](/references/sdk-metrics). +Run an OpenTelemetry Collector as a sidecar in the Worker Pool. Call `ApplyGoogleCloudRunOpenTelemetryDefaults()` +to configure tracing and Temporal Core metrics for OTLP export to the Collector at `localhost:4317`. Load the Client +connection settings with `ClientEnvConfig.LoadClientConnectOptions()`, then configure both plugins before connecting: + + +[src/Gcp/CloudRun/Program.cs](https://github.com/temporalio/samples-dotnet/blob/main/src/Gcp/CloudRun/Program.cs) +```cs +// The Cloud Run Id plugin sets the client identity to "{instanceId}@{revision}" from Cloud Run +// metadata at connect time; every Worker created from the client inherits it. +connectOptions.Plugins = new ITemporalClientPlugin[] { new CloudRunIdPlugin() }; + +// ApplyGoogleCloudRunOpenTelemetryDefaults adds the tracing interceptor and a runtime exporting Core +// metrics + traces over OTLP to the collector sidecar; the returned handle owns the tracer provider. +// Both plugins configure the same connect options. +using var telemetry = connectOptions.ApplyGoogleCloudRunOpenTelemetryDefaults(); + +var client = await TemporalClient.ConnectAsync(connectOptions); +``` + + +After the Worker stops, call `telemetry.FlushAsync(TimeSpan.FromSeconds(2))` to flush traces within the Cloud Run +termination window. For the Collector sidecar configuration and deployment steps, see the +[.NET Cloud Run sample](https://github.com/temporalio/samples-dotnet/tree/main/src/Gcp/CloudRun). + +## Set a Worker identity {/* #worker-identity */} + +Use `CloudRunIdPlugin` to identify each Worker instance as `@`. The plugin reads Cloud Run environment variables and instance metadata when the Client connects, then Workers created from that Client inherit the identity. + +Register the plugin on the Client options before connecting: + +```csharp +using Temporalio.Client; +using Temporalio.Common.EnvConfig; +using Temporalio.Extensions.Gcp.CloudRun.Id; + +// ... + var connectOptions = ClientEnvConfig.LoadClientConnectOptions(); + connectOptions.Plugins = new ITemporalClientPlugin[] { new CloudRunIdPlugin() }; + var client = await TemporalClient.ConnectAsync(connectOptions); + // ... +``` + +The [preceding example](#opentelemetry) registers `CloudRunIdPlugin` alongside the OpenTelemetry defaults. + +The plugin requires the Cloud Run metadata server, so it fails when the Worker runs outside Cloud Run. For a complete example, see the [.NET Cloud Run sample](https://github.com/temporalio/samples-dotnet/tree/main/src/Gcp/CloudRun). diff --git a/docs/develop/go/workers/serverless-workers/cloud-run.mdx b/docs/develop/go/workers/serverless-workers/cloud-run.mdx index 8bdab7b9c4..ce6dd36486 100644 --- a/docs/develop/go/workers/serverless-workers/cloud-run.mdx +++ b/docs/develop/go/workers/serverless-workers/cloud-run.mdx @@ -21,7 +21,7 @@ import { ReleaseNoteHeader } from '@site/src/components'; On a [GCP Cloud Run worker pool](https://cloud.google.com/run/docs/resource-model#worker-pools), you run a standard long-lived Temporal Worker. Register Workflows and Activities the same way you would with any other Go Worker, and Temporal Cloud scales the pool up and down as work arrives and drains. -A Cloud Run Worker needs no Cloud Run-specific package. +A Cloud Run Worker needs no Cloud Run-specific runtime or handler. The one addition to a standard Worker is [Worker Versioning](/worker-versioning), which is required for Serverless Workers. For the end-to-end deployment guide covering the Worker Pool, IAM, and compute configuration, see [Deploy a Serverless Worker on GCP Cloud Run](/production-deployment/worker-deployments/serverless-workers/cloud-run). @@ -110,7 +110,152 @@ func MyActivity(ctx context.Context, input MyInput) (string, error) { For how scale-in decisions are made, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run#lifecycle). -## Add observability {/* #add-observability */} +## Configure OpenTelemetry {/* #opentelemetry */} -A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else. -For how to configure metrics export and OpenTelemetry tracing interceptors, see [Observability - Go SDK](/develop/go/platform/observability) and the [SDK metrics reference](/references/sdk-metrics). +Run an OpenTelemetry Collector as a sidecar in the Worker Pool. The Cloud Run OpenTelemetry plugin exports metrics and traces by OTLP gRPC to the Collector at `localhost:4317`. By default, it derives the service name from `OTEL_SERVICE_NAME`, `CLOUD_RUN_WORKER_POOL`, or `K_SERVICE`. + +Create both Cloud Run plugins before connecting, then add them to the Client options. Client plugins that implement +`worker.Plugin` also apply to Workers created from that Client: + + +[gcp/cloudrun/worker/main.go](https://github.com/temporalio/samples-go/blob/main/gcp/cloudrun/worker/main.go) +```go +// ... + // Exports OTLP metrics and traces to the collector sidecar (defaults to localhost:4317). + otelPlugin, err := otel.NewPlugin(ctx, otel.PluginOptions{}) + if err != nil { + log.Fatalln("Unable to create OpenTelemetry plugin", err) + } + // Sets the client identity to "@" from Cloud Run instance metadata. + idPlugin := id.NewCloudRunIDPlugin() + + // A client plugin that also implements worker.Plugin is applied to workers automatically. + clientOptions, err := envconfig.LoadDefaultClientOptions() + if err != nil { + log.Fatalln("Unable to load Temporal client options", err) + } + clientOptions.Plugins = append(clientOptions.Plugins, otelPlugin, idPlugin) + + c, err := client.Dial(clientOptions) + if err != nil { + log.Fatalln("Unable to create Temporal client", err) + } + log.Println("Client identity:", idPlugin.Metadata().Identity()) +``` + + +Configure the Collector sidecar to receive OTLP gRPC on `localhost:4317`, export traces to Google Cloud, and export metrics to Google Managed Service for Prometheus: + + +[gcp/cloudrun/otel-collector-config.yaml](https://github.com/temporalio/samples-go/blob/main/gcp/cloudrun/otel-collector-config.yaml) +```yaml +# Google-Built OpenTelemetry Collector sidecar: metrics -> Managed Service for Prometheus, traces -> Cloud Trace. +# Auth uses the worker-pool service account (ADC). +receivers: + otlp: + protocols: + grpc: + endpoint: localhost:4317 + +processors: + memory_limiter: + check_interval: 1s + limit_percentage: 65 + spike_limit_percentage: 20 + resource_detection: + detectors: [gcp] + timeout: 10s + # Batch traces for throughput. Do NOT batch the cumulative-metrics pipeline: a shutdown flush + # could be merged with a recent periodic export of the same series and rejected as a duplicate. + batch/traces: + send_batch_size: 200 + timeout: 5s + # Rename Temporal datapoint labels that collide with the target labels Managed Service for Prometheus injects (e.g. namespace). + transform/collision: + metric_statements: + - context: datapoint + statements: + - set(attributes["exported_location"], attributes["location"]) + - delete_key(attributes, "location") + - set(attributes["exported_cluster"], attributes["cluster"]) + - delete_key(attributes, "cluster") + - set(attributes["exported_namespace"], attributes["namespace"]) + - delete_key(attributes, "namespace") + - set(attributes["exported_job"], attributes["job"]) + - delete_key(attributes, "job") + - set(attributes["exported_instance"], attributes["instance"]) + - delete_key(attributes, "instance") + - set(attributes["exported_project_id"], attributes["project_id"]) + - delete_key(attributes, "project_id") + # The Telemetry API expects the Google Cloud project in gcp.project_id. + transform/set_project_id: + error_mode: ignore + trace_statements: + - set(resource.attributes["gcp.project_id"], resource.attributes["gcp.project.id"]) where resource.attributes["gcp.project.id"] != nil + - set(resource.attributes["gcp.project_id"], resource.attributes["cloud.account.id"]) where resource.attributes["gcp.project_id"] == nil and resource.attributes["cloud.account.id"] != nil + +exporters: + googlemanagedprometheus: + otlp_grpc: + endpoint: telemetry.googleapis.com:443 + compression: none + balancer_name: pick_first + auth: + authenticator: googleclientauth + +extensions: + googleclientauth: + health_check: + endpoint: 0.0.0.0:13133 + +service: + extensions: [googleclientauth, health_check] + pipelines: + metrics: + receivers: [otlp] + processors: [memory_limiter, resource_detection, transform/collision] + exporters: [googlemanagedprometheus] + traces: + receivers: [otlp] + processors: [memory_limiter, resource_detection, transform/set_project_id, batch/traces] + exporters: [otlp_grpc] + telemetry: + logs: + level: info +``` + + +On shutdown, stop the Worker and call `otelPlugin.Shutdown` with a deadline shorter than Cloud Run's termination window so telemetry can flush. For a Collector configuration that exports traces to Google Cloud and metrics to Google Managed Service for Prometheus, see the [Go Cloud Run sample](https://github.com/temporalio/samples-go/tree/main/gcp/cloudrun). + +## Set a Worker identity {/* #worker-identity */} + +Use the Cloud Run ID plugin to identify each Worker instance as `@`. The plugin reads Cloud Run environment variables and instance metadata once when the Client connects, then Workers created from that Client inherit the identity. + +Register the plugin on the Client options before connecting: + +```go +import ( + "log" + + "go.temporal.io/sdk/client" + "go.temporal.io/sdk/contrib/envconfig" + "go.temporal.io/sdk/contrib/gcp/cloudrun/id" +) + +// ... + clientOptions, err := envconfig.LoadDefaultClientOptions() + if err != nil { + log.Fatalln("Unable to load Temporal client options", err) + } + clientOptions.Plugins = append(clientOptions.Plugins, id.NewCloudRunIDPlugin()) + c, err := client.Dial(clientOptions) + if err != nil { + log.Fatalln("Unable to create Temporal client", err) + } + defer c.Close() + // ... +``` + +The [preceding example](#opentelemetry) registers `id.NewCloudRunIDPlugin()` alongside the OpenTelemetry plugin. + +The plugin requires the Cloud Run metadata server, so it fails when the Worker runs outside Cloud Run. For a complete example, see the [Go Cloud Run sample](https://github.com/temporalio/samples-go/tree/main/gcp/cloudrun). diff --git a/docs/develop/java/workers/serverless-workers/cloud-run.mdx b/docs/develop/java/workers/serverless-workers/cloud-run.mdx index 51325bca20..3ed29ad8de 100644 --- a/docs/develop/java/workers/serverless-workers/cloud-run.mdx +++ b/docs/develop/java/workers/serverless-workers/cloud-run.mdx @@ -20,7 +20,7 @@ import { ReleaseNoteHeader } from '@site/src/components'; On a [GCP Cloud Run worker pool](https://cloud.google.com/run/docs/resource-model#worker-pools), you run a standard long-lived Temporal Worker. Register Workflows and Activities the same way you would with any other Java Worker, and Temporal Cloud scales the pool up and down as work arrives and drains. -A Cloud Run Worker needs no Cloud Run-specific package. +A Cloud Run Worker needs no Cloud Run-specific runtime or handler. The one addition to a standard Worker is [Worker Versioning](/worker-versioning), which is required for Serverless Workers. For the end-to-end deployment guide covering the Worker Pool, IAM, and compute configuration, see [Deploy a Serverless Worker on GCP Cloud Run](/production-deployment/worker-deployments/serverless-workers/cloud-run). @@ -146,7 +146,146 @@ public class GreetingActivitiesImpl implements GreetingActivities { For how scale-in decisions are made, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run#lifecycle). -## Add observability {/* #add-observability */} +## Configure OpenTelemetry {/* #opentelemetry */} -A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else. -For how to configure metrics export and OpenTelemetry tracing interceptors, see [Observability - Java SDK](/develop/java/platform/observability) and the [SDK metrics reference](/references/sdk-metrics). +Run an OpenTelemetry Collector as a sidecar in the Worker Pool. The Cloud Run OpenTelemetry plugin exports metrics and traces by OTLP gRPC to the Collector at `localhost:4317`. It configures the endpoint and service name from Cloud Run defaults. + +Create both Cloud Run plugins and register them on `WorkflowServiceStubsOptions` before you create the Client: + + +[gcp/cloud-run/src/main/java/io/temporal/samples/gcp/cloudrun/CloudRunWorker.java](https://github.com/temporalio/samples-java/blob/main/gcp/cloud-run/src/main/java/io/temporal/samples/gcp/cloudrun/CloudRunWorker.java) +```java +ClientConfigProfile profile = ClientConfigProfile.load(); +CloudRunOpenTelemetryPlugin otelPlugin = CloudRunOpenTelemetryPlugin.newBuilder().build(); +CloudRunIdPlugin idPlugin = new CloudRunIdPlugin(); + +WorkflowServiceStubsOptions serviceOptions = + WorkflowServiceStubsOptions.newBuilder(profile.toWorkflowServiceStubsOptions()) + .setPlugins(otelPlugin, idPlugin) + .build(); +WorkflowServiceStubs service = WorkflowServiceStubs.newServiceStubs(serviceOptions); +``` + + +Configure the Collector sidecar to receive OTLP gRPC on `localhost:4317`, export traces to Google Cloud, and export metrics to Google Managed Service for Prometheus: + + +[gcp/cloud-run/collector-config.yaml](https://github.com/temporalio/samples-java/blob/main/gcp/cloud-run/collector-config.yaml) +```yaml +receivers: + otlp: + protocols: + grpc: + endpoint: localhost:4317 + +processors: + # Batch traces for throughput. Do not add this processor to the cumulative metrics pipeline: + # a shutdown flush can otherwise be batched with a recent periodic export of the same series. + batch/traces: + send_batch_max_size: 200 + send_batch_size: 200 + timeout: 5s + memory_limiter: + # This is the collector's memory polling cadence, not the SDK metric export interval. + check_interval: 1s + limit_percentage: 65 + spike_limit_percentage: 20 + resource_detection: + detectors: [gcp] + timeout: 10s + # Avoid collisions with labels that Google Managed Service for Prometheus adds. + transform/collision: + metric_statements: + - context: datapoint + statements: + - set(attributes["exported_location"], attributes["location"]) + - delete_key(attributes, "location") + - set(attributes["exported_cluster"], attributes["cluster"]) + - delete_key(attributes, "cluster") + - set(attributes["exported_namespace"], attributes["namespace"]) + - delete_key(attributes, "namespace") + - set(attributes["exported_job"], attributes["job"]) + - delete_key(attributes, "job") + - set(attributes["exported_instance"], attributes["instance"]) + - delete_key(attributes, "instance") + - set(attributes["exported_project_id"], attributes["project_id"]) + - delete_key(attributes, "project_id") + # The Telemetry API expects the Google Cloud project in gcp.project_id. + transform/set_project_id: + error_mode: ignore + trace_statements: + - set(resource.attributes["gcp.project_id"], resource.attributes["gcp.project.id"]) where resource.attributes["gcp.project.id"] != nil + - set(resource.attributes["gcp.project_id"], resource.attributes["cloud.account.id"]) where resource.attributes["gcp.project_id"] == nil and resource.attributes["cloud.account.id"] != nil + +exporters: + googlemanagedprometheus: + # Google Cloud's supported OTLP path for traces is the Telemetry API. + otlp_grpc: + endpoint: telemetry.googleapis.com:443 + compression: none + balancer_name: pick_first + auth: + authenticator: googleclientauth + +extensions: + # Cloud Run container dependencies require a startup probe. This endpoint is also used for the + # collector liveness probe in worker-pool.yaml. + health_check: + endpoint: 0.0.0.0:13133 + googleclientauth: + +service: + extensions: + - health_check + - googleclientauth + pipelines: + metrics/otlp: + receivers: [otlp] + processors: [memory_limiter, resource_detection, transform/collision] + exporters: [googlemanagedprometheus] + traces: + receivers: [otlp] + processors: [memory_limiter, resource_detection, transform/set_project_id, batch/traces] + exporters: [otlp_grpc] + # Feed collector self-metrics back through the metrics pipeline. + telemetry: + metrics: + readers: + - periodic: + exporter: + otlp: + protocol: grpc + endpoint: http://localhost:4317 + insecure: true +``` + + +On shutdown, stop the `WorkerFactory`, then run `otelPlugin.newFlushHook()` before the Cloud Run termination window ends. For a Collector configuration that exports traces to Google Cloud and metrics to Google Managed Service for Prometheus, see the [Java Cloud Run sample](https://github.com/temporalio/samples-java/tree/main/gcp/cloud-run). + +## Set a Worker identity {/* #worker-identity */} + +Use `CloudRunIdPlugin` to identify each Worker instance as `@`. Register it on +`WorkflowServiceStubsOptions` before creating the service stubs. Clients and Workers created from those service stubs +inherit the identity: + +```java +import io.temporal.client.WorkflowClient; +import io.temporal.envconfig.ClientConfigProfile; +import io.temporal.gcp.cloudrun.id.CloudRunIdPlugin; +import io.temporal.serviceclient.WorkflowServiceStubs; +import io.temporal.serviceclient.WorkflowServiceStubsOptions; + +// ... + ClientConfigProfile profile = ClientConfigProfile.load(); + WorkflowServiceStubsOptions serviceOptions = + WorkflowServiceStubsOptions.newBuilder(profile.toWorkflowServiceStubsOptions()) + .setPlugins(new CloudRunIdPlugin()) + .build(); + WorkflowServiceStubs service = WorkflowServiceStubs.newServiceStubs(serviceOptions); + WorkflowClient client = WorkflowClient.newInstance(service, profile.toWorkflowClientOptions()); + // ... +``` + +The [preceding example](#opentelemetry) registers `CloudRunIdPlugin` alongside the OpenTelemetry plugin. + +The plugin requires Cloud Run metadata, so Client creation fails when the Worker runs outside Cloud Run. For a complete example, see the [Java Cloud Run sample](https://github.com/temporalio/samples-java/tree/main/gcp/cloud-run). diff --git a/docs/develop/python/workers/serverless-workers/cloud-run.mdx b/docs/develop/python/workers/serverless-workers/cloud-run.mdx index 5b0c58ff8f..df3281222b 100644 --- a/docs/develop/python/workers/serverless-workers/cloud-run.mdx +++ b/docs/develop/python/workers/serverless-workers/cloud-run.mdx @@ -21,7 +21,7 @@ import { ReleaseNoteHeader } from '@site/src/components'; On a [GCP Cloud Run worker pool](https://cloud.google.com/run/docs/resource-model#worker-pools), you run a standard long-lived Temporal Worker. Register Workflows and Activities the same way you would with any other Python Worker, and Temporal Cloud scales the pool up and down as work arrives and drains. -A Cloud Run Worker needs no Cloud Run-specific package. +A Cloud Run Worker needs no Cloud Run-specific runtime or handler. The one addition to a standard Worker is [Worker Versioning](/worker-versioning), which is required for Serverless Workers. For the end-to-end deployment guide covering the Worker Pool, IAM, and compute configuration, see [Deploy a Serverless Worker on GCP Cloud Run](/production-deployment/worker-deployments/serverless-workers/cloud-run). @@ -127,7 +127,133 @@ async def my_activity(items: list[str]) -> str: For how scale-in decisions are made, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run#lifecycle). -## Add observability {/* #add-observability */} +## Configure OpenTelemetry {/* #opentelemetry */} + +Run an OpenTelemetry Collector as a sidecar in the Worker Pool. The Cloud Run OpenTelemetry plugin exports metrics and traces by OTLP gRPC to the Collector at `localhost:4317`. It configures the endpoint, service name, Core SDK metrics, and tracer provider from Cloud Run defaults. + +Create the plugin and pass it to `Client.connect`. The sample also registers `CloudRunIdPlugin` on the same Client and +reads its connection settings from the environment: + + +[gcp/cloud_run/worker.py](https://github.com/temporalio/samples-python/blob/main/gcp/cloud_run/worker.py) +```py +# OpenTelemetryPlugin exports Core metrics and traces to the collector +# sidecar; CloudRunIdPlugin sets the client identity from instance metadata. +plugin = OpenTelemetryPlugin(add_temporal_spans=True) +client = await Client.connect( + settings.address, + namespace=settings.namespace, + api_key=settings.api_key, + tls=settings.tls, + plugins=[plugin, CloudRunIdPlugin()], +) +``` + + +Configure the Collector sidecar to receive OTLP gRPC on `localhost:4317`, export traces to Google Cloud, and export metrics to Google Managed Service for Prometheus: + + +[gcp/cloud_run/collector-config.yaml](https://github.com/temporalio/samples-python/blob/main/gcp/cloud_run/collector-config.yaml) +```yaml +receivers: + otlp: + protocols: + grpc: + endpoint: localhost:4317 + +processors: + batch/traces: + send_batch_max_size: 200 + send_batch_size: 200 + timeout: 5s + memory_limiter: + check_interval: 1s + limit_percentage: 65 + spike_limit_percentage: 20 + resource_detection: + detectors: [gcp] + timeout: 10s + # Rename Temporal datapoint labels that collide with the target labels Managed Service for Prometheus injects (e.g. namespace). + transform/collision: + metric_statements: + - context: datapoint + statements: + - set(attributes["exported_location"], attributes["location"]) + - delete_key(attributes, "location") + - set(attributes["exported_cluster"], attributes["cluster"]) + - delete_key(attributes, "cluster") + - set(attributes["exported_namespace"], attributes["namespace"]) + - delete_key(attributes, "namespace") + - set(attributes["exported_job"], attributes["job"]) + - delete_key(attributes, "job") + - set(attributes["exported_instance"], attributes["instance"]) + - delete_key(attributes, "instance") + - set(attributes["exported_project_id"], attributes["project_id"]) + - delete_key(attributes, "project_id") + # The Telemetry API expects the Google Cloud project in gcp.project_id. + transform/set_project_id: + error_mode: ignore + trace_statements: + - set(resource.attributes["gcp.project_id"], resource.attributes["gcp.project.id"]) where resource.attributes["gcp.project.id"] != nil + - set(resource.attributes["gcp.project_id"], resource.attributes["cloud.account.id"]) where resource.attributes["gcp.project_id"] == nil and resource.attributes["cloud.account.id"] != nil + +exporters: + googlemanagedprometheus: + otlp_grpc: + endpoint: telemetry.googleapis.com:443 + compression: none + balancer_name: pick_first + auth: + authenticator: googleclientauth + +extensions: + googleclientauth: + health_check: + endpoint: 0.0.0.0:13133 + +service: + extensions: [googleclientauth, health_check] + pipelines: + metrics: + receivers: [otlp] + processors: [memory_limiter, resource_detection, transform/collision] + exporters: [googlemanagedprometheus] + traces: + receivers: [otlp] + processors: + [memory_limiter, resource_detection, transform/set_project_id, batch/traces] + exporters: [otlp_grpc] +``` + + +Wait until the Collector accepts connections before starting the Worker. On shutdown, call `plugin.shutdown()` after the Worker stops so traces can flush. For a Collector configuration that exports traces to Google Cloud and metrics to Google Managed Service for Prometheus, see the [Python Cloud Run sample](https://github.com/temporalio/samples-python/tree/main/gcp/cloud_run). + +## Set a Worker identity {/* #worker-identity */} + +Use `CloudRunIdPlugin` to identify each Worker instance as `@`. It reads Cloud Run environment variables and instance metadata when +the Client connects, then Workers created from that Client inherit the identity. + +Register the plugin when connecting the Client: + +```python +from temporalio.client import Client +from temporalio.contrib.gcp.cloud_run.id import CloudRunIdPlugin + +from gcp.cloud_run.settings import load_settings + +# ... + settings = load_settings() + client = await Client.connect( + settings.address, + namespace=settings.namespace, + api_key=settings.api_key, + tls=settings.tls, + plugins=[CloudRunIdPlugin()], + ) + # ... +``` + +The [preceding example](#opentelemetry) registers `CloudRunIdPlugin` alongside the OpenTelemetry plugin. -A Cloud Run Worker emits the same traces and metrics as a Worker anywhere else. -For how to configure metrics export and OpenTelemetry tracing interceptors, see [Observability - Python SDK](/develop/python/platform/observability) and the [SDK metrics reference](/references/sdk-metrics). +The ID plugin requires the Cloud Run metadata server, so it fails when the Worker runs outside Cloud Run. For a complete +example, see the [Python Cloud Run sample](https://github.com/temporalio/samples-python/tree/main/gcp/cloud_run). diff --git a/docs/production-deployment/worker-deployments/serverless-workers/cloud-run/index.mdx b/docs/production-deployment/worker-deployments/serverless-workers/cloud-run/index.mdx index 20ea0f1cfb..fe91ad06a8 100644 --- a/docs/production-deployment/worker-deployments/serverless-workers/cloud-run/index.mdx +++ b/docs/production-deployment/worker-deployments/serverless-workers/cloud-run/index.mdx @@ -27,9 +27,9 @@ A Cloud Run Worker Pool runs long-lived instances that poll the Task Queue conti through the Cloud Run admin API as work arrives and drains. For how the pool scales and how instances are shut down, see [Serverless Workers on GCP Cloud Run](/serverless-workers/cloud-run). -Cloud Run runs a standard long-lived Worker, so there is no Cloud Run-specific handler or package and any Temporal SDK -can run on a Worker Pool. The one addition is [Worker Versioning](/worker-versioning), which Serverless Workers require. The tabs below cover -the SDKs with a Cloud Run guide today. +Cloud Run runs a standard long-lived Worker, so no Cloud Run-specific handler is needed and any Temporal SDK can run on +a Worker Pool. Serverless Workers require [Worker Versioning](/worker-versioning). The Go, Python, Java, and .NET examples +below also register a Cloud Run ID plugin to give each instance a distinct Worker Identity. ## Prerequisites {/* #prerequisites */} @@ -75,6 +75,7 @@ import os from temporalio.client import Client from temporalio.common import VersioningBehavior, WorkerDeploymentVersion +from temporalio.contrib.gcp.cloud_run.id import CloudRunIdPlugin from temporalio.worker import Worker, WorkerDeploymentConfig from my_workflows import MyWorkflow @@ -85,7 +86,8 @@ async def main() -> None: client = await Client.connect( os.environ["TEMPORAL_ADDRESS"], namespace=os.environ["TEMPORAL_NAMESPACE"], - api_key=os.environ.get("TEMPORAL_API_KEY"), + api_key=os.environ["TEMPORAL_API_KEY"].strip(), + plugins=[CloudRunIdPlugin()], tls=True, ) worker = Worker( @@ -125,8 +127,8 @@ class MyWorkflow: ... ``` -For more on the Python Worker setup, see -[Serverless Workers on GCP Cloud Run - Python SDK](/develop/python/workers/serverless-workers/cloud-run). +The example uses the Cloud Run ID plugin to identify each Worker as `@`. For more on the +Python Worker setup, see [Serverless Workers on GCP Cloud Run - Python SDK](/develop/python/workers/serverless-workers/cloud-run). @@ -140,6 +142,7 @@ import ( "go.temporal.io/sdk/client" "go.temporal.io/sdk/contrib/envconfig" + "go.temporal.io/sdk/contrib/gcp/cloudrun/id" "go.temporal.io/sdk/worker" "go.temporal.io/sdk/workflow" @@ -147,7 +150,12 @@ import ( ) func main() { - c, err := client.Dial(envconfig.MustLoadDefaultClientOptions()) + clientOptions, err := envconfig.LoadDefaultClientOptions() + if err != nil { + log.Fatalln("Unable to load Temporal client options", err) + } + clientOptions.Plugins = append(clientOptions.Plugins, id.NewCloudRunIDPlugin()) + c, err := client.Dial(clientOptions) if err != nil { log.Fatalln("Unable to create client", err) } @@ -190,8 +198,8 @@ w := worker.New(c, os.Getenv("TEMPORAL_TASK_QUEUE"), worker.Options{ If a Version is set and neither is specified, registration panics with `workflow type does not have a versioning behavior`. -For more on the Go Worker setup, see -[Serverless Workers on GCP Cloud Run - Go SDK](/develop/go/workers/serverless-workers/cloud-run). +The example uses the Cloud Run ID plugin to identify each Worker as `@`. For more on the +Go Worker setup, see [Serverless Workers on GCP Cloud Run - Go SDK](/develop/go/workers/serverless-workers/cloud-run). @@ -252,9 +260,10 @@ For more on the TypeScript Worker setup, see package example; import io.temporal.client.WorkflowClient; -import io.temporal.client.WorkflowClientOptions; import io.temporal.common.VersioningBehavior; import io.temporal.common.WorkerDeploymentVersion; +import io.temporal.envconfig.ClientConfigProfile; +import io.temporal.gcp.cloudrun.id.CloudRunIdPlugin; import io.temporal.serviceclient.WorkflowServiceStubs; import io.temporal.serviceclient.WorkflowServiceStubsOptions; import io.temporal.worker.Worker; @@ -263,23 +272,17 @@ import io.temporal.worker.WorkerFactory; import io.temporal.worker.WorkerOptions; public class WorkerMain { - public static void main(String[] args) { - String apiKey = System.getenv("TEMPORAL_API_KEY"); + public static void main(String[] args) throws java.io.IOException { + ClientConfigProfile profile = ClientConfigProfile.load(); WorkflowServiceStubs service = WorkflowServiceStubs.newServiceStubs( - WorkflowServiceStubsOptions.newBuilder() - .setTarget(System.getenv("TEMPORAL_ADDRESS")) - .setEnableHttps(true) - .addApiKey(() -> apiKey) + WorkflowServiceStubsOptions.newBuilder(profile.toWorkflowServiceStubsOptions()) + .setPlugins(new CloudRunIdPlugin()) .build()); WorkflowClient client = - WorkflowClient.newInstance( - service, - WorkflowClientOptions.newBuilder() - .setNamespace(System.getenv("TEMPORAL_NAMESPACE")) - .build()); + WorkflowClient.newInstance(service, profile.toWorkflowClientOptions()); WorkerFactory factory = WorkerFactory.newInstance(client); @@ -307,22 +310,21 @@ Each Workflow must have a [versioning behavior](/worker-versioning#versioning-be `AUTO_UPGRADE`. Set it per Workflow with the `@WorkflowVersioningBehavior` annotation, or set a Worker-level default with `setDefaultVersioningBehavior` as shown above. -For more on the Java Worker setup, see -[Serverless Workers on GCP Cloud Run - Java SDK](/develop/java/workers/serverless-workers/cloud-run). +The example uses the Cloud Run ID plugin to identify each Worker as `@`. For more on the +Java Worker setup, see [Serverless Workers on GCP Cloud Run - Java SDK](/develop/java/workers/serverless-workers/cloud-run). ```csharp using Temporalio.Client; +using Temporalio.Common.EnvConfig; +using Temporalio.Extensions.Gcp.CloudRun.Id; using Temporalio.Worker; -var client = await TemporalClient.ConnectAsync(new(Environment.GetEnvironmentVariable("TEMPORAL_ADDRESS")!) -{ - Namespace = Environment.GetEnvironmentVariable("TEMPORAL_NAMESPACE")!, - ApiKey = Environment.GetEnvironmentVariable("TEMPORAL_API_KEY"), - Tls = new(), -}); +var connectOptions = ClientEnvConfig.LoadClientConnectOptions(); +connectOptions.Plugins = new ITemporalClientPlugin[] { new CloudRunIdPlugin() }; +var client = await TemporalClient.ConnectAsync(connectOptions); var options = new TemporalWorkerOptions(Environment.GetEnvironmentVariable("TEMPORAL_TASK_QUEUE")!) { @@ -342,8 +344,8 @@ Each Workflow must have a [versioning behavior](/worker-versioning#versioning-be `AutoUpgrade`. Set it per Workflow with `[Workflow(VersioningBehavior = ...)]`, or set a Worker-level default with `DefaultVersioningBehavior` as shown above. -For more on the .NET Worker setup, see -[Serverless Workers on GCP Cloud Run - .NET SDK](/develop/dotnet/workers/serverless-workers/cloud-run). +The example uses the Cloud Run ID plugin to identify each Worker as `@`. For more on the +.NET Worker setup, see [Serverless Workers on GCP Cloud Run - .NET SDK](/develop/dotnet/workers/serverless-workers/cloud-run). @@ -446,13 +448,15 @@ For more on the Rust Worker setup, see +The Cloud Run ID plugins in the Go, Python, Java, and .NET examples require the Cloud Run metadata server. To run these +Workers outside Cloud Run, omit the plugin registration and add it back before deploying. + :::tip -Workers on Cloud Run use the same code as a traditional long-lived Worker, so the SDK defaults the -[Worker Identity](/workers#worker-identity) to the process ID and hostname. On Cloud Run that resolves to -`1@localhost`, which makes a Worker harder to identify in the Temporal UI. Set your own Worker Identity, built from the -[Cloud Run environment variables](https://cloud.google.com/run/docs/container-contract#worker-pools-env-vars), to help -identify your Serverless Workers. +The TypeScript, Ruby, and Rust examples use the SDK's default [Worker Identity](/workers#worker-identity). On Cloud Run, +it can resolve to `1@localhost`, which does not distinguish Worker instances in the Temporal UI. Set a unique Worker +Identity using [Cloud Run environment variables](https://cloud.google.com/run/docs/container-contract#worker-pools-env-vars) +and instance metadata. ::: @@ -693,8 +697,7 @@ gcloud run worker-pools deploy my-temporal-worker-pool-build-1 \ | `--set-env-vars` | Non-secret Worker configuration. See [Environment configuration](/develop/environment-configuration). | | `--set-secrets` | Maps a Secret Manager secret to an environment variable. Use for `TEMPORAL_API_KEY` or TLS client cert/key material. | -The environment configuration package reads environment variables and configuration files at startup. For the -full list of supported environment variables, config file format, and profiles, see +For the full list of supported environment variables, config file format, and profiles, see [Environment configuration](/develop/environment-configuration). ## 3. Grant Temporal permission to manage the Worker Pool (Cloud only) {/* #configure-iam */}