Skip to content

feat(gcp): read-only access for every connection, metrics switch controls collection only - #1477

Open
JeremyFunk wants to merge 8 commits into
feat/gcp-promql-queryfrom
feat/gcp-live-query-access
Open

JeremyFunk wants to merge 8 commits into
feat/gcp-promql-queryfrom
feat/gcp-live-query-access

Conversation

@JeremyFunk

@JeremyFunk JeremyFunk commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #1475 (feat/gcp-promql-query). Live queries of Google Cloud metrics are offered for every connection, also when the 5-minute poller is off. This PR makes the setup script, the connector API and the card match that.

What changes

Setup script (gcp/setup-scripts.ts)

  • The read-only service account and its grants (Monitoring Viewer and Cloud Asset Viewer on the scope, Service Usage Consumer on the host project, Token Creator for Maple's account, the APIs) are set up for every connection whenever the deployment has a Google identity. metrics_enabled no longer changes the script.
  • A re-run never removes the account. Only the cleanup script does.
  • No Google identity: the section is skipped and nothing is removed (unchanged "left as is" behaviour).
  • Read-only access never blocks log forwarding. It runs after the log section, checks its own permissions just before it, and runs in a subshell. With log forwarding on, a failure there (a missing role, or an organization policy such as domain-restricted sharing refusing Maple's account) prints the cause and what to do. The run then ends with exit 0 and "Done, except read-only access: live metric queries are not available yet", and sends no metrics report. With log forwarding off (a live-queries-only connection), the failure stops the run with exit 1, after the sink is removed. An admin with only logging rights on a folder or organization can still set up or remove log forwarding.
  • A host project without billing gets a note instead of a stop: only Maple's reads need billing.
  • The section is now called "Read-only access" in the script output.

Connector API (GcpConnectorService, v2 contract)

  • metrics_enabled switches the key-metrics poller and the hourly resource sync. It acts in Maple at once and needs no script run.
  • Both flags off is allowed: the connection then serves live queries only.
  • Without a Google identity: turning metrics_enabled on is still GcpMetricsUnavailableError (409), and logs_enabled must stay on (IntegrationsValidationError, 400). Such a deployment can't offer live queries or the poller, so a connection there has to forward logs.
  • v2 field and endpoint descriptions say what each flag now means. The contract shape is unchanged.

Card and Configure dialog

  • The switch is now called "Key metrics". Changing it goes through a confirm dialog that states the immediate effect, saves, and closes. It no longer goes through "Apply in Google Cloud".
  • The card asks for the setup script only for what the script owns: log forwarding, and a missing read-only service account ("Live metric queries start once the script has created the read-only service account.").
  • A run that reported without the read-only account (setup_reported_at set, applied_metrics_enabled null) shows a calm band, "Live queries not set up", with the remedy and Run the script again, instead of a "Setup pending" that never ends. With Key metrics on, the poller's own sign-in error says this, as before.
  • Copy under the Collect box says that live queries of any Google Cloud metric work on every connection, with or without key metrics.
  • The apply recap shows "Read-only access" in place of the metrics row.

Docs (integrations/gcp.md): updated the permissions, the switch table, the states tables and the self-hosting note, and added one sentence about live queries. The full live-query docs come in P3.

Decisions

  • applied_metrics_enabled now means the read-only service account is in place, as the last setup or cleanup run reported it. The setup script still reports "metrics":true after setting up the account, and the cleanup script still reports false. There is no DB migration and no receiver (Rust) change. Runs from before this change that reported false (metrics off) correctly read as "no account": the card shows a pending change until the script runs again.
  • The web state functions take metricsAvailable. Without it, a deployment with no Google identity would show "setup pending" forever.
  • Without a Google identity, the logs-off case keeps the existing IntegrationsValidationError, because GcpMetricsUnavailableError carries param: "metrics_enabled".
  • "Setup pending" and "Changes pending" come from log forwarding and the read-only account only. A deployment that lost its Google identity no longer shows a band for a script that can't fix anything.

Effect on existing connections

  • A logs-only connection now grants the read-only roles on its scope at its next run. Cloud Asset Viewer can also read resource metadata and IAM policies, as before for metrics connections. There is no opt-out: live queries are part of every connection.
  • Its re-run now needs what the read-only grants need: Folder IAM Admin or Organization Administrator on a folder or organization. An admin who had only Logs Configuration Writer there now stops at "Checking access", before anything changes. The docs list the role.
  • If Maple's own service account is ever rotated, the old one keeps Token Creator on each read-only account until a cleanup run. This was already true for metrics connections; what is new is that switching key metrics off no longer clears it.

Notes

  • No migration for rows with applied_metrics_enabled=false from before this change: the GCP stack is unreleased, so only dev and preview databases have them, and they show "Changes pending" until the script runs again.

Known limit

  • Turning Key metrics back on clears the last read, as before. The card measures "No metrics have arrived yet" from the last setup report, so on an established connection that neutral hint shows until the first read lands, within about 10 minutes. Accepted as a documented limit: updated_at is also bumped by the receiver and the poller, so it cannot stand in, and a new column is out of scope.

Not verified without a live system: smoke checklist

  • New project connection, both switches on: the script creates the sink and the account, and the card reaches Receiving.
  • Re-run with Key metrics off: the account and its roles stay, and the output reads "Done. Everything is in place."
  • Old connection that ran the script with metrics off (applied_metrics_enabled=false): the card shows "Changes not applied", and after a re-run the account exists.
  • Toggle Key metrics off and on in Configure: the confirm dialog appears, the dialog closes after saving, and there is no apply stage.
  • Both switches off: the connection is created, the script creates only the account, and the card is healthy.
  • Deployment without MAPLE_GCP_SERVICE_ACCOUNT_EMAIL: Log forwarding is locked on, Key metrics is unavailable, and the card is not stuck on "setup pending".
  • Cleanup script still removes the account.
  • Organization with domain-restricted sharing, log forwarding on: the script exits 0, the row shows "Live queries not set up", and log forwarding works.
  • Host project without billing: switching on monitoring.googleapis.com works. In earlier dev-project testing, cloudasset, iamcredentials and pubsub switched on without billing; monitoring is untested.

…rols collection only

The setup script creates the read-only service account for every connection
whenever the deployment has a Google identity, and only the cleanup script
removes it. metrics_enabled now switches the key-metrics poller and resource
sync alone, in Maple, and a connection may have both flags off for live
queries only.
The key-metrics switch acts in Maple at once and skips the apply stage; the
card asks for the setup script only for log forwarding and a missing read-only
service account. Copy says live queries work on every connection.
Script-pending state comes from the read-only account alone, so a deployment
without a Google identity is never stuck on setup pending; confirm label says
when the script follows; v2 descriptions no longer read as contradicting.
@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 33035206-6440-4cbd-9fc1-f55ef18266af

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Billing is checked for every connection now that each gets the read-only
account. Only Maple's reads need it, so a host project without billing gets a
note and the rest of the setup instead of a stop.
@maple-review-bot

maple-review-bot Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 9/10 · safe to merge
Read every hunk; the one path I distrusted (setup script without removal helpers) is provably unused.
quality 100/100 · no findings · tests covered · risk medium

Makes the read-only service account part of every Google Cloud connection and reduces metrics_enabled to the poller/resource-sync switch, in the setup script, the connector API, the card and the dialog. Consistent end to end and safe to merge.

  • renderGcpSetupScript sets up the reader whenever a Google identity exists, ignoring metrics_enabled, and only the cleanup removes it
  • GcpConnectorService.capabilityError allows both flags off, and refuses metrics-on / logs-off only without a Google identity
  • applied_metrics_enabled now means the reader is in place; web state functions take metricsAvailable
  • gcpCollectLock/gcpEffectNeedsScript drop last-on and keep key-metrics changes from opening the apply stage

Before merge

  • Manual · Tell existing logs-only admins their next script run now needs Folder IAM Admin or Organization Administrator on the scope, and billing on the host project
  • Manual · Cleanup reports "metrics":false, so a cleanup run before this ships makes the card show a pending change until setup runs again
What was checked
  • Setup script without a Google identity: remains/remove/unbind/api_off are only called from the removal sections (setup-scripts.ts:607-618, 694-708), so omitting REMOVAL_HELPERS at `setup-s…
  • GcpPromQlService.ts contains no metricsEnabled/appliedMetricsEnabled reference, so live queries really do not follow the switch the web copy describes
  • Both flags off is reachable in the dialog only on a deployment with a Google identity, since gcpCollectLock locks logs on a deployment without one (gcp-connector-state.ts:335-337)

d48bac5 · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@maple-review-bot

maple-review-bot Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 10/10 · safe to merge
The one file added since the last review is the regenerated OpenAPI spec, and its description matches the source verbatim.
quality 100/100 · no findings · tests not needed · risk low

The head since the last review only regenerates the iOS OpenAPI spec: the Google Cloud group description now matches the v2 domain schema.

  • apps/ios/.../openapi.json regenerates the Google Cloud group description for key metrics
What was checked
  • Spec description at openapi.json:7749 is byte-identical to packages/domain/src/http/v2/integrations-gcp.ts:395 at the head SHA
  • No other API surface (paths, schemas, field names) changed in the regeneration

56b78df · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

The read-only access section checks its own permissions and runs in a subshell
after log forwarding. With log forwarding on, a failure there (a missing role,
an organization policy refusing Maple's account) prints what to do and the run
still succeeds without a metrics report; with it off, the run fails as before.
The card shows a calm 'Live queries not set up' band instead of a setup that
never finishes.
@maple-review-bot

maple-review-bot Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 8/10 · likely safe to merge
applied_metrics_enabled now means "the read-only account exists", so every existing connector reads as a pending change until its script is re-run.
quality 100/100 · no findings · tests covered · risk medium

Read-only access moves out of the key-metrics switch: the setup script creates the reader account for every connection, reader failures are optional when log forwarding is on, and the card shows a new "Live queries not set up" band. Safe to merge.

  • renderGcpSetupScript sets the reader account up for every connection, gated only on a Google identity
  • read_only_access runs in a subshell; exit 3 signals a new account and READER_FAILED keeps a log-only run at exit 0
  • gcpMetricsState off state drops stillSetUp; readerState adds a missing state
  • The card's Key metrics switch saves at once and a new band names the missing reader
What was checked
  • read_only_access && READER=0 || READER=$? with the case on 0/3/other runs the subshell's status, not the script's (setup-scripts.ts:668)
  • Every caller of the changed signatures passes metricsAvailable: card, configure and tests (gcp-configure.tsx:1127, :1133)
  • switchingOn(READER_APIS) reads $APIS_OFF and missing, both defined before access is emitted (setup-scripts.ts:824, :844)

91b1320 · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant