Skip to content

Adrian: AWS bucket policy can omit data planes that use the bucket #2058

Description

@GregorShear

Blocked by estuary/flow#3380. That issue adds the backend query this work consumes.

Summary

The storage mapping dialog generates an AWS bucket policy. The policy must hold the IAM ARN of every data plane that uses the bucket. The UI builds that list from an incomplete client-side join. The generated policy can omit a live data plane. The customer then removes that data plane's access to the bucket.

Bug

useAwsArnsForBucket at src/components/admin/Settings/StorageMappings/Dialog/ConnectionTest/instructions/awsHooks.ts:11 joins three client-side sources:

  • the unsaved connections in the open dialog,
  • the storage mappings from useStorageMappings(),
  • the data planes from useDataPlanes().

AwsInstructions.tsx:108 renders the result as a complete bucket policy and as this command:

aws s3api put-bucket-policy --bucket <bucket> --policy '<policy>'

put-bucket-policy replaces the whole bucket policy. An ARN that the UI omits loses access to the bucket. That data plane then fails to persist journal fragments for the collections under its mapping.

src/lang/en-US/AdminPage.ts:241 states the guarantee to the customer:

This policy includes all data planes that use this bucket - existing connections will be preserved.

The client-side join cannot hold that guarantee. Two causes:

  1. Pagination. useStorageMappings() reads the StorageMappingQuery document at src/api/gql/storageMappings.ts:136. That document sends no first argument and no after argument. The server returns one page.
  2. Read scope. The storageMappings resolver returns only the prefixes that the caller can read. Two tenants can point their mappings at one bucket. No change in the UI reaches the mappings that the resolver hides.

Expected

The generated policy holds the identity of every data plane that uses the bucket. The result does not depend on the caller's read scope or on the size of the first page.

Fix

estuary/flow#3380 adds a query that answers the question on the server:

storageBucketDataPlanes(
    provider: StorageProvider!
    bucket: String!
    containerName: String       # Azure
    storageAccountName: String  # Azure
): [DataPlane!]!

Steps:

  1. Run npm run codegen:local against a local backend that holds the new query. Never edit the files under src/gql-types/ by hand.
  2. Replace the storage mapping source in useAwsArnsForBucket with the new query. Keep the unsaved connections source. The dialog still needs to include the connection that the admin is about to save.
  3. Remove the useDataPlanes() join from the hook. The new query returns the data plane records.
  4. Point the GCP panel and the Azure panel at the same query. They read gcpServiceAccountEmail and azureApplicationClientId.

If the backend review lands a scoped lookup with a complete: Boolean! field, replace the message at src/lang/en-US/AdminPage.ts:241 with a warning. The UI must not state a guarantee that the response does not carry.

Acceptance criteria

  • The generated policy holds the ARN of a data plane that another tenant's mapping attaches to the same bucket.
  • The generated policy holds the ARNs of the unsaved connections in the open dialog.
  • The generated policy holds every ARN when the account holds more storage mappings than one page.
  • The GCP panel and the Azure panel read their identities from the new query.
  • A failed query blocks the policy display. The UI shows an error. The UI never renders a partial policy.

Reference files

Path Role
.../Dialog/ConnectionTest/instructions/awsHooks.ts The ARN join and the policy builder
.../Dialog/ConnectionTest/instructions/AwsInstructions.tsx The rendered policy and the CLI command
.../Dialog/ConnectionTest/ConnectionTestContext.tsx The unsaved connections source
src/api/gql/storageMappings.ts All storage mapping queries and mutations
src/lang/en-US/AdminPage.ts:241 The guarantee message shown to the customer
docs/GRAPHQL.md URQL patterns and migration status

Setup notes

  • The paths above start at src/components/admin/Settings/StorageMappings/.
  • src/gql-types/ is generated output. Run npm run codegen for the production schema. Run npm run codegen:local for a local backend.
  • Codegen emits only the fields that the target backend exposes. This work needs a local backend that holds the change from Adrian: Add a control-plane query for the data planes that use a storage bucket flow#3380.
  • Run npm run typecheck, npm run lint, and npm run format before you open a pull request.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions