Support PostgreSQL connection secrets

.agents/skills/update-against-main/SKILL.md

@@ -0,0 +1,35 @@
+---
+name: update-against-main
+description: Merge agent-substrate/substrate main into the kagent-dev/substrate fork's main branch, resolve conflicts, validate the result, and safely update the fork. Use only when explicitly synchronizing the fork's main branch with upstream main. Do not use for updating, rebasing, or resolving conflicts in feature branches or pull requests.
+---
+
+# Update Against Main
+
+This skill applies only to synchronizing the fork's `main` branch. Do not invoke it for a feature branch or PR merely because that branch is behind or conflicts with `main`.
+
+1. Confirm the worktree, current branch, tracking branch, and remotes. Do not disturb unrelated changes.
+2. Fetch `origin/main` and `upstream/main`, inspect their divergence, and create a dated backup branch from `origin/main`.
+3. Rebuild `main` from `upstream/main` by replaying only intentional fork feature commits in dependency order. Drop merge commits and fork commits superseded by upstream.
+4. Resolve conflicts in favor of current upstream APIs while preserving the remaining fork features. Inspect the resulting diff and linear history.
+5. Keep Helm charts synchronized with their corresponding manifests. When either changes, inspect and update the other while preserving intentional Helm templating and conditionals, then run `make verify-helm-template` and `make verify-crd-chart` and compare any relevant resources not covered by those checks.
+6. Run `make test` and `make verify`.
+7. Run the real Kind E2E matrix from `.github/workflows/pr-workflow.yaml`, but use agentgateway for all fork testing:
+   - Use a dedicated cluster name and kubeconfig; record the temporary assets created by this run. Before recreating with `hack/create-kind-cluster.sh`, delete any old cluster owned by this sync using `hack/kind.sh delete cluster --name "$cluster_name"` with its dedicated `KUBECONFIG`.
+   - Install the control plane with `hack/install-ate-kind.sh --deploy-ate-system --atenet-dataplane=agentgateway`.
+   - Deploy the micro-VM demo with `hack/run-microvm-demo-kind.sh --skip-control-plane` so it does not reinstall the control plane.
+   - Deploy the gVisor counter demo and both standard egress demos.
+   - The full gVisor suite: `hack/run-e2e-kind.sh -v -args --no-color`
+   - The full micro-VM suite with the CI environment: `E2E_SANDBOX_CLASS=microvm hack/run-e2e-kind.sh -v -args --no-color`
+   - Switch egress to agentgateway sdsmint, then run the MITM trust and targeted networking lanes for both runtimes exactly as the workflow specifies.
+   - Verify the live router and egress workloads use agentgateway. Never use Envoy for fork validation.
+8. Treat `go test ./internal/e2e/...` without `-args --e2e` as compilation/package testing, not E2E coverage.
+9. Do not push when unit, verification, or E2E checks fail or cannot run. Report the exact blocker instead.
+10. After all checks pass, verify the worktree and rewritten commits, then update the fork with `git push --force-with-lease origin main`. Never use an unguarded force push.
+11. Clean up temporary assets before finishing, including on failure or cancellation:
+   - Stop this run's test/install processes and port-forwards. Save any diagnostics needed to explain failures before tearing down workloads.
+   - Delete the task-owned Kind cluster with `hack/kind.sh delete cluster --name "$cluster_name"` using its dedicated `KUBECONFIG`. Verify both the cluster and its node containers are gone before removing the kubeconfig.
+   - Remove this run's disposable assets: generated micro-VM disks and images, downloaded bundles, build outputs, scratch scripts, and temporary kubeconfigs. Remove task-only Docker images, containers, and volumes once no longer in use. Preserve shared assets, caches, registries, and unrelated clusters; do not use global Docker prune commands.
+   - After a successful push, remove clean temporary worktrees with `git worktree remove` from another checkout. Preserve backup branches, unpushed commits, uncommitted changes, and diagnostics needed for unresolved failures.
+   - If teardown stalls (for example, Docker reports no exit event), inspect only the task's node containers, retry scoped deletion once, and report any remaining resources and exact blocker. Do not restart the global Docker daemon or kill unrelated processes. Report cleanup separately from validation so leftover assets are not hidden by passing tests.
+
+Use the current CI workflow as the source of truth for cluster setup, images, demos, runtime coverage, and environment variables, with the agentgateway-only override above. Never claim E2E passed unless workloads ran against the cluster.

.github/workflows/agentgateway-image.yaml

@@ -0,0 +1,181 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: Build agentgateway image
+
+on:
+  workflow_dispatch:
+    inputs:
+      repository:
+        description: Agentgateway source repository (owner/repo)
+        type: string
+        required: true
+        default: agentgateway/agentgateway
+      ref:
+        description: Agentgateway full commit SHA, branch, or tag
+        type: string
+        required: true
+      tag:
+        description: Image tag (defaults to the first 12 characters of the resolved commit SHA)
+        type: string
+        required: false
+
+permissions:
+  contents: read
+
+env:
+  REGISTRY_IMAGE: ghcr.io/${{ github.repository }}/agentgateway
+
+jobs:
+  source:
+    runs-on: ubuntu-24.04
+    outputs:
+      sha: ${{ steps.source.outputs.sha }}
+      short_sha: ${{ steps.source.outputs.short_sha }}
+      tag: ${{ steps.source.outputs.tag }}
+    steps:
+      - name: Checkout agentgateway
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          repository: ${{ inputs.repository }}
+          ref: ${{ inputs.ref }}
+          persist-credentials: false
+
+      - name: Resolve source revision and image tag
+        id: source
+        env:
+          IMAGE_TAG: ${{ inputs.tag }}
+        run: |
+          sha=$(git rev-parse HEAD)
+          short_sha=${sha:0:12}
+          tag=${IMAGE_TAG:-$short_sha}
+          if [[ ! "$tag" =~ ^[a-zA-Z0-9_][a-zA-Z0-9_.-]{0,127}$ ]]; then
+            echo "::error::Image tag must be 1-128 characters, start with a letter, digit, or underscore, and contain only letters, digits, underscores, periods, or hyphens."
+            exit 1
+          fi
+          {
+            echo "sha=$sha"
+            echo "short_sha=$short_sha"
+            echo "tag=$tag"
+          } >> "$GITHUB_OUTPUT"
+
+  build:
+    needs: source
+    permissions:
+      contents: read
+      packages: write
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - platform: linux/amd64
+            runner: ubuntu-24.04
+            artifact: linux-amd64
+          - platform: linux/arm64
+            runner: ubuntu-24.04-arm
+            artifact: linux-arm64
+    runs-on: ${{ matrix.runner }}
+    steps:
+      - name: Checkout agentgateway
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          repository: ${{ inputs.repository }}
+          ref: ${{ needs.source.outputs.sha }}
+          persist-credentials: false
+
+      - name: Log in to GHCR
+        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
+
+      - name: Build and push by digest
+        id: build
+        uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
+        with:
+          context: .
+          platforms: ${{ matrix.platform }}
+          tags: ${{ env.REGISTRY_IMAGE }}
+          outputs: type=image,push-by-digest=true,name-canonical=true,push=true
+          build-args: |
+            VERSION=0.0.0-alpha.${{ needs.source.outputs.short_sha }}
+            GIT_REVISION=${{ needs.source.outputs.sha }}
+          labels: |
+            org.opencontainers.image.source=https://github.com/${{ github.repository }}
+            org.opencontainers.image.revision=${{ needs.source.outputs.sha }}
+            org.opencontainers.image.url=https://github.com/${{ inputs.repository }}/commit/${{ needs.source.outputs.sha }}
+
+      - name: Export digest
+        env:
+          DIGEST: ${{ steps.build.outputs.digest }}
+        run: |
+          mkdir -p "$RUNNER_TEMP/digests"
+          touch "$RUNNER_TEMP/digests/${DIGEST#sha256:}"
+
+      - name: Upload digest
+        uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
+        with:
+          name: digest-${{ matrix.artifact }}
+          path: ${{ runner.temp }}/digests/*
+          if-no-files-found: error
+          retention-days: 1
+
+  push:
+    needs:
+      - source
+      - build
+    runs-on: ubuntu-24.04
+    permissions:
+      packages: write
+    steps:
+      - name: Download digests
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
+        with:
+          path: ${{ runner.temp }}/digests
+          pattern: digest-linux-*
+          merge-multiple: true
+
+      - name: Log in to GHCR
+        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
+
+      - name: Push multi-architecture image
+        working-directory: ${{ runner.temp }}/digests
+        env:
+          IMAGE_TAG: ${{ needs.source.outputs.tag }}
+          SOURCE_URL: https://github.com/${{ inputs.repository }}/commit/${{ needs.source.outputs.sha }}
+        run: |
+          images=()
+          for digest in *; do
+            images+=("${REGISTRY_IMAGE}@sha256:${digest}")
+          done
+          image="${REGISTRY_IMAGE}:${IMAGE_TAG}"
+          docker buildx imagetools create --tag "$image" "${images[@]}"
+          docker buildx imagetools inspect "$image"
+          digest=$(docker buildx imagetools inspect "$image" --format '{{json .}}' | jq -er '.manifest.digest')
+          {
+            echo "Image: \`$image@$digest\`"
+            echo "Source: $SOURCE_URL"
+            echo "Platforms: linux/amd64, linux/arm64"
+          } >> "$GITHUB_STEP_SUMMARY"

.github/workflows/helm-e2e.yaml

@@ -0,0 +1,151 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: helm-e2e
+on:
+  pull_request:
+  push:
+    branches: [main]
+permissions:
+  contents: read
+jobs:
+  e2e-test:
+    runs-on: ubuntu-latest
+    env:
+      VERSION: helm-e2e
+      E2E_ATENET_DATAPLANE: agentgateway
+      E2E_CREDENTIAL_PROVIDER: "1"
+      E2E_EGRESS_MITM: "1"
+      ARTIFACTS: ${{ github.workspace }}/_artifacts
+    steps:
+    - name: Checkout
+      uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5.1.0
+    - name: Setup Go
+      uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
+      with:
+        go-version-file: go.mod
+    - name: Setup Helm
+      uses: azure/setup-helm@v4
+    - name: Test Helm chart
+      run: |
+        helm plugin install https://github.com/helm-unittest/helm-unittest.git --version 1.0.3 --verify=false
+        make helm-test
+    - name: Cache micro-VM assets
+      uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
+      with:
+        path: bin/microvm-assets/amd64
+        key: microvm-assets-amd64-${{ hashFiles('hack/microvm-assets/assemble.sh') }}
+    - name: Enable KVM
+      run: |
+        echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \
+          | sudo tee /etc/udev/rules.d/99-kvm4all.rules
+        sudo udevadm control --reload-rules
+        sudo udevadm trigger --name-match=kvm
+    - name: Create cluster
+      run: hack/create-kind-cluster.sh
+    - name: Label nodes with the installed version
+      run: kubectl label nodes --all ate.dev/substrate-version=${VERSION}
+    - name: Create install namespace
+      run: kubectl create namespace ate-system
+    - name: Install observability fixtures
+      run: |
+        kubectl apply -f manifests/ate-install/kind/otel-collector.yaml
+        kubectl apply -f manifests/ate-install/kind/prometheus.yaml
+    - name: Build chart images
+      run: |
+        # Pushed under the image's real path (kagent-dev/substrate/<component>):
+        # the chart composes {registry}/{repository}/{component}, so the local
+        # registry serves each image where the default repository expects it --
+        # the same path-preserving rule a production mirror follows.
+        for component in ateapi atecontroller atelet podcertcontroller atenet credential-provider/kubernetes-secrets; do
+          KO_DOCKER_REPO="localhost:5001/kagent-dev/substrate/${component##*/}" \
+            ./hack/run-tool.sh ko build --bare --tags helm-e2e \
+              --platform linux/amd64 "./cmd/${component}"
+        done
+    - name: Install Agent Substrate with Helm
+      run: |
+        helm upgrade --install substrate-crds charts/substrate-crds
+        helm upgrade --install substrate charts/substrate \
+          --namespace ate-system \
+          --create-namespace \
+          -f internal/e2e/suites/credentials/values.yaml \
+          --set image.registry=localhost:5001 \
+          --set image.tag=helm-e2e \
+          --set 'atelet.extraArgs[0]=--localhost-registry-replacement=kind-registry:5000' \
+          --set otel.endpoint=http://opentelemetry-collector.otel-system.svc:4317 \
+          --set postgres.resources.requests.cpu=500m
+    - name: Bootstrap mTLS authorities
+      run: |
+        hack/install-ate-kind.sh --create-podcertificate-controller-cas
+        hack/install-ate-kind.sh --create-jwt-authority-pool-secret
+        hack/install-ate-kind.sh --create-actor-id-ca-pool-secret
+        hack/install-ate-kind.sh --create-actor-id-ca-certs-secret
+        hack/install-ate-kind.sh --create-api-authentication-config
+        hack/install-ate-kind.sh --create-egress-mitm-ca-pool-secret
+    - name: Wait for Helm install
+      run: |
+        helm upgrade substrate charts/substrate \
+          --namespace ate-system \
+          --reuse-values \
+          --wait --timeout=10m
+    - name: Enable NFS
+      run: |
+        sudo modprobe nfs || true
+        sudo modprobe nfsd || true
+    - name: Install CSI NFS driver
+      run: hack/install-ate-kind.sh --setup-csi=nfs
+    - name: Deploy micro-VM counter demo
+      # The deploy creates the substrate ActorTemplate and waits for its golden
+      # snapshot internally; the ActorTemplate CRD (and its Ready condition)
+      # no longer exists to wait on.
+      run: hack/run-microvm-demo-kind.sh --skip-control-plane
+    - name: Deploy gVisor counter demo
+      run: hack/install-ate-kind.sh --deploy-demo-counter
+    - name: Deploy egress demo
+      run: hack/install-ate-kind.sh --deploy-demo-egress-mitm
+    - name: Run E2E tests (gVisor)
+      # Identity rotates the cluster-wide egress CA. Run it last so HTTPS
+      # tests keep a stable trust bundle and gateway signing certificate.
+      env:
+        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-gvisor.xml
+      run: hack/run-e2e-kind.sh -v -skip '^TestActorIdentity_' -args --no-color
+    - name: Run E2E tests (micro-VM)
+      env:
+        E2E_SANDBOX_CLASS: microvm
+        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-microvm.xml
+      run: hack/run-e2e-kind.sh ./internal/e2e/suites/demo -v -args --no-color
+    - name: Run E2E tests (identity and trust rotation)
+      env:
+        E2E_JUNIT_FILE: ${{ github.workspace }}/_artifacts/helm-identity.xml
+      run: hack/run-e2e-kind.sh ./internal/e2e/suites/identity -v -args --no-color
+    - name: Require every lane to have run tests
+      if: always()
+      run: |
+        make build-junittool
+        bin/junittool verify -manifest "${ARTIFACTS}/expected-junit.txt"
+    - name: Upload test results
+      if: always()
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+      with:
+        name: test-results-helm-e2e
+        path: ${{ env.ARTIFACTS }}
+        if-no-files-found: error
+    - name: Dump diagnostics on failure
+      if: failure()
+      run: |
+        kubectl --context kind-kind get workerpool,pods -A -o wide || true
+        for p in $(kubectl --context kind-kind get pods -n ate-system -o name 2>/dev/null); do
+          echo "=== logs: ate-system/${p} ==="
+          kubectl --context kind-kind logs -n ate-system "$p" --all-containers --tail=300 || true
+        done