diff --git a/.claude/commands/release-prep.md b/.claude/commands/release-prep.md deleted file mode 100644 index fabff03f..00000000 --- a/.claude/commands/release-prep.md +++ /dev/null @@ -1,65 +0,0 @@ -Prepare a release for the changes on the current branch. Do the following steps in order: - -## Step 1 — Understand the changes - -Run `git diff main...HEAD` and `git log main...HEAD --oneline` to understand what was changed on this branch. Read all modified source files to understand the nature of the changes (new feature, bug fix, breaking change, docs, etc.). - -## Step 2 — Determine the new version - -Read `pyproject.toml` to get the current version. Apply SemVer rules based on the changes: -- PATCH bump: bug fixes, docs, refactors, non-breaking improvements -- MINOR bump: new non-breaking features or new public API surface -- MAJOR bump: breaking changes (changes to existing public API signatures or behavior) - -Compute the new version string (no leading `v`). - -## Step 3 — Bump version in pyproject.toml - -Edit `pyproject.toml` and update `version = "..."` to the new version. - -## Step 4 — Create PULL_REQUEST.md - -Create `PULL_REQUEST.md` at the repo root following the structure of `.github/pull_request_template.md`. Fill it in based on what you learned from the diff: - -- **Description**: clear summary of what changed and why -- **Related Issue**: leave as `Closes #` with a note to fill in the issue number -- **Type of Change**: check the relevant boxes (use `[x]`) -- **How to Test**: concrete steps to verify the change works -- **Checklist**: check all boxes that apply given the changes made -- **Breaking Changes**: fill in if applicable, otherwise remove the section -- **Additional Notes**: any relevant context for reviewers - -> **Disclaimer:** Do not include SAP-internal or customer-specific information in this PR (e.g. internal system URLs, customer names, tenant IDs, or confidential configurations). This is a public repository. - -## Step 5 — Create RELEASE.md - -Create `RELEASE.md` at the repo root using this exact template structure: - -``` -## [vX.Y.Z] - MM DD, YYYY - -### What's New -- ... - -### Improvements -- ... - -### Bug Fixes -- ... - -### Breaking Changes -> ⚠️ **Important**: This section is critical for users upgrading from previous versions -- **[Breaking Change]**: ... - -### Contributors -... -``` - -Fill in the version and today's date. Remove sections that don't apply. Infer the content from the diff — be specific about what changed (class names, method names, parameter names) so that users upgrading know exactly what to update. - -## Step 6 — Summarize - -Tell the user: -- The old and new version -- Which files were created/updated -- Any sections in PULL_REQUEST.md or RELEASE.md that still need manual input (e.g. issue number, contributors) diff --git a/.claude/skills/prep-pr/SKILL.md b/.claude/skills/prep-pr/SKILL.md new file mode 100644 index 00000000..94f1f367 --- /dev/null +++ b/.claude/skills/prep-pr/SKILL.md @@ -0,0 +1,229 @@ +--- +name: prep-pr +description: Fill in the pull request template for the current branch. Diffs the current branch against its base branch (or reads the existing PR diff), infers description, type of change, testing steps, breaking changes, and checklist state, then either creates a new PR or edits the body of an existing one. +tools: Bash, Read +compatibility: gh CLI ≥ 2.0, git, GitHub access to SAP/cloud-sdk-python +--- + +# PR Prep: SAP Cloud SDK for Python + +Fills in `.github/pull_request_template.md` from the diff of the current branch against its base, then creates or updates the GitHub PR. + +--- + +## Phase 1: Pre-flight + +Run in parallel: + +```bash +# 1a. Current branch name +git rev-parse --abbrev-ref HEAD + +# 1b. Confirm branch is pushed to origin +git ls-remote --heads origin $(git rev-parse --abbrev-ref HEAD) + +# 1c. Check if a PR already exists for this branch +gh pr list --repo SAP/cloud-sdk-python --head $(git rev-parse --abbrev-ref HEAD) \ + --json number,title,url,state,baseRefName --jq '.[0] // empty' +``` + +**Fail fast** if `1b` returns empty — the branch is not on origin. Tell the user: +> "Push the branch first (`git push -u origin HEAD`), then re-run `/prep-pr`." Stop. + +Capture: +- `CURRENT_BRANCH` — from 1a +- `EXISTING_PR` — from 1c (number, url, baseRefName — or empty if none) +- `BASE_BRANCH` — from `EXISTING_PR.baseRefName` if a PR exists; otherwise `main` + +--- + +## Phase 2: Gather the diff + +If `EXISTING_PR` exists, fetch the actual PR diff from GitHub (most accurate, includes only what the PR changes): + +```bash +gh pr diff {EXISTING_PR.number} --repo SAP/cloud-sdk-python +``` + +Otherwise diff the branch against its base locally: + +```bash +git fetch origin {BASE_BRANCH} +git diff origin/{BASE_BRANCH}...HEAD +``` + +Also get the commit log and changed file list in parallel: + +```bash +# Commits on this branch not in base +git log origin/{BASE_BRANCH}...HEAD --pretty=format:"%H %s" --reverse + +# Changed file paths only +git diff origin/{BASE_BRANCH}...HEAD --name-only +``` + +If the log is empty, tell the user: "No commits ahead of `{BASE_BRANCH}`. Nothing to open a PR for." Stop. + +--- + +## Phase 3: Analyse the changes + +### 3.1 Classify the change types + +Read the commit subjects and diff. Determine which of the following apply: + +| Type | Signal | +|---|---| +| **Bug fix** | `fix(...)` commits, regression tests added | +| **New feature** | `feat(...)` commits, new public classes/methods/exports | +| **Breaking change** | Removed/renamed public API, changed return types, made optional params required, `!` in commit subject or `BREAKING CHANGE` footer | +| **Documentation update** | Only `docs/` or docstring changes | +| **Code refactoring** | `refactor(...)` commits, no API surface change | +| **Dependency update** | Changes to `pyproject.toml` dependency pins only | + +More than one type can apply. + +### 3.2 Detect breaking changes + +A change is breaking if any of the following are true in the diff: +- A public function/method signature in `src/` was removed or renamed +- A required parameter was added to a public method +- A return type of a public method changed incompatibly +- A public class was removed or renamed +- A commit subject contains `!` after the type/scope (e.g. `feat!:`) or a `BREAKING CHANGE:` trailer + +### 3.3 Infer the related issue number + +Check commit messages and the branch name for a `#NNN` reference or an issue number pattern (e.g. branch `fix/123-something`). If found, use it as the linked issue. If not found, leave the placeholder `Closes #` and note it in the summary. + +### 3.4 Draft testing steps + +From the changed files and commit messages, derive 2–4 concrete steps a reviewer can follow to verify the change. Steps should be specific to the actual code changed (function names, CLI commands, env var names) — not generic placeholders. + +### 3.5 Evaluate the checklist + +For each checklist item, determine the most accurate state given what you can observe in the diff: + +| Item | How to evaluate | +|---|---| +| Read Contributing Guidelines | Always `[x]` — assume the contributor has read them | +| Changes solve the issue | `[x]` if a `Closes #N` reference is present; `[ ]` otherwise | +| Tests added/updated | `[x]` if `tests/` files changed alongside `src/` changes; `[ ]` if `src/` changed with no test changes | +| All tests pass locally | Always `[ ]` — leave for the author to confirm | +| Code follows guidelines | `[x]` if pre-commit/ruff checks pass in CI; `[ ]` if unsure | +| Documentation updated | `[x]` if `docs/` or `user-guide.md` files changed; `[ ]` if new public API was added without docs | +| Type hints for public APIs | `[x]` if all new/modified public functions have return types and parameter annotations in the diff; `[ ]` if any are missing | +| No sensitive information | Always `[x]` — flag in summary if anything suspicious was found in Phase 3 | +| Conventional Commits | `[x]` if all commit subjects match the pattern; `[ ]` if any don't | + +--- + +## Phase 4: Build the PR body + +Fill in the template exactly as structured below. Remove the `## Breaking Changes` section entirely if no breaking changes were detected. Remove `## Additional Notes` if there is nothing meaningful to add. + +```markdown +> **Disclaimer:** Do not include SAP-internal or customer-specific information in this PR (e.g. internal system URLs, customer names, tenant IDs, or confidential configurations). This is a public repository. + +## Description + + + +## Related Issue + +Closes # + +## Type of Change + +- [x or space] Bug fix (non-breaking change that fixes an issue) +- [x or space] New feature (non-breaking change that adds functionality) +- [x or space] Breaking change (fix or feature that would cause existing functionality to change) +- [x or space] Documentation update +- [x or space] Code refactoring +- [x or space] Dependency update + +## How to Test + + + +## Checklist + +- [x or space] I have read the [Contributing Guidelines](../CONTRIBUTING.md) +- [x or space] I have verified that my changes solve the issue +- [x or space] I have added/updated automated tests to cover my changes +- [ ] All tests pass locally +- [x or space] I have verified that my code follows the [Code Guidelines](../docs/GUIDELINES.md) +- [x or space] I have updated documentation (if applicable) +- [x or space] I have added type hints for all public APIs +- [x] My code does not contain sensitive information (credentials, tokens, etc.) +- [x or space] I have followed [Conventional Commits](https://www.conventionalcommits.org/) for commit messages + +## Breaking Changes + + +- What breaks: +- Migration path: +- Alternative approaches considered: + +## Additional Notes + + +``` + +--- + +## Phase 5: Create or update the PR + +### If no existing PR (`EXISTING_PR` is empty): + +Propose a PR title using the Conventional Commit format derived from the dominant change type: +- Single commit: use its subject directly +- Mixed commits: synthesise a title like `feat(scope): add X and fix Y` + +Show the user the proposed title, base branch, and body. Ask: +> "Ready to create the PR targeting `{BASE_BRANCH}` with this title and body? Reply `yes`, `no`, or provide an alternate title." + +On confirmation, run: + +```bash +gh pr create \ + --repo SAP/cloud-sdk-python \ + --base {BASE_BRANCH} \ + --head {CURRENT_BRANCH} \ + --title "{PROPOSED_TITLE}" \ + --body "{PR_BODY}" +``` + +### If PR already exists (`EXISTING_PR` has a number): + +Show the user the proposed body and ask: +> "PR #{NUMBER} already exists ({URL}), targeting `{BASE_BRANCH}`. Update its body with this content? Reply `yes` or `no`." + +On confirmation, run: + +```bash +gh pr edit {NUMBER} \ + --repo SAP/cloud-sdk-python \ + --body "{PR_BODY}" +``` + +--- + +## Phase 6: Summary + +After creating or updating the PR, print: + +``` +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + PR Ready +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + PR: #{NUMBER} — {URL} + Branch: {CURRENT_BRANCH} → {BASE_BRANCH} + Action: created | updated + + Items needing manual attention: + - + - + - +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` diff --git a/.claude/skills/release-prep/SKILL.md b/.claude/skills/release-prep/SKILL.md new file mode 100644 index 00000000..c9c12001 --- /dev/null +++ b/.claude/skills/release-prep/SKILL.md @@ -0,0 +1,230 @@ +--- +name: release-prep +description: Prepare a release issue for the SAP Cloud SDK for Python. Diffs the current branch against the latest tag, generates structured release notes, proposes a SemVer bump, confirms with the user, then creates a GitHub release issue with the correct labels so the automation pipeline can pick it up. +tools: Bash, Read +compatibility: gh CLI ≥ 2.0, git, GitHub write access to SAP/cloud-sdk-python +--- + +# Release Prep: SAP Cloud SDK for Python + +Prepares a release by analysing what changed since the last tag, generating release notes, and creating the GitHub issue that drives the rest of the pipeline. + +Run from the root of the `cloud-sdk-python` repository on the branch you intend to release. + +--- + +## Phase 1: Pre-flight checks + +Run all checks **in parallel**: + +```bash +# 1a. Confirm we are inside the repo +git rev-parse --show-toplevel + +# 1b. Get current branch name +git rev-parse --abbrev-ref HEAD + +# 1c. Confirm the branch is pushed to origin and remote ref exists +git ls-remote --heads origin $(git rev-parse --abbrev-ref HEAD) + +# 1d. Get the latest semver tag (the release baseline) +git tag --sort=-version:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+' | head -1 + +# 1e. Read current version from pyproject.toml +grep '^version = ' pyproject.toml | cut -d'"' -f2 +``` + +**Fail fast** if: +- `1c` returns empty — the branch is not on origin. Tell the user: "Push the branch first (`git push -u origin HEAD`), then re-run `/release-prep`." Stop here. +- `1d` returns empty — no semver tag exists yet. Use the initial commit as the baseline and note this in the summary. + +Capture: +- `CURRENT_BRANCH` — from 1b +- `LATEST_TAG` — from 1d (e.g. `v0.56.1`) +- `CURRENT_VERSION` — from 1e (e.g. `0.56.1`) + +--- + +## Phase 2: Collect the diff + +Run both commands **in parallel**: + +```bash +# All commits between the latest tag and HEAD, in reverse chronological order +git log {LATEST_TAG}..HEAD --pretty=format:"%H %s" --reverse + +# List of changed files (for breaking-change detection) +git diff {LATEST_TAG}..HEAD --name-only +``` + +If the log is empty (no commits since the last tag), tell the user: "No commits since `{LATEST_TAG}`. Nothing to release." Stop here. + +--- + +## Phase 3: Classify commits and propose version + +### 3.1 Classify by Conventional Commit type + +For each commit subject, extract the type prefix (`feat`, `fix`, `chore`, `refactor`, `docs`, `test`, `ci`, `perf`, `style`, `build`, `revert`). + +Group commits into four buckets: + +| Bucket | Conventional Commit types | +|---|---| +| **Breaking Changes** | any subject containing `!` after the type/scope, or `BREAKING CHANGE` in the footer | +| **New Features** | `feat` | +| **Bug Fixes** | `fix` | +| **Improvements & Other** | `refactor`, `perf`, `chore`, `docs`, `test`, `ci`, `style`, `build`, `revert`, unrecognised | + +### 3.2 Determine bump type + +Apply SemVer rules to the commits since `LATEST_TAG`: + +| Condition | Bump | +|---|---| +| Any breaking change present | **major** | +| Any `feat` commit present (no breaking changes) | **minor** | +| Only fixes, refactors, docs, chores, etc. | **patch** | + +### 3.3 Compute proposed version + +Parse `CURRENT_VERSION` as `MAJOR.MINOR.PATCH` and increment the appropriate segment. Reset lower segments to `0`. + +Example: `0.56.1` + minor → `0.57.0`. + +--- + +## Phase 4: Generate release notes + +Produce a markdown string following the exact structure below. Omit sections that have no entries. + +```markdown +### What's New +- **[scope]**: description of feat commit (commit abc1234) +- ... + +### Improvements +- **[scope]**: description (commit abc1234) +- ... + +### Bug Fixes +- **[scope]**: description (commit abc1234) +- ... + +### Breaking Changes +> ⚠️ **Important**: Migration steps for users upgrading from `{LATEST_TAG}`. +- **[scope]**: description and what the user must change (commit abc1234) +- ... +``` + +Rules: +- Use the commit scope (the part in parentheses) as the bold prefix. If no scope, omit the bold prefix. +- Write in present tense, third person (e.g. "adds", "fixes", "removes"). +- Be specific about class/method/parameter names that changed. +- Include the short commit SHA `(commit abc1234)` at the end of each bullet so it is traceable. +- Do **not** include `chore`, `ci`, `test`, `docs`-only commits in the release notes unless they represent a notable change visible to SDK users. + +--- + +## Phase 5: Present and confirm + +Show the user a preview in this exact format: + +``` +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + Release Prep Summary +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + Branch: {CURRENT_BRANCH} + Baseline tag: {LATEST_TAG} + Commits: N commits analysed + Bump type: patch | minor | major + Version: {CURRENT_VERSION} → {PROPOSED_VERSION} + + Release Notes preview: + ────────────────────── + {RELEASE_NOTES} +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` + +Ask the user: + +> "Does this look correct? You can accept, or provide a different version / bump type. Reply `yes` to create the release issue, `no` to cancel, or type a version override (e.g. `0.58.0`)." + +If the user provides a version override, re-derive the bump type from the override vs `CURRENT_VERSION` and update `PROPOSED_VERSION`. If the override is not a valid SemVer / PEP 440 string, tell the user and ask again. + +If the user says `no`, stop here with: "Release prep cancelled. No issue was created." + +--- + +## Phase 6: Create the release issue + +Once the user confirms, run the following (do **not** modify `pyproject.toml` — the GitHub Action owns that step): + +### 6.1 Build the issue body + +Construct the issue body using the GitHub issue form field IDs so the structured data is machine-parseable by the release workflow: + +```markdown +### Version + +{PROPOSED_VERSION} + +### Bump Type + +{bump_type} + +### Branch + +{CURRENT_BRANCH} + +### Release Notes + +{RELEASE_NOTES} + +### Breaking Changes + +{BREAKING_CHANGES_SECTION or "_None_"} +``` + +> **Important:** The section headers (`### Version`, `### Branch`, etc.) must match the issue template field labels exactly — the release GitHub Action parses the body using these anchors. + +### 6.2 Create the issue + +```bash +gh issue create \ + --repo SAP/cloud-sdk-python \ + --title "Release v{PROPOSED_VERSION}" \ + --label "release" \ + --label "status: pending tests" \ + --body "{ISSUE_BODY}" +``` + +If `gh issue create` fails because the labels do not exist yet, tell the user: + +> "Labels are missing from the repo. Create them first by running: +> `gh label create 'release' --color '0052CC' --description '...' --repo SAP/cloud-sdk-python` +> (or apply `.github/labels.yml` via the sync-labels workflow), then re-run `/release-prep`." + +### 6.3 Summary + +After the issue is created, print: + +``` +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + Release issue created +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + Issue: #{NUMBER} — {ISSUE_URL} + Version: {PROPOSED_VERSION} + Branch: {CURRENT_BRANCH} + + Next steps (automated): + 1. Automation test repo picks up the issue (label: status: pending tests) + 2. Tests run against branch {CURRENT_BRANCH} + 3. On success → label updated to "status: tests passed" + 4. Release workflow triggers → bumps pyproject.toml, builds, publishes to PyPI, + creates GitHub Release, tags commit, closes this issue + + If tests fail: + → Issue is labelled "status: tests failed". Fix the branch and re-run /release-prep. +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` diff --git a/.github/ISSUE_TEMPLATE/release.yml b/.github/ISSUE_TEMPLATE/release.yml new file mode 100644 index 00000000..f4ff9110 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/release.yml @@ -0,0 +1,71 @@ +name: Release +description: Initiate a new SDK release. Created by the /release-prep skill — do not fill in manually unless preparing a hotfix. +labels: + - release + - "status: pending tests" +body: + - type: markdown + attributes: + value: | + > **Internal use only.** This issue drives the automated release pipeline. + > It is normally created by the `/release-prep` skill. Fill it in manually only for hotfix releases. + > Do not include SAP-internal or customer-specific information. + + - type: input + id: version + attributes: + label: Version + description: "New version to release (no leading `v`). Must be a valid PEP 440 / SemVer string." + placeholder: "0.57.0" + validations: + required: true + + - type: dropdown + id: bump_type + attributes: + label: Bump Type + description: "SemVer change type for this release." + multiple: false + options: + - patch + - minor + - major + validations: + required: true + + - type: input + id: branch + attributes: + label: Branch + description: "Branch to release from. Must already be pushed to origin." + placeholder: "feature/my-feature" + validations: + required: true + + - type: textarea + id: release_notes + attributes: + label: Release Notes + description: "Changelog entry for this release. Will be used as the GitHub Release body." + placeholder: | + ### What's New + - ... + + ### Improvements + - ... + + ### Bug Fixes + - ... + validations: + required: true + + - type: textarea + id: breaking_changes + attributes: + label: Breaking Changes + description: "List breaking changes and migration steps. Leave blank if there are none — the release workflow handles an empty value correctly." + placeholder: | + > ⚠️ Important: migration steps for users upgrading from the previous version. + - **[Breaking Change]**: ... + validations: + required: false diff --git a/.github/labels.yml b/.github/labels.yml new file mode 100644 index 00000000..dd8bcc04 --- /dev/null +++ b/.github/labels.yml @@ -0,0 +1,52 @@ +# Label definitions for SAP Cloud SDK for Python +# Apply with: gh label create --repo SAP/cloud-sdk-python (see docs/DEVELOPMENT.md) +# Or use the sync-labels workflow to reconcile all labels at once. + +# ── Release pipeline ──────────────────────────────────────────────────────── + +- name: release + color: "0052CC" + description: "Marks an issue as a release pipeline issue. Monitored by the automation test repo." + +# Release status state machine (exactly one status label is set at a time): +# pending tests → tests running → tests passed → releasing → released +# └──→ tests failed (terminal failure) +# └──→ release failed (terminal failure) + +- name: "skip-integration-tests" + color: "E4E669" + description: "Add to a release issue to skip the integration test step in the release workflow. Use only when tests are known-good or infra is unavailable." + + + color: "FBCA04" + description: "Release issue created; waiting for the automation test repo to pick it up." + +- name: "status: tests running" + color: "E4E669" + description: "Automation test repo has started tests for this release branch." + +- name: "status: tests passed" + color: "0E8A16" + description: "All automation tests passed. Triggers the release GitHub Action." + +- name: "status: tests failed" + color: "B60205" + description: "Automation tests failed. Release is blocked — investigate before retrying." + +- name: "status: releasing" + color: "1D76DB" + description: "Release workflow is running: bumping version, tagging, publishing to PyPI." + +- name: "status: released" + color: "6F42C1" + description: "Package published to PyPI and GitHub Release created. Issue will be closed automatically." + +- name: "status: release failed" + color: "B60205" + description: "Release workflow failed after tests passed. Check the workflow run and clean up the tag before retrying." + +# ── Existing labels (kept for reference — do not duplicate) ───────────────── +# bug, documentation, duplicate, enhancement, good first issue, help wanted, +# invalid, question, wontfix, dependencies, python, python:uv, +# sdk-review: ✅ passed, sdk-review: ❌ blocked, sdk-review: ⚠️ flagged, +# sdk-review: skipped, feature-request, github-actions, tok:pd1, tok:pd2, tok:pd9 diff --git a/.github/workflows/check-version-bump.yaml b/.github/workflows/check-version-bump.yaml index 8391f443..2cd0c3f7 100644 --- a/.github/workflows/check-version-bump.yaml +++ b/.github/workflows/check-version-bump.yaml @@ -18,30 +18,23 @@ jobs: with: fetch-depth: 0 - - name: Set up Python - uses: actions/setup-python@v6 - with: - python-version: "3.11" - - - name: Install version comparison dependency - run: python -m pip install packaging - - - name: Check that version was bumped if src/ was modified + - name: Reject manual version bumps in pyproject.toml run: | BASE_SHA="${{ github.event.pull_request.base.sha }}" HEAD_SHA="${{ github.event.pull_request.head.sha }}" MERGE_BASE=$(git merge-base "$BASE_SHA" "$HEAD_SHA") - CHANGED_FILES=$(git diff --name-only "$MERGE_BASE" "$HEAD_SHA") - - if echo "$CHANGED_FILES" | grep -qE "^src/.*\.(py|pyi|proto)$"; then - BASE_VERSION=$(git show "$MERGE_BASE:pyproject.toml" | grep "^version = " | cut -d'"' -f2) - HEAD_VERSION=$(git show "$HEAD_SHA:pyproject.toml" | grep "^version = " | cut -d'"' -f2) - - # Use PEP 440 ordering because sort -V ranks RCs above final releases - python .github/scripts/check_version_bump.py \ - "$BASE_VERSION" "$HEAD_VERSION" - - else - echo "No source file changes under src/. Version bump not required." + BASE_VERSION=$(git show "$MERGE_BASE:pyproject.toml" | grep "^version = " | cut -d'"' -f2) + HEAD_VERSION=$(git show "$HEAD_SHA:pyproject.toml" | grep "^version = " | cut -d'"' -f2) + + if [[ "$BASE_VERSION" != "$HEAD_VERSION" ]]; then + echo "ERROR: Manual version bump detected in pyproject.toml." + echo " Base: $BASE_VERSION" + echo " PR: $HEAD_VERSION" + echo "" + echo "The version is managed automatically by the release pipeline." + echo "Remove the version change from pyproject.toml and use /release-prep to trigger a release." + exit 1 fi + + echo "Version unchanged ($HEAD_VERSION) — OK." diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml index 1dd5aec7..f1960fe5 100644 --- a/.github/workflows/integration-tests.yml +++ b/.github/workflows/integration-tests.yml @@ -8,6 +8,7 @@ on: - 'src/**/*.py' - 'tests/**/*.py' - 'pyproject.toml' + - 'uv.lock' push: branches: [main] paths: @@ -15,12 +16,101 @@ on: - 'src/**/*.py' - 'tests/**/*.py' - 'pyproject.toml' + - 'uv.lock' jobs: + # ── Determine which integration test directories to run ────────────────────── + detect-scope: + name: Detect test scope + runs-on: ubuntu-latest + outputs: + run_all: ${{ steps.scope.outputs.run_all }} + test_paths: ${{ steps.scope.outputs.test_paths }} + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Compute scope + id: scope + env: + EVENT: ${{ github.event_name }} + run: | + # On push to main or workflow_dispatch → always run everything + if [[ "$EVENT" == "push" || "$EVENT" == "workflow_dispatch" ]]; then + echo "run_all=true" >> "$GITHUB_OUTPUT" + echo "test_paths=tests/*/integration/" >> "$GITHUB_OUTPUT" + echo "Scope: full run (push/workflow_dispatch)" + exit 0 + fi + + # On pull_request → diff against the base branch + BASE_SHA="${{ github.event.pull_request.base.sha }}" + HEAD_SHA="${{ github.event.pull_request.head.sha }}" + CHANGED=$(git diff --name-only "$BASE_SHA" "$HEAD_SHA") + + echo "Changed files:" + echo "$CHANGED" + + # Global triggers: any of these → run everything + # core/ is foundational — changes there can affect any module + GLOBAL_PATTERNS=( + "pyproject.toml" + "uv.lock" + ".github/workflows/integration-tests.yml" + "src/sap_cloud_sdk/core/" + "tests/core/" + ) + + for pattern in "${GLOBAL_PATTERNS[@]}"; do + if echo "$CHANGED" | grep -q "^${pattern}"; then + echo "run_all=true" >> "$GITHUB_OUTPUT" + echo "test_paths=tests/*/integration/" >> "$GITHUB_OUTPUT" + echo "Scope: full run (global file changed: $pattern)" + exit 0 + fi + done + + # Modules that have integration test directories + MODULES_WITH_INTEGRATION=( + adms + agent_memory + agentgateway + aicore + destination + dms + dpi_ng + objectstore + outputmanagement + ) + + PATHS=() + for module in "${MODULES_WITH_INTEGRATION[@]}"; do + if echo "$CHANGED" | grep -qE "^(src/sap_cloud_sdk|tests)/${module}/"; then + PATHS+=("tests/${module}/integration/") + echo "Module affected: $module" + fi + done + + if [[ ${#PATHS[@]} -eq 0 ]]; then + echo "run_all=false" >> "$GITHUB_OUTPUT" + echo "test_paths=" >> "$GITHUB_OUTPUT" + echo "Scope: no integration tests affected — skipping" + else + echo "run_all=false" >> "$GITHUB_OUTPUT" + echo "test_paths=${PATHS[*]}" >> "$GITHUB_OUTPUT" + echo "Scope: ${PATHS[*]}" + fi + + # ── Run the integration tests ───────────────────────────────────────────────── integration-tests: name: Integration Tests - # Skip integration tests for PRs from forks (they don't have access to secrets) - if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository + needs: detect-scope + # Skip for fork PRs (no secrets), and skip when no integration tests are affected + if: > + (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && + needs.detect-scope.outputs.test_paths != '' runs-on: ${{ contains(github.server_url, 'github.com') && 'ubuntu-latest' || fromJSON('["self-hosted"]') }} permissions: contents: read @@ -43,21 +133,27 @@ jobs: run: | echo "Setting up environment variables for integration tests..." - # Process GitHub secrets (all configuration stored as secrets) - echo '${{ toJSON(secrets) }}' | jq -r 'to_entries[] | select(.key | startswith("CLOUD_SDK_CFG_") or startswith("AICORE_")) | "\(.key)=\(.value)"' | while read line; do - echo "$line" >> $GITHUB_ENV - var_name=$(echo "$line" | cut -d= -f1) - echo "Set secret: $var_name" - done + echo '${{ toJSON(secrets) }}' \ + | jq -r 'to_entries[] + | select(.key | startswith("CLOUD_SDK_CFG_") or startswith("AICORE_")) + | "\(.key)=\(.value)"' \ + | while read line; do + echo "$line" >> $GITHUB_ENV + echo "Set secret: $(echo "$line" | cut -d= -f1)" + done - # Process GitHub variables (all configuration stored as variables) - echo '${{ toJSON(vars) }}' | jq -r 'to_entries[] | select(.key | startswith("CLOUD_SDK_CFG_") or startswith("AICORE_")) | "\(.key)=\(.value)"' | while read line; do - echo "$line" >> $GITHUB_ENV - var_name=$(echo "$line" | cut -d= -f1) - echo "Set variable: $var_name" - done + echo '${{ toJSON(vars) }}' \ + | jq -r 'to_entries[] + | select(.key | startswith("CLOUD_SDK_CFG_") or startswith("AICORE_")) + | "\(.key)=\(.value)"' \ + | while read line; do + echo "$line" >> $GITHUB_ENV + echo "Set variable: $(echo "$line" | cut -d= -f1)" + done - echo "Environment setup complete - automatically configured all CLOUD_SDK_CFG_* and AICORE_* environment variables and secrets" + echo "Environment setup complete" - name: Run integration tests - run: uv run pytest tests/*/integration/ -v --tb=short + env: + TEST_PATHS: ${{ needs.detect-scope.outputs.test_paths }} + run: uv run pytest $TEST_PATHS -v --tb=short diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bfc156cd..a47a5b0e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,21 +1,93 @@ -name: Publish Package to PyPI +name: Release on: - release: - types: [published] + issues: + types: [labeled] jobs: - publish: - permissions: - contents: read - id-token: write - name: Publish Package to PyPI + release: + name: Release + if: > + github.event.label.name == 'status: tests passed' && + contains(github.event.issue.labels.*.name, 'release') runs-on: ubuntu-latest environment: 'pypi:sap-cloud-sdk' + permissions: + contents: write + issues: write + id-token: write + steps: - - name: Checkout repository + # ── 1. Parse the release issue body ───────────────────────────────────── + - name: Parse release issue + id: issue + env: + BODY: ${{ github.event.issue.body }} + run: | + parse_field() { + local heading="$1" + # Extract the text between "### " and the next "###" or end-of-body + echo "$BODY" \ + | awk "/^### ${heading}/{found=1; next} found && /^###/{exit} found{print}" \ + | sed '/^[[:space:]]*$/d' \ + | head -1 \ + | xargs + } + + VERSION=$(parse_field "Version") + BRANCH=$(parse_field "Branch") + BUMP_TYPE=$(parse_field "Bump Type") + + if [[ -z "$VERSION" || -z "$BRANCH" ]]; then + echo "ERROR: Could not parse Version or Branch from the release issue body." + echo "Issue body:" + echo "$BODY" + exit 1 + fi + + echo "version=$VERSION" >> "$GITHUB_OUTPUT" + echo "branch=$BRANCH" >> "$GITHUB_OUTPUT" + echo "bump_type=$BUMP_TYPE" >> "$GITHUB_OUTPUT" + echo "tag=v${VERSION}" >> "$GITHUB_OUTPUT" + + echo "Parsed — version: $VERSION, branch: $BRANCH, bump_type: $BUMP_TYPE" + + # ── 2. Validate version format ─────────────────────────────────────────── + - name: Validate version string + run: | + pip install packaging -q + python3 -c " + from packaging.version import Version, InvalidVersion + try: + v = Version('${{ steps.issue.outputs.version }}') + print(f'Version is valid PEP 440: {v}') + except InvalidVersion as e: + print(f'ERROR: {e}') + raise SystemExit(1) + " + + # ── 3. Mark issue as in-progress ──────────────────────────────────────── + - name: Update issue — release in progress + env: + GH_TOKEN: ${{ github.token }} + run: | + gh issue edit ${{ github.event.issue.number }} \ + --repo "${{ github.repository }}" \ + --remove-label "status: tests passed" \ + --add-label "status: releasing" || true + gh issue comment ${{ github.event.issue.number }} \ + --repo "${{ github.repository }}" \ + --body "🚀 Release workflow started for **v${{ steps.issue.outputs.version }}** from branch \`${{ steps.issue.outputs.branch }}\`." + + # ── 4. Checkout the release branch ────────────────────────────────────── + - name: Checkout release branch uses: actions/checkout@v4 + with: + ref: ${{ steps.issue.outputs.branch }} + fetch-depth: 0 + token: ${{ github.token }} + # ── 5. Set up Python and uv (shared by integration tests and build) ────── - name: Set up Python uses: actions/setup-python@v6 with: @@ -26,43 +98,181 @@ jobs: with: enable-cache: true - - name: Read package name and version from pyproject.toml - id: metadata + # ── 6. Run integration tests ───────────────────────────────────────────── + - name: Install dependencies + if: ${{ !contains(github.event.issue.labels.*.name, 'skip-integration-tests') }} + run: uv sync --dev + + - name: Set up integration test environment + if: ${{ !contains(github.event.issue.labels.*.name, 'skip-integration-tests') }} + run: | + echo '${{ toJSON(secrets) }}' \ + | jq -r 'to_entries[] + | select(.key | startswith("CLOUD_SDK_CFG_") or startswith("AICORE_")) + | "\(.key)=\(.value)"' \ + | while read line; do + echo "$line" >> $GITHUB_ENV + echo "Set secret: $(echo "$line" | cut -d= -f1)" + done + echo '${{ toJSON(vars) }}' \ + | jq -r 'to_entries[] + | select(.key | startswith("CLOUD_SDK_CFG_") or startswith("AICORE_")) + | "\(.key)=\(.value)"' \ + | while read line; do + echo "$line" >> $GITHUB_ENV + echo "Set variable: $(echo "$line" | cut -d= -f1)" + done + + - name: Run integration tests + if: ${{ !contains(github.event.issue.labels.*.name, 'skip-integration-tests') }} + run: uv run pytest tests/*/integration/ -v --tb=short + + # ── 7. Validate the version bump is an increase ───────────────────────── + - name: Validate version bump run: | - python -m pip install toml packaging - PKG_NAME=$(python -c "import toml; print(toml.load('pyproject.toml')['project']['name'])") - VERSION=$(python -c "import toml; print(toml.load('pyproject.toml')['project']['version'])") - echo "Publishing $PKG_NAME version $VERSION to PyPI" - echo "name=$PKG_NAME" >> $GITHUB_OUTPUT - echo "version=$VERSION" >> $GITHUB_OUTPUT - - - name: Validate pre-release status + CURRENT=$(grep '^version = ' pyproject.toml | cut -d'"' -f2) + python .github/scripts/check_version_bump.py "$CURRENT" "${{ steps.issue.outputs.version }}" + + # ── 8. Bump version in pyproject.toml and commit ──────────────────────── + - name: Bump version in pyproject.toml + run: | + CURRENT=$(grep '^version = ' pyproject.toml | cut -d'"' -f2) + sed -i "s/^version = \"${CURRENT}\"/version = \"${{ steps.issue.outputs.version }}\"/" pyproject.toml + grep '^version = ' pyproject.toml + echo "Updated pyproject.toml → ${{ steps.issue.outputs.version }}" + + - name: Commit version bump + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git add pyproject.toml + git commit -m "chore(release): bump version to ${{ steps.issue.outputs.version }}" + git push origin ${{ steps.issue.outputs.branch }} + + # ── 9. Create and push the tag ─────────────────────────────────────────── + - name: Create and push tag + run: | + git tag "${{ steps.issue.outputs.tag }}" -m "Release ${{ steps.issue.outputs.tag }}" + git push origin "${{ steps.issue.outputs.tag }}" + + # ── 10. Build the distribution ─────────────────────────────────────────── + - name: Build release distribution + run: uv build + + - name: Verify build artifacts + run: | + wheel_count=$(ls -1 dist/*.whl 2>/dev/null | wc -l) + tar_count=$(ls -1 dist/*.tar.gz 2>/dev/null | wc -l) + [[ "$wheel_count" -ge 1 && "$tar_count" -ge 1 ]] || \ + { echo "ERROR: Missing build artifacts (wheels: $wheel_count, sdists: $tar_count)"; exit 1; } + echo "Build artifacts: $wheel_count wheel(s), $tar_count sdist(s)" + ls -lh dist/ + + # ── 11. Extract release notes from the issue body ─────────────────────── + - name: Extract release notes + id: notes env: - PACKAGE_VERSION: ${{ steps.metadata.outputs.version }} - RELEASE_IS_PRERELEASE: ${{ github.event.release.prerelease }} + BODY: ${{ github.event.issue.body }} run: | - python .github/scripts/validate_prerelease.py \ - "$PACKAGE_VERSION" "$RELEASE_IS_PRERELEASE" + # Extract Release Notes section into temp file + echo "$BODY" \ + | awk '/^### Release Notes/{found=1; next} found && /^### (Version|Bump Type|Branch|Breaking Changes)/{exit} found{print}' \ + | sed '/^[[:space:]]*$/d' \ + > /tmp/release_notes.md - - name: Check if version already exists on PyPI + # Append Breaking Changes section if present and non-empty + BREAKING=$(echo "$BODY" \ + | awk '/^### Breaking Changes/{found=1; next} found && /^###/{exit} found{print}' \ + | sed '/^[[:space:]]*$/d') + + if [[ -n "$BREAKING" && "$BREAKING" != *"_None_"* ]]; then + printf '\n%s\n' "$BREAKING" >> /tmp/release_notes.md + fi + + echo "notes_file=/tmp/release_notes.md" >> "$GITHUB_OUTPUT" + + # ── 12. Create the GitHub Release ─────────────────────────────────────── + - name: Create GitHub Release + id: gh_release + env: + GH_TOKEN: ${{ github.token }} run: | - PKG_NAME="${{ steps.metadata.outputs.name }}" - VERSION="${{ steps.metadata.outputs.version }}" + IS_PRERELEASE=$(python3 -c " + from packaging.version import Version + v = Version('${{ steps.issue.outputs.version }}') + print('true' if v.is_prerelease else 'false') + ") - echo "Checking if $PKG_NAME version $VERSION exists on PyPI..." + RELEASE_URL=$(gh release create "${{ steps.issue.outputs.tag }}" \ + --repo "${{ github.repository }}" \ + --title "v${{ steps.issue.outputs.version }}" \ + --notes-file "${{ steps.notes.outputs.notes_file }}" \ + $([ "$IS_PRERELEASE" = "true" ] && echo "--prerelease") \ + dist/*.whl dist/*.tar.gz) - HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "https://pypi.org/pypi/$PKG_NAME/$VERSION/json") + echo "release_url=$RELEASE_URL" >> "$GITHUB_OUTPUT" + echo "GitHub Release created: $RELEASE_URL" - if [ "$HTTP_STATUS" = "200" ]; then - echo "ERROR: Version $VERSION of $PKG_NAME already exists on PyPI!" - echo "Please increment the version in pyproject.toml before publishing." + # ── 13. Guard: confirm version is not already on PyPI ─────────────────── + - name: Check version does not already exist on PyPI + run: | + PKG_NAME=$(python3 -c "import toml; print(toml.load('pyproject.toml')['project']['name'])") + VERSION="${{ steps.issue.outputs.version }}" + HTTP=$(curl -s -o /dev/null -w "%{http_code}" "https://pypi.org/pypi/$PKG_NAME/$VERSION/json") + if [[ "$HTTP" == "200" ]]; then + echo "ERROR: $PKG_NAME $VERSION already exists on PyPI." exit 1 - else - echo "Version $VERSION is available for upload to PyPI" fi + echo "$PKG_NAME $VERSION is not yet on PyPI — proceeding." - - name: Build release distribution - run: uv build - + # ── 14. Publish to PyPI (OIDC trusted publishing) ─────────────────────── - name: Publish to PyPI uses: pypa/gh-action-pypi-publish@v1.14.2 + + # ── 15. Close the release issue ───────────────────────────────────────── + - name: Close release issue + env: + GH_TOKEN: ${{ github.token }} + VERSION: ${{ steps.issue.outputs.version }} + TAG: ${{ steps.issue.outputs.tag }} + RELEASE_URL: ${{ steps.gh_release.outputs.release_url }} + run: | + PKG_NAME=$(python3 -c "import toml; print(toml.load('pyproject.toml')['project']['name'])") + PYPI_URL="https://pypi.org/project/${PKG_NAME}/${VERSION}/" + + gh issue edit "${{ github.event.issue.number }}" \ + --repo "${{ github.repository }}" \ + --remove-label "status: releasing" \ + --add-label "status: released" || true + + gh issue comment "${{ github.event.issue.number }}" \ + --repo "${{ github.repository }}" \ + --body "$(cat < The version in `pyproject.toml` is **not** touched at this point — the release workflow owns that step. -To promote the final release candidate, open a pull request that changes the version from `X.Y.ZrcN` to `X.Y.Z` and updates `uv.lock`. The stable release should otherwise contain the same code as the final release candidate. +--- -## Create and Publish GitHub Release +## Step 3 — Wait for automated tests + +Once the issue is created, the automation test repo detects it (via the `status: pending tests` label) and runs the integration test suite against the branch. You can track progress on the release issue — the label updates automatically: + +| Label | Meaning | +|---|---| +| `status: pending tests` | Waiting for the automation repo to pick up the issue | +| `status: tests running` | Tests in progress | +| `status: tests passed` | Tests passed — release workflow will trigger shortly | +| `status: tests failed` | Tests failed — see [Handling failures](#handling-failures) | + +--- + +## Step 4 — Automated release (no action required) + +When the label changes to `status: tests passed`, the `Release` workflow triggers automatically and: + +1. Runs the full integration test suite against the branch (skippable — see below) +2. Bumps `version` in `pyproject.toml`, commits `chore(release): bump version to X.Y.Z`, and pushes to the branch +3. Creates and pushes the annotated git tag `vX.Y.Z` +4. Builds the distribution with `uv build` +5. Creates the GitHub Release with the release notes and build artifacts attached +6. Publishes to PyPI via OIDC trusted publishing +7. Posts a comment on the issue with links to PyPI and the GitHub Release, then closes it + +Monitor progress in the **Actions** tab. On success the package is available at: + +``` +https://pypi.org/project/sap-cloud-sdk/X.Y.Z/ +``` -5. Create GitHub release (this will automatically publish to PyPI) +--- - - Go to the repository's **Releases** page - - Click **"Draft a new release"** - - Create or select the tag that exactly matches the project version with a leading `v` - - Target the merged release commit on `main` - - Fill in the release title: `vX.Y.Z - Month D, YYYY` - - For an RC, select **Set as a pre-release** and do not set it as latest - - Add release notes: - - Highlight key features and changes - - Include breaking changes (if any) - - Reference relevant issues/PRs - - Click **"Publish release"** +## Step 5 — Merge the branch -6. Automated PyPI publication +Once the release workflow completes, the branch has the `chore(release): bump version to X.Y.Z` commit on it. Merge (or complete the PR merge) into `main` so the version bump lands on the main branch. - - The [Publish Package to PyPI](../.github/workflows/release.yml) workflow will automatically trigger - - The workflow will: - - Extract version from `pyproject.toml` - - Check if version already exists on PyPI (prevents duplicates) - - Build the package with `uv build` - - Publish to PyPI using trusted publishing (OIDC) - - Monitor the workflow in the **Actions** tab to confirm successful publication - - Package will be available at: `https://pypi.org/project/sap-cloud-sdk/X.Y.Z/` +--- -> **Note:** The version in `pyproject.toml` must match the release tag (without the 'v' prefix). For example, tag `vX.Y.Z` requires `version = "X.Y.Z"` in `pyproject.toml`. +## Release candidates -## Install and Verify +To publish a release candidate, follow the same process with a version like `0.57.0rc1`. The skill will propose a pre-release version if all unreleased commits are on a feature-frozen RC branch. -Install a specific release candidate explicitly: +The GitHub Release is automatically marked as pre-release when the version is a PEP 440 pre-release. Install explicitly: ```bash -pip install sap-cloud-sdk==1.0.0rc1 +pip install sap-cloud-sdk==0.57.0rc1 ``` -Install the current stable release normally: +--- + +## Handling failures + +### Tests failed (`status: tests failed`) + +Investigate the failures in the automation test repo. Fix the branch, then either: +- Re-run `/release-prep` to create a new release issue, or +- Manually remove `status: tests failed` and add `status: pending tests` to re-trigger the automation test run on the same issue + +### Release workflow failed (`status: release failed`) + +Check the failed workflow run linked in the issue comment. Common causes: + +| Symptom | Fix | +|---|---| +| Version already on PyPI | The version was already published — bump to the next patch and re-run `/release-prep` | +| Tag already exists | Delete the tag (`git push origin :refs/tags/vX.Y.Z`) and re-trigger | +| Build failed | Fix the source, push to the branch, then force-release (see below) | + +### Force a release (bypass automation tests) + +If you need to publish without waiting for the automation test repo (e.g. for a critical hotfix already validated manually): + +1. Open the release issue +2. Remove any `status: *` label currently on it +3. Add the label `status: tests passed` + +The release workflow triggers immediately. + +### Skip integration tests in the release workflow + +If the release workflow's own integration test step needs to be bypassed (e.g. test infrastructure is temporarily unavailable): + +1. Add the label `skip-integration-tests` to the release issue +2. Then add `status: tests passed` (or re-trigger if it is already set) + +The integration test steps are skipped; all other steps — version bump, tag, build, GitHub Release, PyPI publish — run as normal. Remove `skip-integration-tests` after the release to keep the label state clean. + +--- + +## Install ```bash +# Latest stable pip install sap-cloud-sdk + +# Specific version +pip install sap-cloud-sdk==0.57.0 + +# Release candidate +pip install sap-cloud-sdk==0.57.0rc1 ```