The release workflow's "Build and Test" step runs no tests, so the gate guards compilation only

[rahlk]

Is your feature request related to a problem? Please describe.

.github/workflows/release.yml has a step named "Build and Test" that never runs the tests:

      - name: Build and Test
        id: build
        run: ./gradlew clean fatJar

fatJar is a plain Jar task (build.gradle:203, task fatJar(type: Jar)). Nothing in the file makes it depend on test or check — the only dependsOn edges near it are compileJava.dependsOn spotlessApply (:198) and a dependsOn nativeCompile inside a different task (:257). So the release pipeline compiles, assembles the jar, and publishes, without ever executing the suite.

The tag-deletion gate below it is correct in form and guards the wrong thing:

      - name: Delete tag on failure
        if: steps.build.outcome != 'success'

steps.build is the assembly step, so a tag is deleted on a compile failure and survives a test failure. The outcome-vs-conclusion lesson was applied here (this repo uses outcome, correctly — continue-on-error rewrites conclusion to success), but the step it reads is the build rather than a test run.

Both sibling analyzers gate on a real test step:

repo	release step	gate
codeanalyzer-python	id: test → uv run pytest	steps.test.outcome == 'failure'
codeanalyzer-typescript	id: test → tests	steps.test.outcome == 'failure'
codeanalyzer-java	id: build → gradlew clean fatJar	build only; no tests run

Describe the solution you'd like

• Run the test suite in the release pipeline as its own step with its own id.
• Point the tag-deletion gate at that step's outcome, not at the build's.
• Decide what happens to the tests that need Docker: provide it on the runner, or exclude them from the release gate explicitly and record which coverage the gate therefore does not have. An excluded test that nobody knows is excluded is the same defect one level down.
• Rename the step so its name matches what it does, whichever way the fix goes.

Describe alternatives you've considered

Not stated in the original issue.

Additional context

Why it matters now

The consequence is not hypothetical for this project. In python-sdk, a release workflow whose gate did not reflect the test result shipped v1.5.0 with a red Java suite — run 30300820857 reported 11 failed, 253 passed and concluded success. That was the conclusion/outcome variant of this same defect: a gate that reads something other than "did the tests pass".

Here the effect is broader, because there is no push CI on branches either. A release cut from this workflow has no automated test evidence at any point in its life — the only evidence is whatever the maintainer ran locally, on a machine whose environment differs from the runner's. A concrete example from today: a local run of 3.1.2 was 562 of 563, the single failure being Testcontainers finding no Docker. That is exactly the class of failure a runner would surface and a laptop will not, and in the other direction a runner-only failure would today reach PyPI unnoticed.

Scope boundary

.github/workflows/release.yml, and build.gradle only if the chosen fix is a task dependency rather than a workflow step. Not the test suite itself, and not the Testcontainers/Docker question — whether the runner should provide Docker for the container-backed tests is a separate decision this issue does not make, though it has to be answered to know which tests the gate can require.

Caveats and known risks

• Adding a real test step lengthens every release and will surface pre-existing flakiness that has never blocked a publish before. That is the gate working, but it will feel like a regression the first time it fires.
• If the Docker-dependent tests are excluded, the gate becomes "the tests that can run in CI passed", which is weaker than it reads. Say so in the workflow, next to the exclusion.
• A gate that deletes tags on failure has a sharp edge already proven in python-sdk: the delete step runs git push --delete origin <tag>, so a flaky test destroys a tag someone may have already seen. Worth confirming that is still the behaviour you want once tests can actually fail it.

Definition of done

• The release workflow runs the suite and refuses to publish when it fails, verified by a deliberately-failing run rather than by reading the YAML.
• No step's name overstates what it does.
• If any test is excluded from the gate, the exclusion and its reason are written in the workflow.