Skip to content

Add regression tests for known defects, tagged known-defect - #265

Open
sophiecarreras wants to merge 2 commits into
masterfrom
tests/code-review-findings
Open

sophiecarreras wants to merge 2 commits into
masterfrom
tests/code-review-findings

Conversation

@sophiecarreras

@sophiecarreras sophiecarreras commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Tests only. No production code changes.

Adds regression tests for 56 known findings, written against the current master. Each test name carries its finding id (for example testFindingC21_...).

  • Tests for defects that are still present are tagged @Category(KnownDefect.class). They are excluded from the default test task, so ./gradlew build and CI stay green.
  • ./gradlew knownDefectTests runs only those tests. It is expected to fail until the defects are fixed. When a fix lands, remove the tag so the test joins the normal suite.
  • Tests that pass today (already-fixed defects, guards, already-correct cases) run in the normal suite and protect the current behaviour.
  • Nothing is skipped with @Ignore; the exclusion is the visible category plus the task above.

How to run

Default suite (known-defect tests excluded; 730 tests, all pass):

./gradlew build --offline

Known-defect tests only (57 tests; the task fails by design, every test fails today):

./gradlew knownDefectTests --offline --continue
./gradlew :b2-sdk-core:knownDefectTests --tests '*.B2UtilFindingsTest'   # one class

A class whose tests are all tagged reports "No tests found" if run with the plain test task; use knownDefectTests for it.

What each status means

  • Still defective (40 findings, plus 7 repository-health checks): tagged KnownDefect, run only in knownDefectTests, each fails for the reason the finding describes.
  • Fixed on master (2 findings): the test passes and runs in the default suite.
  • Guards (5 findings): the behaviour is judged correct or an existing test is strengthened; default suite.
  • Already correct (1 finding): default suite.
  • No natural test (1 finding, a claim about missing tests): no test added.

Total 56 findings. The repository-health checks (changelog, README, security policy, one workflow command, one sample option) read repository files and are skipped, not failed, when run outside a full checkout.

Notes: a few tests read the sample source text because the CLI cannot run without a B2 account; these are source-pattern checks, not behaviour tests. One default-suite guard (C44) reflects JDK 11/17 behaviour and creates a sparse 2 GiB file that is deleted afterwards. The C18 guard waits (bounded) for the cleaner thread to park before timing close(), so it is not timing-sensitive at startup, but it can still pass while the race it mentions exists.

Existing tests are unchanged. Build change: buildSrc/src/main/kotlin/b2sdk.gradle.kts (category exclusion and the knownDefectTests task), plus a small marker interface com.backblaze.b2.testing.KnownDefect in the test source of core, httpclient and samples.

The samples module had no tests before. No test needs credentials or network access beyond loopback.

🤖 Generated with Claude Code

Tests only; no production code changes.

Adds 21 test classes covering 56 findings. Tests for defects that are
still present are tagged @category(KnownDefect.class). The default `test`
task (and so `build` and CI) excludes that tag, so the build stays green.
`./gradlew knownDefectTests` runs only the tagged tests and is expected
to fail until each defect is fixed; remove the tag when a fix lands so
the test joins the normal suite.

Tests that pass today (fixed defects, guards, already-correct cases) run
in the default suite and protect the current behaviour. Repository-health
checks skip instead of failing when the tests are not run from a full
checkout.

Build change: buildSrc/src/main/kotlin/b2sdk.gradle.kts excludes the
category from `test` and registers `knownDefectTests` for modules that
contain the marker interface com.backblaze.b2.testing.KnownDefect.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
testGuardC18_closeReturnsPromptly failed intermittently (about 2 of 12 isolated runs): close() right after build() can land before the cleaner thread reaches its first wait. The guard now waits (bounded) for the thread to be parked first, so it only measures close() latency.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants