Skip to content

Retry diverged branches; encode NaN in FCMPG/FCMPL - #74

Closed
nils-loose wants to merge 2 commits into
svcomp27/integrationfrom
feat/diverged-branch-retry
Closed

nils-loose wants to merge 2 commits into
svcomp27/integrationfrom
feat/diverged-branch-retry

Conversation

@nils-loose

Copy link
Copy Markdown
Contributor

What changes

Explorer: retry diverged branches. When the run of a solution does not take the side of the branch it was solved for, the branch used to stay "solved", so its unexplored side was never tried again, and since #65 (18b0af7) SAFE is ruled out. Now the branch becomes a candidate again, up to --divergence-retries times (default 3):

  • a retry excludes every solution that failed so far (a blocking clause over the scalar inputs of each failed model);
  • it tries small values first: integral inputs bounded by 16, 256 and 65536 in turn, then unbounded;
  • UNSAT under these extra constraints proves nothing, so it is never recorded as an infeasible branch.

Only when all retries diverge, or no other solution is left, is the branch given up on. That is recorded as unexpected_divergence, or as an expected divergence for an approximate model, as before. A given-up branch rules out SAFE, unless another run reached its unexplored side by the end of the search. In that case the side is in the tree, so the search covered it like any other.

Executor: FCMPG/FCMPL encode NaN and signed zeros. The formula had no NaN case, so a NaN operand compared as "less", while the JVM pushes 1 (FCMPG) or -1 (FCMPL). Z3 could then pick NaN to satisfy 0.0f <= x, and the run took the other side. The concrete value used Float.compareTo, which orders -0.0f below 0.0f. The fix mirrors DoubleValue.DCMP. A concrete NaN operand also no longer drops the formula.

Why the runs diverged

  • 17 algorithms runs (InsertionSort, MergeSortIterative, BellmanFord-*02, Tsp): Z3 picks a size such as 2^31-1 or 524288, and the allocation throws OutOfMemoryError before the run reaches the branch. Retries with small sizes reach the branch. These tasks have an unbounded nondet size, though, so a sound SAFE is not reachable for the SAFE runs. They now end in timeout, or in a stall from the known RecursionError in Tree.add_recursive, instead of UNKNOWN. Both score 0. Four valid-assert VIOLATIONs are now found.
  • Piecewise_true: the FCMP NaN encoding described above.

Soundness

  • SAFE still requires every branch to be covered or proven UNSAT. A query with retry constraints never marks a branch infeasible.
  • A given-up branch blocks SAFE unless its side was reached by some run.
  • The existing downgrades are unchanged: context loss, precision loss (approximate traces), sa_desync and uncaught exceptions.

Measurements

Configuration: --sv-comp, 300 s CPU, mem15g, no SA. Compared against baseline-results.jsonl of the integration branch.

  • Affected set, 209 runs: every run with a divergence in the Static pre-analysis pruning, with soundness fixes #65 re-run, plus the KB causes diverged-branch-not-retried, missing-violation-after-exhaustion, search-strategy-misses-violation, branch-divergence, solver-model-unbounded-values and fcmp-nan-unordered-not-encoded. Result: +19 / -0.
  • Regression set, 200 runs: baseline runs with any divergence, baseline-correct float and argv runs, unsolved expected-false float and argv runs, and 70 random baseline-correct runs. Result: +1 / -4. The flips were re-run once on both sides and reproduced.
  • Total: +20 / -4, net +16, no new wrong verdicts.
task prop base new pts
argv-tasks/Piecewise_true both unknown safe +4
argv-tasks/FractalTouchHandler_true both unknown safe +4
argv-tasks/ReverseInterpolator_false NRE timeout safe +2
argv-tasks/ReverseInterpolator_false2 NRE / valid timeout safe / violation +3
algorithms/InsertionSort-{Fun,Mem}Unsat01 valid unknown violation +2
algorithms/MergeSortIterative-{Fun,Mem}Unsat01 valid unknown violation +2
float-nonlinear-calculation/coral17, coral43 valid unknown violation +2
argv-tasks/DiskBasedCache valid timeout violation +1
argv-tasks/AbstractChart_exceptionless NRE safe timeout -2
argv-tasks/TouchPadView valid safe timeout -2

Credit for the gains:

  • FCMP fix: Piecewise, FractalTouchHandler, ReverseInterpolator, DiskBasedCache.
  • Retries: algorithms, coral17, coral43.

Tests

  • tests/test_divergence.py (rewritten, 21 tests): the retry and give-up flow, guesses, a side reached later, bound order, LIMIT and UNKNOWN handling, retry_constraints on a real Z3 instance (exclusion, bounds, strings and doubles), and UNSAT under extra constraints not being recorded.
  • tests/test_solver_rlimit.py and test_sa_walk.py pass.
  • FloatValueTest gains symbolic NaN and signed-zero cases for FCMPG/FCMPL. They fail without the fix.

Known limits

  • Losses from the FCMP fix: AbstractChart_exceptionless and TouchPadView lose a SAFE (-4). No retries happen there. The exact NaN case makes NaN paths reachable and makes the FP queries much harder (0.3 s becomes about 70 s per query).
  • Retries cost rounds: runs that used to stop early at UNKNOWN now spend their time on small-size retries. That is score-neutral, but they end as timeout or stall.
  • What the blocking clause covers: it is over scalar inputs only. Arrays are not used to tell models apart, which can only over-restrict a retry.

🤖 Generated with Claude Code

nils-loose and others added 2 commits October 18, 2026 20:36
FCMP built its formula from greaterThan and FP equality only, so a NaN
operand compared as 'less' in the formula, while the JVM pushes 1 (FCMPG)
or -1 (FCMPL). The solver could then pick NaN to satisfy an ordered
comparison such as 0.0f <= x, and the run took the other side. The
concrete value used Float.compareTo, which orders -0.0f below 0.0f and NaN
above everything. Mirror DoubleValue.DCMP: an explicit NaN case in the
formula and IEEE comparisons for the concrete value. A concrete NaN
operand no longer drops the formula.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A run that does not take the side its solution was solved for used to
leave the branch solved, so its unexplored side was never tried again
and SAFE was ruled out (unexpected_divergence). Most such runs end in an
OutOfMemoryError before the branch: Z3 picks a size such as 2^31-1 and
the allocation fails, though small sizes reach the branch.

Now the branch becomes a candidate again, up to --divergence-retries
times (default 3). A retry excludes every solution that failed so far
and looks for small values first: integral inputs bounded by 16, 256 and
65536 in turn, then unbounded. UNSAT under these extra constraints
proves nothing, so it is never recorded as an infeasible branch. Only
when all retries diverge, or no other solution is left, is the branch
given up on and recorded as unexpected_divergence (or as an expected
divergence for an approximate model, as before). A given-up branch
rules out SAFE unless another run reached its unexplored side by the
end of the search.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
SinCosUF.java: deleted as in #80 (its trace-global sample constraint gave
a false SAFE); #71's eager precision loss there goes with it.
StrategyService.py: solve_branch takes both #74's extra_exprs and #80's
approximate_rlimit, on #80's sliced query.
SVCompDriver.py: retries (#74) and first queries both get the
approximate-query cap (#80); a solution keeps #74's record and #80's
query-based guess flag.
SymbolicExplorer.py: both sets of options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
FloatValue.java: one FCMP, #81's bit-vector ite version; it has the same
NaN, signed-zero and ordering cases as #74's (FloatValueTest and
BitvectorEncodingTest pass).
SolverHandler.py: #81's int_to_bv lifting of the query, plus #74's
extra_exprs.
DoubleInvocation/LongValue/ShortValue.java: imports of both sides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
_extract_variables: one copy (#82 cherry-picks #72's patch).
StrategyService.py, Database.py: #82 walks the Database's tree under the
lock; visit_tree passes the solved and #74's retired branches, in #72's
'with lock' style.
SVCompDriver.py: #82's shallow snapshot of the target node for
check_divergence, with #74's solution record and #80's guess flag; the
ingest-failure (#72) and stack-overflow (#78) checks read the Database
without copying the tree.
DTOBuilder.java: #82's FormulaDumper with #71's SymbolRegistry check.
DFS.py: #82 logs the per-node line only with an SA graph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
Combining #74 and #82: check_divergence gets a shallow copy of the
target, so #74's check whether another run reached the given-up side
looked at a node that never changes, and every given-up branch blocked
SAFE. The copy keeps a reference to the tree's node, which give_up_branch
records. The driver test fakes gain #82's Database reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
DoubleValue/FloatValue.java: asStringValue is already #76's (concrete
String.valueOf, context loss when symbolic), the same as #86's.
FloatValueTest.groovy: keep #74's FCMP cases and #86's asStringValue cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
SVCompDriver.py:
- one C1-only option (#82 and #88 both added it), in #88's
  EXECUTOR_JVM_OPTIONS, with #88's budget-derived heap;
- determine_next_step drops a stack-overflow run's violations (#78)
  before #88's out-of-memory and resource-limit handling, and reads the
  Database without copying the tree (#82);
- a run stopped by the budget is not checked for divergence; #72's
  no-trace check applies to the others;
- every query, first or retry (#74), runs under the budget's interrupt;
- #88's cut-short check joins #72's incomplete-search check, and both
  apply to every SAFE;
- the stats keep the pre-check (#73), divergence retries (#74) and
  resources (#88).
DFS.py: the per-node line is at #88's TRACE level, with the CHOICE label.
TestcaseStats.py, test_solver_rlimit.py: both sides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
Combining #72, #74, #78, #82 and #88: the driver tests create #88's
budget and verdict lock, and #88's tests get the stack-overflow (#78) and
copy-free (#82) Database reads and the choice flag (#76). A run reports
one verdict (#88), so the incomplete-search subtests reset the flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nils-loose nils-loose mentioned this pull request Oct 20, 2026
@nils-loose

Copy link
Copy Markdown
Contributor Author

Included in dev through #92 (merge commit 534c182), together with the fixes for interactions between PRs. This PR stays as the record of the change and its measured flips.

@nils-loose nils-loose closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant