Skip to content

PHOENIX-7996 Enable server-side maintenance of immutable indexes by default - #2618

Open
lokiore wants to merge 1 commit into
apache:PHOENIX-7562-feature-newfrom
lokiore:PHOENIX-7996-enable-immutable-default-ccf
Open

lokiore wants to merge 1 commit into
apache:PHOENIX-7562-feature-newfrom
lokiore:PHOENIX-7996-enable-immutable-default-ccf

Conversation

@lokiore

@lokiore lokiore commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This targets the Consistent Failover feature branch (PHOENIX-7562-feature-new). It brings the server-side immutable-index correctness hardening from #2611 to this branch and enables it by default (DEFAULT_SERVER_SIDE_IMMUTABLE_INDEXES_ENABLED flips falsetrue).

With the flag on, immutable, global, non-transactional secondary indexes are maintained server-side by IndexRegionObserver (PHOENIX-7426) rather than by the client: the client ships the serialized IndexMaintainer and the region server builds the index updates exactly once. Immutable data tables that declare a ROW_TIMESTAMP column continue to be maintained client-side regardless of the flag.

This is the companion of #2611. #2611 lands the same hardening on master off by default (a pure correctness fix); this PR is where the default is turned on, on the branch that adopts server-side immutable-index maintenance.

Hardening (identical to #2611):

  • The maintenance-side decision is centralized in IndexUtil.isServerSideImmutableIndexMaintenanceEnabled(...) and every data-table gate routes through it — IndexUtil.getClientMaintainedIndexes, IndexMaintainer.maintainedLocalOrGlobalIndexesWithoutMatchingStorageScheme (the INDEX_UUID gate), MutationState.filterIndexCheckerMutations, DeleteCompiler.isMaintainedOnClient (signature extended to take the data table so ROW_TIMESTAMP resolves against the data table), IndexMetaDataCacheClient.setMetaDataOnMutations, and UpsertCompiler — so client and server never disagree on which side maintains a table.
  • Partial-upsert read-back. IndexRegionObserver skips the current-row read-back for immutable batches; a partial upsert omitting an indexed/covered/index-WHERE column would then build the index entry from the partial mutation alone (dropping a covered column, or writing a spurious null-keyed uncovered entry). The read-back gate now forces a read-back for immutable batches carrying a covered or uncovered global index when an enabled mutation omits one of that index's on-disk columns; full-row upserts and single-cell tables keep the no-read-back fast path.
  • Broadened serialize filter. The immutable server-serialize filter now matches IndexUtil.isGlobalIndex (covering GLOBAL and UNCOVERED_GLOBAL), so an uncovered global immutable index with a matching storage scheme is maintained rather than dropped by both client and server.

Why are the changes needed?

Enabling server-side maintenance by default on this branch removes per-batch client index-mutation generation for immutable tables and lets IndexRegionObserver (the default index path) build the updates, reducing client-side work and mutation payload. The ROW_TIMESTAMP carve-out is required for correctness: server-side maintenance re-stamps every data cell — including the ROW_TIMESTAMP column — with the server batch timestamp, so ROW_TIMESTAMP range predicates (which push an HBase scan TimeRange) would silently drop rows on range reads. This mirrors CANNOT_CREATE_INDEX_ON_MUTABLE_TABLE_WITH_ROWTIMESTAMP for the mutable variant.

Does this PR introduce any user-facing change?

Yes, on this feature branch: immutable, global, non-transactional secondary indexes are maintained server-side by default (previously client-side unless the flag was set). Immutable tables with a ROW_TIMESTAMP column are unaffected (client-maintained). Upgrade region servers before clients; server-side maintenance rides the default-enabled IndexRegionObserver path. The previous behavior can be restored with phoenix.server.side.immutable.indexes.enabled=false.

How was this patch tested?

Same coverage as #2611. Partial-upsert/delete coverage in BaseImmutableIndexIT runs under both ServerSideImmutableIndexIT (flag on) and ClientSideImmutableIndexIT (flag off), parameterized over storage scheme. GlobalIndexCheckerIT#testPartialRowUpdateForImmutable{,Uncovered} lock the read-back fix; the uncovered COUNT invariant is asserted only when server-side maintenance is enabled (read from the effective flag). UncoveredGlobalImmutableNonTxIndexIT/...2IT exercise the broadened serialize filter. RowTimestampIT locks the ROW_TIMESTAMP carve-out. The metrics/RPC ITs pin the flag off for assertions that account for client-side index mutations; IndexToolIT reads the effective flag. Heavy immutable/index ITs are exercised in CI.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

…efault

Brings the server-side immutable-index correctness hardening to the
Consistent Failover feature branch and enables it by default
(DEFAULT_SERVER_SIDE_IMMUTABLE_INDEXES_ENABLED=true). Immutable, global,
non-transactional secondary indexes are maintained server-side by
IndexRegionObserver rather than by the client, while immutable tables
that declare a ROW_TIMESTAMP column stay client-maintained regardless of
the flag.

Hardening (identical to the off-by-default change on master):
- Centralize the maintenance-side decision in
  IndexUtil.isServerSideImmutableIndexMaintenanceEnabled(...) and route
  every data-table gate through it (getClientMaintainedIndexes, the
  INDEX_UUID gate, MutationState.filterIndexCheckerMutations,
  DeleteCompiler.isMaintainedOnClient, IndexMetaDataCacheClient, and
  UpsertCompiler) so client and server never disagree.
- Force a current-row read-back for immutable batches carrying a covered
  or uncovered global index when an enabled mutation omits one of that
  index's on-disk columns, so partial upserts no longer drop a covered
  column or write a spurious null-keyed uncovered entry.
- Broaden the immutable server-serialize filter to IndexUtil.isGlobalIndex
  so uncovered global immutable indexes with a matching storage scheme are
  maintained.

Tests: partial-upsert/delete coverage in BaseImmutableIndexIT runs under
both ServerSideImmutableIndexIT and ClientSideImmutableIndexIT; the ITs
that account for client-side index mutations pin the flag off; the
IndexTool and GlobalIndexCheckerIT assertions read the effective flag so
they stay correct under either default.

Generated-by: Claude Code (Opus 4.8)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lokiore
lokiore marked this pull request as ready for review September 15, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants