Interrupted multi-table writes retain an acquired view lock
perfloop-oss/cassandra · Resource Leak
https://perfloop.ai/t/oss/case_vvrd53sbtq
Verdict
VERIFIED · settled 2026-11-07 · pull request opened as apache/cassandra#5382
What happened: The check fails before this change and passes with it.
Hypothesis
A synchronous multi-table mutation can report interruption while retaining a view lock acquired earlier in the same operation. Other mutations needing that stripe can then wait after the owner has exited. The frequency and duration of this condition in deployed nodes are unknown.
Keyspace.applyInternal acquires locks before entering the try/finally at lines 549-598. In the non-deferrable branch it keeps earlier locks while retrying the current table. If Thread.sleep throws InterruptedException, line 527 throws UncheckedInterruptedException before that finally can run. The explicit prefix cleanup for timeout and deferral does not cover this exception. The supported trigger is a view-affecting synchronous mutation for two tables, a successful first acquisition, a contended second acquisition, and an interrupt flag or interruption at the retry sleep. The public synchronous apply seam selects isDeferrable=false. CommitLogReplayer.MutationInitiator also calls apply(mutation, false, true, false) on Stage.MUTATION, establishing a non-droppable consumer route without establishing that this cancellation occurs frequently there.
The native check `bash .build/perfloop-view-property.sh org.apache.cassandra.db.view.ViewLockExitPropertyTest` on prop_5srw93zcv6 completed at revision 966d96bc73979f9f5375367747e8ab87c7b037d5 in checkProperty call_9r7omnJ3s7BdeS8gJx7cW0oX. It used two real MV base tables, held the second stripe, and invoked public synchronous apply from a pre-interrupted writer. Both tested table orders threw the expected exception while the writer still owned the first stripe. The test recorded ownership before its own owner-thread cleanup and returned replay `0|false`. Exit 1 is the final assertion after both cases, not a startup or join timeout. The fixture retains the exact lock reference for observation. Because ViewManager uses lazy weak locks, this finding does not assert that an otherwise unreferenced abandoned stripe survives garbage collection forever.
This is distinct from hyp_f6dewb9dt8's lock-array allocation repair and from hyp_wgng8e0tmb's polling/backoff proposal. The related polling Case was closed by actor pause-removal because its Session stopped; that reason does not refute this exceptional-exit ownership defect. The larger per-row view-state proposal hyp_pqhhbarb0d does not repair this exit either.
The checked implementation in https://github.com/apache/cassandra/pull/4358 at 05f5aa9f99a6a183f0d79fbc19ad87caeef49fb7 bypasses ordinary view generation only for an opt-in strict-MV, single-table route; it leaves this multi-table acquisition loop intact. Its linked https://issues.apache.org/jira/browse/CASSANDRA-20880 and https://github.com/apache/cassandra/pull/4358#issuecomment-3700897823 describe an open migration option with one recorded approval, not an exclusion of ordinary multi-table writes. The target hunks of https://github.com/apache/cassandra/pull/5254 change table unloading, while https://github.com/apache/cassandra/pull/4627 adds mutation-tracking routing and retains the ordinary loop. The final refresh attempt was refused after the shared history-byte allowance was exhausted, so freshness beyond those checked heads remains an explicit evidence gap.
The smallest useful Case proof is to replay this exception path after a scoped cleanup change and verify that another thread can acquire the released prefix. It should also guard timeout, deferral and normal completion against double release and preserve the propagated exception. No throughput gain or incident rate is required to establish the ownership property. A repair should need only a scoped ownership change, not a new locking API or optional concurrency hardening.
Change to test: Make acquired view-lock ownership exception-safe across acquisition as well as the write phase, so interruption releases the entire acquired prefix before propagating failure. Preserve timeout and deferral behavior without double-unlocking; the current replay checker and file layout are provisional.
Where it lives
perfloop-oss/cassandra · src/java/org/apache/cassandra/db/Keyspace.java
Evidence
The check fails before this change and passes with it: `Synchronous Keyspace.apply (updateIndexes=true) of a mutation that updates three materialized-view base tables, on a thread whose interrupt flag is set while another thread holds the view lock of the last table in the mutation, throws UncheckedInterruptedException caused by InterruptedException, after which another thread can acquire each view lock the writer acquired for the other two tables; checked for droppable and non-droppable writes.`
The check fails before this change and passes with it: `For a droppable two-table MV-affecting mutation whose second table has been dropped, synchronous Keyspace.apply and default Keyspace.applyFuture with makeDurable=false and updateIndexes=true leave both view locks acquirable by another thread on exit; with potential transaction conflicts allowed they skip the missing table and apply the surviving update, otherwise they propagate InvalidRequestException.`
Checks: 2 of 2 passed. Verification: no defect found.
Timeline
2026-11-06· Case opened2026-11-07· PR opened