IDMP expiry cron skips per-slot alloc_size update, overstating CLUSTER SLOT-STATS memory

perfloop/redis · RESOURCE LEAK

https://perfloop.ai/t/oss/case_bkwq58t3zt

Verdict

VERIFIED · settled 2026-09-28 · pull request opened as redis/redis#15887

What happened: The assertion is violated on the comparison and satisfied with this change.

Hypothesis

Expired stream idempotency entries release their allocations, but their hash slot can retain the old memory charge. `CLUSTER SLOT-STATS memory-bytes` may then overstate memory even after the stream is deleted. The submitted size discrepancy is from a local build and has not been remeasured in this run.

At revision 20bb2cfc54aa08c8fdfb8c4c0a8b8258e811711e, `serverCron` calls `handleExpiredIdmpEntries` every second (src/server.c:1812-1815). `XADD ... IDMP` registers the stream and updates the slot from the stream's old and new sizes (src/t_stream.c:2562-2565, 2640-2665). The cron function removes expired entries, empty producers, and sometimes their radix tree, but never calls `updateSlotAllocSize` (src/t_stream.c:6259-6341). `idmpEntryFree` and `idmpProducerFree` reduce `s->alloc_size`, which `kvobjAllocSize` includes (src/t_stream.c:6091-6097, 6115-6133; src/object.c:1362-1385). Its `keyModified` call only updates LRM here (src/db.c:1217-1223). `updateSlotAllocSize` adjusts the per-slot charge, and `CLUSTER SLOT-STATS` reads that charge (src/db.c:138-158; src/cluster_slot_stats.c:105-126).

With memory tracking enabled at startup by `cluster-slot-stats-enabled mem` (src/server.c:3087-3091) and `stream-idmp-duration 1`, the submitter reports a stream whose `MEMORY USAGE` fell from 442 to 298 after expiry while its slot still counted 442. After `DEL`, the slot reportedly counted 168 instead of the 32-byte baseline for its remaining key; these are not production measurements. A regression can `SET {t}keep`, record that slot's `memory-bytes`, `XADD {t}a IDMP p0 i0 * f v`, wait until `XINFO STREAM` reports no tracked producer, then `DEL {t}a` and compare slot bytes with the recorded baseline. A persistent positive remainder confirms the accounting drift; equality refutes it. This checks the accounting charge, not a physical heap leak.

Change to test: With memory tracking enabled, have handleExpiredIdmpEntries capture kvobjAllocSize(kv) before expiry and call updateSlotAllocSize(db, getKeySlot(key->ptr), kv, old_alloc, kvobjAllocSize(kv)) after all removals and any producer-tree free, including the drained-producer branch. Add a stream IDMP regression with cluster-slot-stats-enabled mem, stream-idmp-duration 1, expiry and DEL; require the slot's memory-bytes to match the remaining key's original size.

Where it lives

perfloop/redis · src/t_stream.c

Evidence

The assertion is violated on the comparison and satisfied with this change: `In cluster mode with cluster-slot-stats-enabled mem at startup, after IDMP cron expiry and deletion of the stream key, CLUSTER SLOT-STATS memory-bytes for its hash slot equals the pre-stream baseline for both drained and partially expired producer histories.`

Checks: 3 of 3 passed. Verification: no defect found.

Timeline