Send single-partition ingest blocks directly to their resolved partition

perfloop/victoriametrics · FAN OUT

https://perfloop.ai/t/oss/case_n9d4377mmw

Verdict

PROPOSED · opened 2026-08-26

Hypothesis

Storage.add already obtains a partitionWrapper while resolving each row's indexDB. When the accepted block stays in that partition, it releases that wrapper and then calls table.MustAddRows anyway. MustAddRows snapshots every table partition, increments every wrapper reference, and tests the batch against each partition before reaching the same partition.AddRows call. Thus a current-partition block fans out across retained historical partitions after its route was already known.

Keep the lookup wrapper through raw-row dispatch and use it directly only when no accepted row left its timestamp range. Blocks spanning partitions should retain the existing MustAddRows path, including its bucketing and creation behavior. The remaining floor for the direct case is TSID resolution, per-day index maintenance, and raw-row shard insertion rather than table-wide partition discovery.

A later case should profile end-to-end remote-write ingestion with cache-hit single-partition blocks while sweeping retained partition count and concurrent writers. It should verify that GetAllPartitions, wrapper refcount traffic, and MustAddRows time scale away on the direct path, and falsify the move if those terms are not material or total ingestion CPU and tail latency do not improve without changing backfill correctness.

Change to test: Have Storage.add retain the partitionWrapper selected during TSID/index lookup and record whether every accepted raw row stayed in that partition. For that case, call ptw.pt.AddRows(rows) before releasing the held wrapper; release it and retain tb.MustAddRows only for blocks that cross partition ranges, preserving the existing multi-partition and partition-creation fallback.

Where it lives

perfloop/victoriametrics · app/vminsert/promremotewrite/request_handler.go

Evidence

No usable result yet.

Timeline