Default propagation uses one worker

perfloop/weaviate · SERIAL WORK

https://perfloop.ai/t/oss/case_djvgdcg85s

Verdict

CLOSED · opened 2026-08-12

What happened: Closed; the assigned hypothesis was retired.

Hypothesis

I traced `hashBeat` into `propagateObjects` and ran `go test -tags=integrationTest ./adapters/repos/db -run '^TestPropagateObjects$' -count=1`; it passed. That focused test exercises the existing two-worker path and verifies that all objects are forwarded, so multi-worker propagation already has a functional test path. It does not measure remote latency.

The default configuration sets a propagation limit of 1,000, a batch size of 100, and propagation concurrency of one. A full propagation cycle therefore creates ten disjoint batches, while one worker performs each batch's local read and remote `Overwrite` before taking the next. The worker-pool implementation already permits more workers, and the batches share only read-only input and later result aggregation. This removes up to nine sequential inter-batch waits if overwrite spans are comparable and the target, network, and local storage have headroom; that latency share remains a hypothesis. The fast configured cadence is five seconds while propagation remains active.

A case session should inject 1,000 stale objects and compare the current one-worker default with a bounded multi-worker default. Record per-batch storage and `Overwrite` spans plus `AsyncReplicationPropagationDuration` and end-to-end hashbeat duration. Confirm the baseline is a serial waterfall, all objects and error behavior remain unchanged, and the faster setting does not increase timeout, retry, or target-overload signals.

Change to test: Use a bounded multi-worker default for multi-batch propagation while retaining an operator-configurable cap. Preserve the existing cancellation, result aggregation, and failure semantics.

Where it lives

perfloop/weaviate · adapters/repos/db/async_replication_scheduler.go

Evidence

Timeline