The BloomRF Filter Node

A storage cluster reads every batch a query's time window names, on every Storage Node, whatever the query's predicates say. The BloomRF filter node is the optional role that changes that: it keeps a small filter per batch and, before the Storage Cluster Controller (SCC) fans a query out, answers one question about it — of these batches, which cannot possibly match? Those batches are never read. Most deployments run without one, and a cluster with no filter node is the normal case rather than a degraded one.

What it is

A Storage Node that writes an index instead of records. The filter node joins the SCC's ingest broadcast exactly as a Storage Node does, computes the same chain hash over the same records, and verifies that hash with the SCC over the same stream. Where a Storage Node's second path writes the records to a device, this one folds them into a per-batch filter and discards them. Its store lives in memory, is marked usable only once the SCC has confirmed the batch's hash, and is bounded: the oldest batches are evicted first, and an evicted batch simply reads as "no filter".

Two structures are built in the same single pass over each batch:

Structure	Answers	Cost
Zone map — the minimum and maximum of every column	equality and ranges on a column whose values are clustered in time, such as a timestamp or a counter	a few bytes per column per batch
BloomRF — a presence filter per column, shaped by the column's width	equality on any column, and ranges on columns of up to eight bytes; a column of one or two bytes is answered exactly	by default 32 bits per record for an eight-byte column, 8 KiB per batch flat for a two-byte one

The zone map is asked first, because it is a handful of comparisons. Only a sub-query the zone map cannot settle reaches the BloomRF. A filter is built for every column unless the node is told which ones, and the zone map covers every column regardless.

What it never does

• It never decides what exists. The SCC resolves batches from its own index, as it always has, and then asks the filter node about that list. The reply is always a subset of what was asked, so the filter can never add a batch, resurrect one, or disagree with the SCC about what is held.
• It persists its filters, and only its filters. They are written to the node's local storage as batches close (FERMIHDI_BLOOMRF_DATA_DIR, default /var/lib/fermihdi/bloomrf; give it a volume) and reloaded at start, so a restart keeps the node's selectivity for the batches it retains. Losing that directory costs selectivity, never answers: a batch it holds no filter for is reported found. There is nothing else to migrate.
• It is never allowed to be wrong in the harmful direction. A false positive costs one batch read. A false negative would silently drop data, so every decision falls on the side of "read it": a missing filter, an unverified batch, a batch built under an older schema, a predicate it cannot decode soundly, an unreachable node or a lookup past its time budget — each means the batch is reported found. A Bloom filter has false positives and no false negatives, which is the right way round.

Every client checks. The SDK is the only component that sees both the filter's verdict and the records that arrive, so it compares them on every query: a batch the filter said could not match that nonetheless delivers a record is logged as an error the moment the query completes. A range filter is allowed no false negatives at all — one is a defect to report, not a rate to tolerate.

The three modes

The SCC's trust in the filter is a setting of the whole HD instance, pushed by the DMS to every SCC at once, so a cluster moves between modes in one action:

Mode	The SCC	Use it
off	never consults the filter	to take a filter node out of the query path without stopping it
passive	consults the filter, records what it would exclude, and reads every batch anyway	to measure a new filter node against real queries before trusting it
active	consults the filter and leaves the excluded batches unread	in operation

Once a filter node is provisioned for a cluster, the DMS pushes active unless the deployment names a mode; passive and off are what an operator chooses deliberately, on the Deployments page of the WebUX, which pushes the choice to every SCC of the instance at once. In every mode a lookup that fails or exceeds its time budget (10 ms by default) excludes nothing and the query proceeds unfiltered.

active is what the licence meters. When an SCC's mode becomes active with a filter node to consult, it takes one HD_BLOOMRF slot from the licence (or one from the HD_ANY pool when the element is not granted), and gives it back when it leaves active. An SCC that is refused a slot runs the filter passive instead: the filter is consulted and measured, every batch is still read, and the SCC's log carries bloomrf_license_refused with the reason. It asks again every five minutes, so a slot freed elsewhere or a larger licence is picked up without a restart. The filter node itself needs no slot.

The mode in force is visible on every query. The Proxy's manifest carries three batch lists beside the found and missing ones:

Field	Meaning
excluded_batches	batches that exist and were proven unable to match; nothing will be streamed for them
would_exclude_batches	in passive mode, what the filter would have excluded; the batches are read anyway
bloomrf_mode	off, passive, active — or mixed while a mode change is still rolling across the SCCs

A client on the current SDK drops excluded batches from its wait. An older client that does not know the field waits for a batch nothing will stream and hangs to its timeout, which is the one reason to upgrade clients before switching a cluster to active.

Provisioning and placement

The filter node is provisioned in the DMS WebUX like a Storage Node — Provision New Node, type BloomRF Filter Node, the instance, and the storage cluster whose SCC it serves. It is always authorised: the licence is drawn on by the SCC that puts it into active mode, not by the node (see the modes above). It registers with its own bootstrap token, like every node, and receives its runtime configuration from the DMS: record size, the hash election, the SCC's address and the dataset schema. A schema change discards every filter and they rebuild from the stream.

Two rules follow from how the SCC learns about it:

• One per storage cluster. The DMS hands the filter node's address to that cluster's SCC alone, and the SCC consults the first filter target it is given. Name it after the cluster — sc-1-bloomrf serves sc-1-scc — which is what the installer's validator checks.
• It must be up before the SCC's data plane starts. An SCC takes its relay list once, at that moment. The DMS therefore holds the SCC's targets back until every provisioned Storage Node and filter node in the cluster reports ready. Provision only what you will run: a provisioned filter node that never starts holds the whole cluster's ingest with it. To add one to a running cluster, provision it, start it, wait for it to report ready, then restart the SCC; to remove one, decommission it in the DMS and restart the SCC.

The installer knows the role as bloomrf: the interview offers it, the validator sizes it, and Ansible and both Helm charts run it. The steps around it — provisioning, the token, the mode — are in the Administrator's Guide, and the WebUX pages that do them in the WebUX walk-through.

Sizing

The filter node is an engine role: two cores (one for the ingest broadcast, one for the control plane), 300 × 2 MB hugepages for a 512 MB engine arena, no 1 GB pages and no devices. POSIX networking is enough for it; DPDK is optional as on every engine role.

Memory is the real cost, and it follows the data. A filter is built at a fixed size and folded at batch close to records × bits per key × levels bits, so it never needs the batch size in advance and the stored size follows the batch. Budget the store as retained batches × bytes per batch, where the node's metrics report the bytes held and its data-plane log records filter_bytes per batch. At the defaults an eight-byte column costs 32 bits per record, a two-byte column 8 KiB per batch flat, and a sixteen-byte column 8 bits per record. The store keeps 512 batches by default.

Variable	Default
FERMIHDI_DMS_URL, FERMIHDI_NODE_NAME, FERMIHDI_BOOTSTRAP_TOKEN	required	as every node
FERMIHDI_PRODUCTION_PORT	6987	the ingest broadcast; the SCC's question is answered on this port plus ten, 6997
FERMIHDI_OBSERVABILITY_PORT	8080	/healthz, /readyz, /metrics
FERMIHDI_SEASTAR_MEMORY	512M	the engine arena
FERMIHDI_BLOOMRF_RETAIN_BATCHES	512	the store's bound; oldest evicted first
FERMIHDI_BLOOMRF_COLUMNS	all	comma-separated column names to build a BloomRF for; the zone map covers every column regardless
FERMIHDI_BLOOMRF_BITS_PER_KEY	8	Bloom bits per record per level, after folding
FERMIHDI_BLOOMRF_LEVELS	4	prefix levels for columns of up to eight bytes; 1 answers equality only
FERMIHDI_BLOOMRF_MAX_PROBES	256	the budget for decomposing a range; a range past it gets "no opinion"
FERMIHDI_BLOOMRF_MAX_KIB_PER_COLUMN	1024	build-time allocation per column, folded down at close

On the SCC, FERMIHDI_BLOOMRF_MODE is the bootstrap value the DMS overrides, and FERMIHDI_BLOOMRF_TIMEOUT_MS (10) is the lookup budget.

What to watch

The node serves Prometheus metrics on its observability port. The ones that say whether it is doing its job:

Metric	Means
fermihdi_bloomrf_filters_verified / _pending / _rejected	filters the SCC has confirmed, is still to confirm, or refused. Pending that never drains means the verification stream to the SCC is not up
fermihdi_bloomrf_filters_bytes	the store's size — the figure to budget the memory limit from
fermihdi_bloomrf_filters_evicted	batches dropped for space. A steady climb means the retained-batch bound is smaller than the windows queries ask about
fermihdi_bloomrf_lookups, _lookup_candidates, _lookup_evaluated, _lookup_excluded	how often the SCC asks, how many batches it asks about, how many had a usable filter, how many were excluded — selectivity is excluded over candidates
fermihdi_bloomrf_lookup_excluded_zone_map / _excluded_bloomrf	which structure decided each exclusion
fermihdi_bloomrf_lookup_no_filter	candidates with no usable filter — evicted, unverified, or from before the node started
fermihdi_bloomrf_verify_mismatch	a hash the SCC disagreed with. Should stay at zero

The SCC logs one bloomrf_lookup line per query with the same figures on its side — mode, candidates, evaluated, no_filter, would_exclude, protocol_error, lookup_ms — which is where to look first when a filter seems to do nothing: no_filter equal to candidates means the node has no verified filters for the window, and mode: off means the SCC was never told about the node.

A filter that excludes nothing on a workload is not necessarily broken. Predicates on the batch-id column become the time window rather than a filter, != proves nothing, a range wider than the probe budget gets no opinion, and a value that is genuinely present in every batch is genuinely present. The filter earns its keep on selective predicates — an address, a port, a small value on a wide column — over windows of many batches.