FermiHDI SDK — R¶

BETA. Builds, installs and loads on every change. Newly repaired: this package could not compile in any checkout before 2026-09-05, because it called a method the SDK has never had.

An Rcpp package: R functions wrap the C++ SDK classes one to one, so an R session drives the same query and ingest clients the other bindings do. The SDK overview describes those clients; its §1, the control plane / data plane split is the part an R user most needs.

Modes¶

The Rcpp package wraps the C++ query client through the C ABI, so it has what the C ABI has: querying in proxy mode, and explicit SCC targets, but no register_with_dms and no ingest client. An R session cannot register as an HD_CLIENT node or ingest until the C ABI gains those calls. The SDK overview shows the two modes side by side.

Install¶

export FERMIHDI_SDK_ROOT=/opt/fermihdi-sdk
GLIBC_TUNABLES=glibc.rtld.optional_static_tls=1048576 R CMD INSTALL r

Both settings are required:

FERMIHDI_SDK_ROOT tells src/Makevars where the headers and library are. It also sets CXX_STD = CXX20, because R defaults to gnu++17 and the SDK headers will not parse under it, and passes the full library list with -Wl,--no-as-needed — without which the package links cleanly and then fails at dyn.load() on an undefined boost symbol.
GLIBC_TUNABLES because R dlopens the package and the SDK carries the HD engine runtime's initial-exec TLS. Without it: cannot allocate memory in static TLS block. See §5.3.

If you are rebuilding after editing, remove stale objects first — R reuses any .o it finds, so a changed Makevars can appear to do nothing:

rm -f r/src/*.o r/src/*.so

API¶

sdk_version()                                  # SDK version string
sdk_discover_targets(proxy_url)                # logical
sdk_health(proxy_url)                          # JSON: status (liveness), ready
sdk_load_schema(proxy_url, schema_json)        # logical
sdk_submit_query(proxy_url, query_json, query_id)
    # list(ok, empty, found_batches, excluded_batches,
    #      would_exclude_batches, bloomrf_mode, error)
sdk_filter_verdict(proxy_url, verdict_json)    # JSON: bloomrf_mode, false_negatives

excluded_batches are batches a filter service proved cannot match: not in found_batches, nothing will arrive for them, and not an error. would_exclude_batches is what an enforcing filter would have excluded in passive mode; those were dispatched anyway. bloomrf_mode says which of off, passive, active or mixed the cluster is in.

sdk_submit_query() places the query through the proxy and holds the connection for the life of the query, reporting the ending itself — see §2.3. You do not need to relay endings when you use it.

Example¶

library(FermiHDISDK)

cat(sdk_version(), "\n")

health <- jsonlite::fromJSON(sdk_health("https://proxy.internal:443"))
if (!isTRUE(health$ready)) message("client is alive but not ready yet")

schema <- readLines("schema.json", warn = FALSE)
sdk_load_schema("https://proxy.internal:443", paste(schema, collapse = ""))

res <- sdk_submit_query(
  "https://proxy.internal:443",
  query_json,
  query_id = sample.int(.Machine$integer.max, 1)   # distinct per query
)

if (!res$ok) stop("dispatch failed: ", res$error)
if (res$empty) message("no batches in that window")
length(res$found_batches)

Use cases¶

Statistical analysis over a time window. Resolve the batches through sdk_submit_query, then work in R's native ecosystem.

Health checks from an R service. sdk_health() distinguishes liveness from readiness — wire status to a liveness probe and ready to a readiness probe; they are deliberately different answers.

Gotchas¶

status: "ok" with ready: false is normal for a client whose engine has not booted. It is alive; restarting it will not make the proxy reachable.
Stale .o files silently defeat Makevars changes.
Query IDs must be distinct across all clients talking to the cluster.