Skip to content

FermiHDI SDK — Rust

BETA. Builds on every change (cargo build), but the exposed surface is narrow — a bridge, not yet a full binding. Expect to extend it.

A cxx bridge: Rust calls the C++ SDK directly through generated shims, so the ownership and ordering rules of the C++ API apply unchanged. The SDK overview explains those rules, starting with §1, the control plane / data plane split.

Modes

The bridge exposes the C ABI's query client and nothing else: set_scc_target is there, register_with_dms and an ingest client are not, because the C ABI does not have them yet. A Rust program runs in proxy mode through the Query Proxy, or can be handed SCC targets by something that registered on its behalf, but cannot itself register as an HD_CLIENT node or ingest. The SDK overview compares the modes and says which bindings have each.

Add the crate

The crate ships in the SDK package image at /opt/fermihdi-sdk/rust. Copy the package into your build image and reference the crate by path:

[dependencies]
fermihdi-sdk = { path = "/opt/fermihdi-sdk/rust" }

Linking against the real library needs the full library list — -lfermihdi_client alone leaves fmt, hwloc, NORM, boost, c-ares and yaml-cpp undefined, because the shared object embeds the HD engine's runtime through a static archive and does not re-export its dependencies.

// build.rs
println!("cargo:rustc-link-search=native={}/lib", sdk_root);
for lib in ["fermihdi_client","arrow","norm","fmt","hwloc","yaml-cpp",
            "boost_program_options","boost_thread","cares","curl"] {
    println!("cargo:rustc-link-lib=dylib={lib}");
}

API

The bridge currently exposes:

fn new_ingest_client() -> UniquePtr<FermiHDIIngestClient>;
fn load_dataset_schema(self: Pin<&mut FermiHDIIngestClient>, schema: &CxxString);

fn new_query_client(proxy_url: &CxxString) -> UniquePtr<FermiHDIQueryClient>;
fn generate_hd_query(self: Pin<&mut FermiHDIQueryClient>, query: &HDQuery)
       -> UniquePtr<CxxString>;
fn execute_to_sql(self: Pin<&mut FermiHDIQueryClient>,
                  hd_sql: &CxxString, duck_sql: &CxxString) -> bool;
fn prepare_query(self: Pin<&mut FermiHDIQueryClient>, query_id: &CxxString);
fn execute_sql_on_empty(self: Pin<&mut FermiHDIQueryClient>, duck_sql: &CxxString) -> bool;
fn set_filter_verdict_json(self: Pin<&mut FermiHDIQueryClient>,
                           verdict_json: &CxxString) -> bool;

QueryClient wraps the last two as prepare_query(&str) and set_filter_verdict_json(&str). The verdict is for a query you dispatched yourself: the proxy's result event JSON, verbatim, so the SDK knows what the filter service would have excluded. Without it the false-negative check at the end of the query never runs.

Everything is UniquePtr / Pin<&mut> because these are C++ objects with C++ lifetimes; cxx will not let you forget it.

Example

use cxx::let_cxx_string;

let_cxx_string!(proxy = "https://proxy.internal:443");
let mut client = ffi::new_query_client(&proxy);

let_cxx_string!(schema = SCHEMA_JSON);
// ... populate an HDQuery, including a distinct query_id and reachable targets

let body = client.pin_mut().generate_hd_query(&query);
if body.is_empty() {
    // the full-dataset-scan guard refused an open query
    return Err("open query not acknowledged".into());
}

let_cxx_string!(sql = "SELECT ip_src, bytes FROM ipflows LIMIT 10");
let ok = client.pin_mut().execute_to_sql(&body, &sql);

Use cases

A Rust ingest or query sidecar that must not carry a Python or JVM runtime. Link the static archive and ship one binary.

Extending the bridge. The C++ API is much wider than what is exposed here. Add the method to the #[cxx::bridge] block and it becomes available; the compiler will tell you immediately if the signature is wrong, which is the main reason to prefer cxx over hand-written FFI for this.

Gotchas

  • An empty generate_hd_query result is the full-dataset-scan guard.
  • You own the ending. This bridge does not submit the query, so relay complete or cancel by whatever route you dispatched.
  • Query IDs must be distinct across every client talking to the cluster — seed at random, not from the clock.