Skip to content

HD SDK — Client Sizing

What a client costs to run, and how many of them to run.

What one client reserves

Constructing either FermiHDIIngestClient or FermiHDIQueryClient boots the HD engine on its own thread and reserves one core and 1 GB of memory for it — see What construction actually does. That reservation is made once, in the constructor, and held for the life of the object.

Because of that reservation, a client is a long-lived object, not a per-request one:

  • Construct once per process, and keep it. The constructor blocks until the engine reports ready, so constructing one per record or per request costs both the core/memory reservation and that startup latency, repeatedly.
  • One client per process is the normal shape. A service that needs both read and write access holds one FermiHDIQueryClient and one FermiHDIIngestClient, not a pool of either.

GPU probing at construction

Constructing a client probes for a GPU runtime and logs which one it found, falling back to CPU routines when there is none. This is a detection step, not a promise of GPU-accelerated execution for every call — see Architecture for exactly which embedded engines exist today (execute_to_sql() via DuckDB, execute_vector_search() via USearch) and which declared methods (execute_graph(), set_local_aggregations()) are not implemented.

Sizing a host that runs several clients

Each client instance is one core and 1 GB, reserved for as long as the object lives. A host running several client processes — several query services, for example — needs that budget multiplied by however many long-lived clients it hosts, on top of whatever else those processes need. This is separate from, and much smaller than, the per-role core and hugepage budget for the engine roles (SCC, SN, the filter node) themselves, which the Administrator's Guide covers.

Languages that dlopen the SDK

Python, R and Node.js load the SDK's shared library into an already-running process rather than linking it at build time, which needs a static-TLS tunable set before the host process starts — see Dynamically loaded consumers need static TLS surplus. This is a startup-time requirement, not an ongoing resource cost, but it has to be set correctly or the import/require fails outright.

Where to go next