docs.fermihdi.io — data_handling.md

/ › hd › 2.0.5 › md › procurement

Download data_handling.mdView raw

Data handling and opt-in

HD runs entirely inside your environment. Your records, schemas, queries and user identities stay there. The only data that can leave is operational telemetry and crash information, and only when an administrator opts in. This page sets out what that data is, how you control it, where it goes and who can see it.

The short version

Three kinds of data

Kind Examples Leaves your deployment?
Your data Records, dataset schemas and names, node names, IP addresses, user identities, query text, anything shown in a console Never. It is not part of what the collector forwards, and FermiHDI's intake discards anything outside the permitted fields listed below.
Operational telemetry Licence ID, product version and build, node and cluster counts, feature usage counts, latency measurements, error types, crash stacks without memory Only while the opt-in is on.
Full crash reports The memory regions captured at a crash, which can contain record bytes, key material or a query in flight Only when an operator presses Share full report for that report.

Permitted telemetry fields

Operational telemetry leaves with these attributes and no others. Everything else is removed by your collector before sending, and discarded again at FermiHDI's intake:

service.name, service.version, service.namespace, deployment.environment, fermihdi.build.commit, fermihdi.product, fermihdi.license_id, fermihdi.region, host.arch, os.type, fermihdi.opt_in, http.request.method, http.response.status_code, http.method, http.status_code, http.route, rpc.method, rpc.service, error.type, fermihdi.feature, fermihdi.node_role, log.level, event.category, event.action, gen_ai.operation.name, gen_ai.tool.name.

How opt-in works

  1. Installation. The installer can disable sharing outright (fermihdi_dms_enable_analysis: false). A deployment installed that way cannot be switched on from the console.
  2. The switch. An administrator turns on Cloud sharing on the WebUX Telemetry & Logs page. DMS-Core records the decision and it survives restarts.
  3. Registration. DMS-Core registers the deployment with FermiHDI's opt-in service over mutual TLS, using a certificate chained to FermiHDI's root. The registration carries the licence ID, the region, a certificate request for the collector's identity, and the crash vault's relay identity and public key. None of these is a credential or customer data.
  4. The collector. DMS-Core renders the collector's configuration. The FermiHDI export pipeline exists in that configuration only while sharing is on and the registration is valid. It removes every attribute not on the permitted list, then sends over mutual TLS.
  5. Turning it off. Turning the switch off revokes the registration and removes the export pipeline. The collector stops forwarding immediately.

No engine node or service is told about the decision, and none of them changes what it reports. They always report to your own collector and crash vault, and those two decide what, if anything, goes further. For network controls, the collector reaches FermiHDI's telemetry endpoint and the crash vault its crash endpoint, both in the Singapore region.

Filters and destinations you control

The crash vault

Every crash in the deployment is captured on your own infrastructure first. The engine roles report through Crashpad, DMS-Core and the Proxy through the Go runtime, and the WebUX through Node.js. Reports go to the crash vault, a service on the DMS host.

Where shared data goes

Shared telemetry and crash data go to FermiHDI's opt-in service in DigitalOcean's Singapore region (SGP1), on infrastructure FermiHDI controls. DigitalOcean's data centres and platform, including SGP1, are covered by DigitalOcean's SOC 2 Type II report and ISO/IEC 27001 certification. DigitalOcean provides the infrastructure only and has no product-level access to the data.

Access to the opt-in service

The opt-in service runs in a private network (a DigitalOcean VPC) with no public access. The only connections it accepts from outside are these:

FermiHDI sends none of this data to any other external system: no third-party analytics, advertising, error-tracking or other software-as-a-service provider receives it.

If you never opt in

Nothing described above leaves your network. HD operates fully without the opt-in, and every capability, including crash capture, symbolication and export, works locally.

Download data_handling.mdView raw