The SQL Server

hd_sql is a read-only SQL front door onto an HD instance. It accepts SQL over three client protocols at once and answers all three from one engine: a client sends a statement, the service works out which batches and which predicates the Storage Nodes should apply, has them stream only those records, and lets the analytical engine embedded in the SDK evaluate the rest. Nothing in a BI tool, a notebook or a JDBC pipeline has to know what an HD query is.

The rules shared by every query door — the time window, what pushes down, the sixteen-branch limit, addresses, empty windows — are on the querying page and are not repeated here.

Connecting

Port	Protocol	Clients
5432	PostgreSQL wire	psql, JDBC and ODBC drivers, DBeaver, Grafana's PostgreSQL source, Tableau
31337	Arrow Flight SQL over gRPC	columnar clients: ADBC, the Arrow Flight SQL JDBC driver, pandas and polars through ADBC
8081	HTTP	POST /v1/query with {"sql": "..."}; GET /v1/datasets and /v1/datasets/{id} for the catalog
8080	HTTP	/health and Prometheus /metrics
6989	FHDWP	where the Storage Nodes stream results to; the service's own listener

psql "host=hd-sql port=5432 dbname=hd user=analyst sslmode=disable" \
  -c "SELECT ip_src, sum(bytes) FROM ipflow WHERE timestamp > 1788660000 AND timestamp < 1788663600 GROUP BY 1 ORDER BY 2 DESC LIMIT 10"

curl -s http://hd-sql:8081/v1/query \
  -d '{"sql": "SELECT count(*) FROM ipflow WHERE timestamp > 1788660000 AND timestamp < 1788663600"}'

from adbc_driver_flightsql import dbapi
with dbapi.connect("grpc://hd-sql:31337") as c:
    table = c.cursor().execute("SELECT * FROM ipflow LIMIT 5").fetch_arrow_table()

The service reports itself as PostgreSQL so that drivers work, and the PostgreSQL listener takes an optional password (HD_SQL_PGWIRE_PASSWORD_FILE). It has no access control of its own beyond that: it is meant to run behind the same authenticating front door as the Proxy, which supplies the identity headers the Proxy's policy is evaluated against. A listener reachable from outside that front door gives every caller the access the service itself has, and the service says so in its start-up log when no password is set. The identity the Proxy evaluates for a query through this service is the service's own, not the SQL user's; see the front doors page.

What is there

SHOW TABLES;
SELECT * FROM information_schema.columns WHERE table_name = 'ipflow';

Each dataset is one table, named by the dataset's id; its human-readable name works too. HD's types map to the nearest SQL type that can hold them, widened rather than reinterpreted so that a uint32 above 2³¹ does not arrive as a negative number:

HD type	SQL type	Notes
uint8, int8, int16	smallint
uint16, int32	integer
uint32, int64, timestamp, ipv4	bigint	a timestamp is Unix seconds, not a SQL timestamp
uint64, uint128, int128, ipv6	numeric	no signed type is wide enough
uint256, int256	numeric	exact for storing, comparing, sorting and min/max; arithmetic over one is double precision
numeric	numeric(p,s)
float / double	real / double precision
boolean, bytes, string	boolean, bytea, text	a blob column holds the object key, not the object

An ipv4 or ipv6 column is presented as inet text and accepts an address literal in a predicate; comparison is on the integer underneath.

An enumerated column, an unsigned integer whose schema entry names one of the dataset's enums, is presented as text: its labels. A predicate takes a label, WHERE proto = 'tcp', and the storage nodes filter on the code it stands for. A code the table does not define reads as the enumeration's default label, and ORDER BY on the column orders labels, not codes.

A delta-compressed time column, an integer whose schema entry carries time_reference, is presented as bigint: the absolute time the stored delta stands for, in the column's own time unit or the source field's. A predicate on such a column is evaluated by the SQL server on the presented time, after the storage nodes have answered, and is never pushed down: the absolute time differs from record to record and has no byte pattern a node could match. It narrows the result, not the read, so a time range still belongs on the source field.

A chess column is presented as text: a chess_move as its Standard Algebraic Notation (Nf3) and a chess_fen as its Forsyth-Edwards string, and a predicate on either takes the notation, WHERE opening_move = 'e4' or WHERE position = '<FEN>', pushed to the storage nodes as the code or the packed position. Two position predicates are pushed as masked comparisons on the packed bytes: fen_occupied(position, 'e4') (also NOT, = FALSE) and fen_turn(position) = 'b'. The other position functions -- fen_piece(position, 'e4'), fen_castling, fen_en_passant, fen_halfmove, fen_fullmove, fen_piece_count, and san_code / san_text for moves -- are evaluated by the server over the rows the nodes return.

An array column, a bytes field whose schema entry declares an array, is presented as its element type followed by [] -- integer[], text[] -- a DuckDB LIST that the PostgreSQL wire carries as an array literal ({1,2}) with the matching array type, and the HTTP API as a JSON array. DuckDB's list functions apply: list_contains(tags, 'vip'), len(tags), unnest(tags). list_contains(col, literal) (or array_contains), and NOT list_contains, are pushed to the storage nodes as a walk over the stored elements, so they narrow the fetch like a scalar equality; the literal may be a label where the elements carry an enumeration. Any other predicate on an array column (a comparison against the whole column, len, a nested expression) is evaluated by DuckDB over the fetched rows.

What SQL is accepted

The statement is evaluated by DuckDB, so the dialect is DuckDB's: aggregates, window functions, CASE, GROUP BY, ORDER BY, LIMIT, the full expression language. What the service constrains is the shape:

• One table per statement. No joins, no subqueries in FROM, no UNION across tables.
• Read-only. INSERT, UPDATE, DELETE and DDL are refused. Data enters through ingestion.
• SET, BEGIN, COMMIT and SHOW are accepted and acknowledged. There are no transactions to be inside of.

A tool that assumes a full PostgreSQL will find gaps: no pg_catalog beyond the information_schema views above, no cursors, no COPY, no server-side prepared statements with binary parameters, and no out-of-band cancellation — closing the connection cancels the query.

Notices

The service tells you when it answered something other than exactly what you asked, as a PostgreSQL NOTICE, in the notices array over HTTP, and in its log for Flight SQL:

• No time bound was given, so the default window was read (the last hour, unless HD_SQL_DEFAULT_LOOKBACK_SEC says otherwise). A one-sided bound counts as none: timestamp > x alone cannot be turned into a list of batches. A statement with no bound at all reads the entire dataset and says so.
• A predicate could not be pushed to the Storage Nodes and was applied after the transfer. >= and <= are rewritten as the strict comparison beside them, which is exact on whole-number columns and left local on float and double; NOT anywhere disables pushdown; one OR branch the planner cannot read leaves the whole clause unconstrained.
• The clause needed more than sixteen branches, so none were pushed.

Errors

What happened	SQLSTATE	gRPC	HTTP
Statement is not supported	0A000	Unimplemented	400 unsupported_statement
Unknown table	42P01	NotFound	404 unknown_dataset
Could not parse the statement	42601	InvalidArgument	400 invalid_statement
Timed out waiting for the Storage Nodes	57014	DeadlineExceeded	504 query_timeout
Query Proxy unreachable	08006	Unavailable	502 upstream_unavailable

A timeout means the Storage Nodes did not deliver every batch the Proxy said existed within HD_SQL_QUERY_TIMEOUT_MS (five minutes by default). The query is cancelled at the Storage Cluster Controllers when that happens.

Running it

Setting	Default
HD_PROXY_ENDPOINT	—	the HD Proxy, e.g. https://proxy.hd.internal:6986
HD_DMS_ENDPOINT, HD_DMS_CA_FILE, HD_DMS_CLIENT_CERT_FILE, HD_DMS_CLIENT_KEY_FILE	—	where the dataset schema comes from, over mTLS. Without them the service takes the Proxy's copy
FERMIHDI_DATA_PLANE_SUBNET	10.150.0.0/24	the CIDR whose local address is advertised to the Storage Nodes; FERMIHDI_DATA_RETURN_HOST overrides discovery outright
FERMIHDI_DATA_RETURN_PORT	6989	where results arrive
HD_TIME_INDEX_INTERVAL	60	the batch width in seconds — must match the dataset
HD_SQL_DEFAULT_LOOKBACK_SEC	3600	the window a query with no time bound reads
HD_SQL_QUERY_TIMEOUT_MS	300000	the whole pipeline: dispatch, the wait for every Storage Node, the local evaluation
HD_SQL_MAX_ROWS	0	a cap on rows returned; 0 is unlimited
HD_SQL_QUERY_QOS	1	priority weight 1–255, sent to the Storage Nodes
HD_SQL_PGWIRE_ADDRESS, HD_SQL_FLIGHTSQL_ADDRESS, HD_SQL_HTTP_ADDRESS	:5432, :31337, :8081	each listener can be moved or disabled (HD_SQL_*_ENABLED)

The service reaches the Storage Nodes' results on the data plane, so it is placed like any query client: on a host the Storage Nodes can reach, with the data-plane subnet declared. The address it advertises is written into every query; if it is wrong the query is accepted and the results go nowhere, with no error on either side, which is the first thing to check when every query times out. /health reports the Proxy connection and the schema source.