HD — Nomad Deployment Example¶

This directory contains a reference HashiCorp Nomad deployment for a HD instance and its supporting services.

Note: These job files are examples. They are derived from the sandbox Docker Compose configuration and the docs/k8s_example Helm chart. Adapt them to your environment's secrets management, network layout, and node topology before production use.

Directory Structure¶

nomad_example/
├── README.md                            # This file
├── node_meta.hcl                        # Node meta key reference (apply to all nodes)
├── variables/
│   └── fermihdi.hcl                     # Nomad Variable definitions (secret placeholders)
├── jobs/
│   ├── dms.nomad.hcl                    # DMS-plane services (service job)
│   ├── sc.nomad.hcl                     # Storage Cluster (parameterized batch job)
│   ├── edge.nomad.hcl                   # HD Proxy (service job)
│   └── optional.nomad.hcl               # Optional services (service job, count-gated)
└── scripts/
    ├── deploy_sc.sh                     # Helper: dispatch SC job per cluster
    └── sc_config.example.json           # Example SC config for deploy_sc.sh

How Storage Clusters Work¶

The jobs/sc.nomad.hcl is a parameterized batch job. This means: 1. The job template is registered once with nomad job run jobs/sc.nomad.hcl. 2. Each Storage Cluster is deployed by dispatching the job with meta parameters specific to that SC. 3. The deploy_sc.sh script automates multi-SC dispatch from a JSON config file.

+---------+     dispatch meta     +------------------+
| sc.json | ──────────────────▶  | fermihdi-sc (job)|
+---------+                       |   SCC allocation  |
                                  |   SN-1 allocation |
                                  |   SN-2 allocation |
                                  |   SN-3 allocation |
                                  +------------------+

Quick Start¶

1. Set up Nomad Variables (secrets)¶

# Copy the template and fill in your secrets:
cp variables/fermihdi.hcl /tmp/fermihdi.hcl
# Edit /tmp/fermihdi.hcl with real secret values

# Upload to Nomad:
nomad var put fermihdi/secrets @/tmp/fermihdi.hcl

# Verify:
nomad var get fermihdi/secrets

2. Apply node meta to all nodes¶

# Set global meta on every Nomad client node:
for NODE_ID in $(nomad node status -short | awk 'NR>1 {print $1}'); do
  nomad node meta apply -node-id "$NODE_ID" \
    "fermihdi_image_registry=cr.fermihdi.io/hd" \
    "fermihdi_image_tag=2.0.2" \
    "fermihdi_dms_host=192.168.10.10" \
    "fermihdi_dms_url=https://192.168.10.10:6986" \
    "fermihdi_dms_internal_url=https://192.168.10.10:6985" \
    "fermihdi_otel_url=http://192.168.10.10:4318" \
    "fermihdi_control_plane_cidr=10.147.0.0/24" \
    "fermihdi_data_plane_cidr=10.150.0.0/24" \
    "fermihdi_observability_cidr=10.146.0.0/24" \
    "fermihdi_data_dir=/opt/fermihdi" \
    "fermihdi_log_level=INFO"
done

# Assign roles to specific nodes:
nomad node meta apply -node-id <dms-node-id>       fermihdi_role=dms
nomad node meta apply -node-id <proxy-node-id>     fermihdi_role=proxy
nomad node meta apply -node-id <optional-node-id>  fermihdi_role=optional

3. Deploy DMS services¶

nomad job run jobs/dms.nomad.hcl
nomad job status fermihdi-dms

4. Deploy Storage Clusters¶

# Register the parameterized job template:
nomad job run jobs/sc.nomad.hcl

# Configure your SCs:
cp scripts/sc_config.example.json scripts/sc_config.json
# Edit sc_config.json: set Nomad node UUIDs and tokens for each SC/SN

# Dispatch all SCs:
chmod +x scripts/deploy_sc.sh
./scripts/deploy_sc.sh --config sc_config.json

# Check status:
nomad job dispatches fermihdi-sc

5. Deploy HD Proxy¶

nomad job run jobs/edge.nomad.hcl

6. Enable optional services¶

nomad job run \
  -var "enable_ingester=1" \
  -var "enable_fe_traefik=1" \
  -var "enable_hd_mcp=1" \
  -var "enable_hd_graphql=1" \
  -var "enable_hd_data_explore=1" \
  jobs/optional.nomad.hcl

Node Targeting Summary¶

Job	Targeting Method	Meta Key
`fermihdi-dms`	`meta.fermihdi_role = dms`	`fermihdi_role`
`fermihdi-sc` (SCC group)	`node.unique.id = NOMAD_META_scc_node_id`	Dispatch meta
`fermihdi-sc` (SN groups)	`node.unique.id = sn_node_ids[N]`	Dispatch meta
`fermihdi-edge`	`meta.fermihdi_role = proxy`	`fermihdi_role`
`fermihdi-optional`	`meta.fermihdi_role = optional`	`fermihdi_role`

Network Port Reference¶

Port	Protocol	Plane
`6985`	TLS/mTLS	DMS Internal
`6986`	TLS/mTLS	Control Plane
`6987`	TCP	Data Plane (FHDWP)
`6988`	UDP	NORM Multicast (Auxiliary)
`6989`	TLS/mTLS	DMS-CA Plane
`8080`	HTTP	Observability

Prerequisites¶

Nomad ≥ 1.7 with the Docker task driver enabled on all client nodes
Nomad Variables feature available (Nomad ≥ 1.4)
Docker installed on all client nodes
Hugepages configured on SCC and SN nodes (hugepagesz=2M hugepages=1024 in GRUB)
jq installed on the machine running deploy_sc.sh
NOMAD_ADDR and NOMAD_TOKEN environment variables set for CLI access