HD — Nomad Deployment Example

This directory contains a reference HashiCorp Nomad deployment for a HD instance and its supporting services.

Note: These job files are examples. They are derived from the sandbox Docker Compose configuration and the docs/k8s_example Helm chart. Adapt them to your environment's secrets management, network layout, and node topology before production use.

Directory Structure

nomad_example/
├── README.md                            # This file
├── node_meta.hcl                        # Node meta key reference (apply to all nodes)
├── variables/
│   └── fermihdi.hcl                     # Nomad Variable definitions (secret placeholders)
├── jobs/
│   ├── dms.nomad.hcl                    # DMS-plane services (service job)
│   ├── sc.nomad.hcl                     # Storage Cluster (parameterized batch job)
│   ├── edge.nomad.hcl                   # HD Proxy (service job)
│   └── optional.nomad.hcl               # Optional services (service job, count-gated)
└── scripts/
    ├── deploy_sc.sh                     # Helper: dispatch SC job per cluster
    └── sc_config.example.json           # Example SC config for deploy_sc.sh

How Storage Clusters Work

The jobs/sc.nomad.hcl is a parameterized batch job. This means:

1. The job template is registered once with nomad job run jobs/sc.nomad.hcl.
2. Each Storage Cluster is deployed by dispatching the job with meta parameters specific to that SC.
3. The deploy_sc.sh script automates multi-SC dispatch from a JSON config file.

+---------+     dispatch meta     +------------------+
| sc.json | ──────────────────▶  | fermihdi-sc (job)|
+---------+                       |   SCC allocation  |
                                  |   SN-1 allocation |
                                  |   SN-2 allocation |
                                  |   SN-3 allocation |
                                  +------------------+

Quick Start

1. Set up Nomad Variables (secrets)

# Copy the template and fill in your secrets:
cp variables/fermihdi.hcl /tmp/fermihdi.hcl
# Edit /tmp/fermihdi.hcl with real secret values

# Upload to Nomad:
nomad var put fermihdi/secrets @/tmp/fermihdi.hcl

# Verify:
nomad var get fermihdi/secrets

2. Apply node meta to all nodes

# Set global meta on every Nomad client node:
for NODE_ID in $(nomad node status -short | awk 'NR>1 {print $1}'); do
  nomad node meta apply -node-id "$NODE_ID" \
    "fermihdi_image_registry=cr.fermihdi.io/hd" \
    "fermihdi_image_tag=2.0.2" \
    "fermihdi_dms_host=192.168.10.10" \
    "fermihdi_dms_url=https://192.168.10.10:6986" \
    "fermihdi_dms_internal_url=https://192.168.10.10:6985" \
    "fermihdi_otel_url=http://192.168.10.10:4318" \
    "fermihdi_control_plane_cidr=10.147.0.0/24" \
    "fermihdi_data_plane_cidr=10.150.0.0/24" \
    "fermihdi_observability_cidr=10.146.0.0/24" \
    "fermihdi_data_dir=/opt/fermihdi" \
    "fermihdi_log_level=INFO"
done

# Assign roles to specific nodes:
nomad node meta apply -node-id <dms-node-id>       fermihdi_role=dms
nomad node meta apply -node-id <proxy-node-id>     fermihdi_role=proxy
nomad node meta apply -node-id <optional-node-id>  fermihdi_role=optional

3. Deploy DMS services

nomad job run jobs/dms.nomad.hcl
nomad job status fermihdi-dms

4. Deploy Storage Clusters

# Register the parameterized job template:
nomad job run jobs/sc.nomad.hcl

# Configure your SCs:
cp scripts/sc_config.example.json scripts/sc_config.json
# Edit sc_config.json: set Nomad node UUIDs and tokens for each SC/SN

# Dispatch all SCs:
chmod +x scripts/deploy_sc.sh
./scripts/deploy_sc.sh --config sc_config.json

# Check status:
nomad job dispatches fermihdi-sc

5. Deploy HD Proxy

nomad job run jobs/edge.nomad.hcl

6. Enable optional services

nomad job run \
  -var "enable_ingester=1" \
  -var "enable_fe_traefik=1" \
  -var "enable_hd_mcp=1" \
  -var "enable_hd_graphql=1" \
  -var "enable_hd_data_explore=1" \
  jobs/optional.nomad.hcl

Node Targeting Summary

Network Port Reference

Prerequisites

• Nomad ≥ 1.7 with the Docker task driver enabled on all client nodes
• Nomad Variables feature available (Nomad ≥ 1.4)
• Docker installed on all client nodes
• Hugepages configured on SCC and SN nodes (hugepagesz=2M hugepages=1024 in GRUB)
• jq installed on the machine running deploy_sc.sh
• NOMAD_ADDR and NOMAD_TOKEN environment variables set for CLI access