HD — Kubernetes Deployment Example¶

This directory contains a reference Kubernetes deployment for a HD instance and its supporting services.

Where this now lives: the chart under helm/fermihdi/ was promoted to install/helm/fermihdi, which is the copy that is kept current and the one install/fermihdi-k8s renders values for from cluster.yaml. This directory stays as a reference — where the two disagree, the one under install/ is the one to trust.

Note: These manifests are examples. They are derived from the sandbox Docker Compose configuration and must be adapted to your cluster's storage classes, ingress controllers, secret management strategy, and certificate infrastructure before use in any environment.

Directory Structure¶

k8s_example/
├── README.md                    # This file
├── deployment.yaml              # Flat "all-in-one" K8s manifest (reference)
└── helm/
    └── fermihdi/
        ├── Chart.yaml           # Helm chart metadata
        ├── values.yaml          # Default values (override per-environment)
        └── templates/
            ├── _helpers.tpl                   # Shared template helpers
            ├── namespaces.yaml                # Namespace definitions
            ├── secrets.yaml                   # Secret placeholders (populate externally)
            ├── dms/
            │   ├── dms-ca-deployment.yaml
            │   ├── dms-core-deployment.yaml
            │   ├── dms-opa-engine-deployment.yaml
            │   ├── dms-webux-deployment.yaml
            │   ├── otel-collector-deployment.yaml
            │   └── services.yaml
            ├── sc/
            │   ├── scc-deployment.yaml
            │   ├── sn-statefulset.yaml
            │   └── services.yaml
            ├── edge/
            │   ├── ingester-deployment.yaml
            │   ├── proxy-deployment.yaml
            │   └── services.yaml
            └── extra/
                ├── hd-mcp-deployment.yaml
                ├── hd-graphql-deployment.yaml
                ├── hd-data-explore-deployment.yaml
                └── services.yaml

Service Groups¶

Group	Services
DMS	DMS-Core, DMS-CA, DMS-WebUX, DMS-OPA-Engine, OTEL Collector
SC (Storage Cluster)	SCC (Storage Cluster Controller), SNs (Storage Nodes — StatefulSet)
Edge	Ingester (hd_message_bus_ingester), HD Proxy
Extra	hd_mcp, hd_graphql, hd_data_explore

Key Design Decisions¶

Namespaces¶

Three namespaces are used to reflect network plane separation: - fermihdi-dms — DMS services - fermihdi-sc — Storage Cluster (SCC + SNs) - fermihdi-edge — Ingester and Proxy - fermihdi-extra — Extra consumer-facing services

Network Planes¶

In the Docker Compose sandbox, three logical planes are used via separate Docker networks: - Control Plane (port 6986) — mTLS REST communication between nodes and DMS-Core - Data Plane (port 6987) — FHDWP binary protocol for record transport - Observability Plane (port 8080) — Prometheus scraping, healthchecks

In Kubernetes, these planes are enforced via NetworkPolicy resources rather than separate Docker networks. Each service is labelled with fermihdi.dev/plane and NetworkPolicies restrict cross-plane traffic accordingly.

Secrets¶

All secrets (TLS certificates, bootstrap tokens, encryption keys) are represented as placeholder Kubernetes Secrets. You must populate these via your secrets management solution (Vault, External Secrets Operator, Sealed Secrets, etc.) before deployment.

Storage Nodes (SNs)¶

SNs are deployed as a StatefulSet to ensure stable Pod identities and ordered startup. Each SN Pod requests a PersistentVolumeClaim via volumeClaimTemplates. The number of replicas is controlled by values.yaml.

Hugepages¶

Both the SCC and SNs require hugepages (/dev/hugepages). In Kubernetes this is expressed as a resource request/limit of hugepages-2Mi or hugepages-1Gi. Your nodes must be pre-configured with hugepages.

DMS-CA Isolation¶

The DMS-CA is placed on its own isolated network segment. In this K8s layout it runs in fermihdi-dms but has a dedicated NetworkPolicy restricting access to DMS-Core only.

Quick Start (Helm)¶

# 1. Create namespaces
kubectl apply -f helm/fermihdi/templates/namespaces.yaml

# 2. Populate secrets (edit helm/fermihdi/templates/secrets.yaml with real values,
#    or use External Secrets / Vault)

# 3. Install the chart
helm install fermihdi ./helm/fermihdi \
  --namespace fermihdi-dms \
  --values ./helm/fermihdi/values.yaml

# 4. For a specific environment override:
helm install fermihdi ./helm/fermihdi \
  --values ./helm/fermihdi/values.yaml \
  --values ./my-env-overrides.yaml

Network Port Reference¶

Port	Protocol	Plane
`6985`	TLS/mTLS	DMS Internal
`6986`	TLS/mTLS	Control Plane
`6987`	TCP	Data Plane (FHDWP)
`6988`	UDP	NORM Multicast (Auxiliary)
`6989`	TLS/mTLS	DMS-CA Plane
`8080`	HTTP	Observability