HD — Kubernetes Deployment Example

This directory contains a reference Kubernetes deployment for a HD instance and its supporting services.

Where this now lives: the chart under helm/fermihdi/ was promoted to install/helm/fermihdi, which is the copy that is kept current and the one install/fermihdi-k8s renders values for from cluster.yaml. This directory stays as a reference — where the two disagree, the one under install/ is the one to trust.

Note: These manifests are examples. They are derived from the sandbox Docker Compose configuration and must be adapted to your cluster's storage classes, ingress controllers, secret management strategy, and certificate infrastructure before use in any environment.

Directory Structure

k8s_example/
├── README.md                    # This file
├── deployment.yaml              # Flat "all-in-one" K8s manifest (reference)
└── helm/
    └── fermihdi/
        ├── Chart.yaml           # Helm chart metadata
        ├── values.yaml          # Default values (override per-environment)
        └── templates/
            ├── _helpers.tpl                   # Shared template helpers
            ├── namespaces.yaml                # Namespace definitions
            ├── secrets.yaml                   # Secret placeholders (populate externally)
            ├── dms/
            │   ├── dms-ca-deployment.yaml
            │   ├── dms-core-deployment.yaml
            │   ├── dms-opa-engine-deployment.yaml
            │   ├── dms-webux-deployment.yaml
            │   ├── otel-collector-deployment.yaml
            │   └── services.yaml
            ├── sc/
            │   ├── scc-deployment.yaml
            │   ├── sn-statefulset.yaml
            │   └── services.yaml
            ├── edge/
            │   ├── ingester-deployment.yaml
            │   ├── proxy-deployment.yaml
            │   └── services.yaml
            └── extra/
                ├── hd-mcp-deployment.yaml
                ├── hd-graphql-deployment.yaml
                ├── hd-data-explore-deployment.yaml
                └── services.yaml

Service Groups

Key Design Decisions

Namespaces

Three namespaces are used to reflect network plane separation:

• fermihdi-dms — DMS services
• fermihdi-sc — Storage Cluster (SCC + SNs)
• fermihdi-edge — Ingester and Proxy
• fermihdi-extra — Extra consumer-facing services

Network Planes

In the Docker Compose sandbox, three logical planes are used via separate Docker networks:

• Control Plane (port 6986) — mTLS REST communication between nodes and DMS-Core
• Data Plane (port 6987) — FHDWP binary protocol for record transport
• Observability Plane (port 8080) — Prometheus scraping, healthchecks

In Kubernetes, these planes are enforced via NetworkPolicy resources rather than separate Docker networks. Each service is labelled with fermihdi.dev/plane and NetworkPolicies restrict cross-plane traffic accordingly.

Secrets

All secrets (TLS certificates, bootstrap tokens, encryption keys) are represented as placeholder Kubernetes Secrets. You must populate these via your secrets management solution (Vault, External Secrets Operator, Sealed Secrets, etc.) before deployment.

Storage Nodes (SNs)

SNs are deployed as a StatefulSet to ensure stable Pod identities and ordered startup. Each SN Pod requests a PersistentVolumeClaim via volumeClaimTemplates. The number of replicas is controlled by values.yaml.

Hugepages

Both the SCC and SNs require hugepages (/dev/hugepages). In Kubernetes this is expressed as a resource request/limit of hugepages-2Mi or hugepages-1Gi. Your nodes must be pre-configured with hugepages.

DMS-CA Isolation

The DMS-CA is placed on its own isolated network segment. In this K8s layout it runs in fermihdi-dms but has a dedicated NetworkPolicy restricting access to DMS-Core only.

Quick Start (Helm)

# 1. Create namespaces
kubectl apply -f helm/fermihdi/templates/namespaces.yaml

# 2. Populate secrets (edit helm/fermihdi/templates/secrets.yaml with real values,
#    or use External Secrets / Vault)

# 3. Install the chart
helm install fermihdi ./helm/fermihdi \
  --namespace fermihdi-dms \
  --values ./helm/fermihdi/values.yaml

# 4. For a specific environment override:
helm install fermihdi ./helm/fermihdi \
  --values ./helm/fermihdi/values.yaml \
  --values ./my-env-overrides.yaml

Network Port Reference