HD — Ansible Deployment Example

This directory contains a reference Ansible deployment for a HD instance and its supporting services.

Note: These playbooks are examples. They are derived from the sandbox Docker Compose configuration and the docs/k8s_example Helm chart. Adapt them to your environment's secrets management, network layout, and certificate infrastructure before production use.

Directory Structure

ansible_example/
├── README.md                            # This file
├── site.yml                             # Master playbook (imports all groups)
├── inventory/
│   ├── hosts.yml                        # Host groups — set your target hosts here
│   └── group_vars/
│       ├── all.yml                      # Global vars (registry, CIDRs, OTEL URL)
│       ├── dms.yml                      # DMS group vars + secret placeholders
│       ├── sc.yml                       # Storage Cluster topology (storage_clusters list)
│       └── optional.yml                 # Optional service toggles & vars
├── playbooks/
│   ├── dms.yml                          # DMS services play
│   ├── sc.yml                           # Storage Cluster play (SCCs + SNs)
│   ├── edge.yml                         # Edge play (HD Proxy)
│   └── optional.yml                     # Optional services play
└── roles/
    ├── fermihdi_common/                 # Common: dirs, Docker install
    ├── dms_ca/                          # DMS-CA (certificate authority)
    ├── dms_core/                        # DMS-Core (management engine)
    ├── dms_webux/                       # DMS-WebUX (Astro.build web UI)
    ├── dms_opa_engine/                  # DMS-OPA-Engine (policy engine)
    ├── dms_traefik/                     # DMS front door, lab shape (Traefik)
    ├── dms_nginx/                       # DMS front door, production example (NGINX + OpenID Connect)
    ├── otel_collector/                  # OpenTelemetry Collector
    ├── sentry_relay/                    # Sentry Relay (error aggregation)
    ├── sc_scc/                          # Storage Cluster Controller
    ├── sc_sn/                           # Storage Node
    ├── hd_proxy/                        # HD Proxy (query gateway)
    ├── hd_ingester/                     # [optional] Message bus ingester
    ├── fe_traefik/                      # [optional] query front door, lab shape (Traefik)
    ├── fe_nginx/                        # [optional] query front door, production example (NGINX + OpenID Connect + client certificates)
    ├── hd_graphql/                      # [optional] HD GraphQL server
    ├── hd_mcp/                          # [optional] HD MCP server
    └── hd_data_explore/                 # [optional] Data Explorer UI

Quick Start

Configure your inventory

Edit inventory/hosts.yml and set the ansible_host of each service group to your actual server IPs/hostnames.

Configure global variables

Edit inventory/group_vars/all.yml:

• Set fermihdi_image_tag to your release tag
• Set fermihdi_control_plane_cidr, fermihdi_data_plane_cidr, fermihdi_observability_cidr to match your host network layout

Configure Storage Clusters

Edit inventory/group_vars/sc.yml — this is where you define how many Storage Clusters you want and how many SNs each one has:

storage_clusters:
  - name: "sc-1"
    scc_host: "sc1-scc-host"       # Must match inventory hostname
    sn_hosts:
      - "sc1-sn-host-01"           # Must match inventory hostnames
      - "sc1-sn-host-02"
      - "sc1-sn-host-03"
    scc_token: "..."               # Inject via ansible-vault
    sn_tokens: ["...", "...", "..."]

  - name: "sc-2"
    scc_host: "sc2-scc-host"
    sn_hosts:
      - "sc2-sn-host-01"
    scc_token: "..."
    sn_tokens: ["..."]

The playbook automatically derives node names as <name>-scc and <name>-sn-<N>.

Inject secrets

Never commit real secrets. Use ansible-vault to encrypt group_vars/dms.yml:

ansible-vault encrypt inventory/group_vars/dms.yml

Or use the Vault lookup plugin:

dms_ca_bundle_pem: "{{ lookup('hashi_vault', 'secret=fermihdi/dms:ca_bundle_pem') }}"

Enable optional services

Edit inventory/group_vars/optional.yml and set any of the enable_* flags to true:

enable_ingester: true
enable_fe_traefik: true
enable_hd_graphql: true
enable_hd_mcp: true
enable_hd_data_explore: true

Run the deployment

# Full deployment
ansible-playbook -i inventory/hosts.yml site.yml --ask-vault-pass

# Deploy only DMS services
ansible-playbook -i inventory/hosts.yml playbooks/dms.yml

# Deploy only Storage Clusters
ansible-playbook -i inventory/hosts.yml playbooks/sc.yml

# Deploy only optional services
ansible-playbook -i inventory/hosts.yml playbooks/optional.yml

Service Groups

The two front doors, and replacing them

Nothing in HD authenticates a person: the WebUX, the Proxy and the query services trust the X-Forwarded-User, -Email, -Groups and -Role headers on the requests that reach them. Two reverse proxies set those headers, and both are the customer's to provide, configure or replace. The contract they keep — what each authenticates, which services refuse a request without an identity, that the client's own headers must be stripped, the client certificate the Proxy requires — is front_door.md in the manual. This example ships each door in two shapes:

Switch a door to the production example with enable_dms_nginx: true (group_vars/dms.yml) or enable_fe_nginx: true (group_vars/optional.yml, with enable_fe_traefik false), and supply what each role asserts on: the server certificate, the OpenID Connect issuer, client id, client secret and cookie secret, and, for the query door, the client certificate it presents to the Proxy and the CA of the certificates it accepts from services. The roles' defaults/main.yml list every setting. The Proxy is told to trust the front door's client certificate through FERMIHDI_NORTHBOUND_CA_PEM.

Bringing your own ingress — Envoy, Kong, HAProxy, a cloud load balancer, an identity-aware proxy — means reproducing what roles/fe_nginx/templates/nginx.conf.j2 and roles/dms_nginx/templates/nginx.conf.j2 do, whichever mechanism sets the headers: authenticate the caller, decide who may reach what, set the five identity headers from the established identity, discard the ones the client sent, present a client certificate to the Proxy, and keep the DMS door independent of the DMS-CA. Whether that happens through Traefik's ForwardAuth, NGINX's auth_request, Envoy's external authorization or a gateway plugin is immaterial to the services. Then disable both roles for that door and point the ingress at the same upstream ports: the WebUX on 3000, the Proxy on 6986, the SQL server's HTTP listener on 8081, the GraphQL services on 8083 and 8080, the MCP server on 8082 and 8083, Data Explore on 4321.

Network Port Reference

Prerequisites

• Ansible ≥ 2.14 with community.docker collection installed:

  ansible-galaxy collection install community.docker
  

• Docker installed on all target hosts (roles will attempt to install it if absent)
• Hugepages configured on SCC and SN hosts (2Mi hugepages required):

  # /etc/default/grub — add to GRUB_CMDLINE_LINUX:
  # hugepagesz=2M hugepages=1024
  

• SSH access to all target hosts with sudo privileges

DMS-CA Isolation

The DMS-CA must only be reachable from DMS-Core. In this example the DMS-CA runs on the same host as DMS-Core (fermihdi_dms group). Use your firewall (iptables, nftables, security groups) to restrict port 6989 inbound to DMS-Core's IP only.