HD — Puppet Deployment Example

This directory contains a reference Puppet module for deploying a HD instance and its supporting services using Docker containers managed by the puppetlabs/docker module.

Note: These manifests are examples. They are derived from the sandbox Docker Compose configuration and the docs/k8s_example Helm chart. Adapt them to your environment's secrets management, certificate infrastructure, and node classification strategy before production use.

Directory Structure

puppet_example/
├── README.md                          # This file
├── hiera.yaml                         # Hiera 5 configuration
├── manifests/
│   └── site.pp                        # Node classification example
├── modules/
│   └── fermihdi/
│       └── manifests/
│           ├── init.pp                # Main class (optional services gate)
│           ├── dms_ca.pp              # DMS-CA
│           ├── dms_core.pp            # DMS-Core
│           ├── dms_webux.pp           # DMS-WebUX
│           ├── dms_opa_engine.pp      # DMS-OPA-Engine
│           ├── dms_traefik.pp         # DMS-internal Traefik
│           ├── otel_collector.pp      # OTEL Collector
│           ├── sentry_relay.pp        # Sentry Relay
│           ├── sc_scc.pp              # Storage Cluster Controller
│           ├── sc_sn.pp               # Storage Node
│           ├── hd_proxy.pp            # HD Proxy
│           ├── hd_ingester.pp         # [optional]
│           ├── fe_traefik.pp          # [optional]
│           ├── hd_mcp.pp              # [optional]
│           ├── hd_graphql.pp          # [optional]
│           └── hd_data_explore.pp     # [optional]
└── data/
    ├── common.yaml                    # Hiera global defaults
    └── hosts/
        ├── sc1-scc.example.com.yaml   # Per-host SCC params example
        └── sc1-sn-01.example.com.yaml # Per-host SN params example

How Storage Clusters Work

The fermihdi::sc_scc and fermihdi::sc_sn classes get their SC-specific parameters from per-host Hiera data (in data/hosts/<certname>.yaml), not from a shared list. This is idiomatic Puppet: each node knows only about itself.

SCC host (data/hosts/sc1-scc.example.com.yaml):

fermihdi::sc_scc::sc_name:  "sc-1"
fermihdi::sc_scc::scc_token: "ENC[PKCS7,...]"   # hiera-eyaml encrypted

The class derives FERMIHDI_NODE_NAME=sc-1-scc automatically.

SN host (data/hosts/sc1-sn-01.example.com.yaml):

fermihdi::sc_sn::sc_name:  "sc-1"
fermihdi::sc_sn::sn_index: 1        # 1-based position in this SC
fermihdi::sc_sn::sn_token: "ENC[PKCS7,...]"

The class derives FERMIHDI_NODE_NAME=sc-1-sn-1 automatically.

To add more Storage Clusters: create additional per-host Hiera files with the appropriate sc_name, sn_index, and token for each SCC/SN host, then add the corresponding node blocks to site.pp.

Quick Start

1. Install the puppetlabs/docker module

puppet module install puppetlabs-docker --version '>=7.0.0'

Or declare it in a Puppetfile if using r10k:

mod 'puppetlabs/docker', '>= 7.0.0'

2. Configure Hiera

Edit data/common.yaml:

• Set fermihdi::image_tag, fermihdi::dms_host, and all network CIDRs.
• Inject real secrets via hiera-eyaml (see below).

3. Configure per-host Hiera

For each SCC host, create data/hosts/<certname>.yaml with:

fermihdi::sc_scc::sc_name:  "sc-N"
fermihdi::sc_scc::scc_token: "ENC[PKCS7,...]"

For each SN host:

fermihdi::sc_sn::sc_name:  "sc-N"
fermihdi::sc_sn::sn_index: <1-based index>
fermihdi::sc_sn::sn_token: "ENC[PKCS7,...]"

4. Classify nodes

Edit manifests/site.pp with your actual node names, or use your ENC.

5. Enable optional services

In the optional host's Hiera file (data/hosts/<certname>.yaml):

fermihdi::optional::enable_ingester:        true
fermihdi::optional::enable_fe_traefik:      true
fermihdi::optional::enable_hd_graphql:      true
fermihdi::optional::enable_hd_mcp:          true
fermihdi::optional::enable_hd_data_explore: true

6. Apply

# On each node:
puppet agent --test

# Or from Puppet Server:
puppet job run --nodes sc1-scc.example.com,sc1-sn-01.example.com,...

Secret Injection (hiera-eyaml)

# Install hiera-eyaml
gem install hiera-eyaml

# Generate key pair (run once on Puppet Server)
eyaml createkeys

# Encrypt a secret
eyaml encrypt -s 'my-scc-bootstrap-token'
# → ENC[PKCS7,...]

# Use the encrypted value in Hiera YAML:
# fermihdi::sc_scc::scc_token: "ENC[PKCS7,...]"

Network Port Reference

Prerequisites

• Puppet Agent ≥ 7 on all target nodes
• puppetlabs/docker module ≥ 7.0.0 installed on Puppet Server
• Docker installed on all target nodes (puppetlabs/docker can manage this)
• Hugepages configured on SCC and SN hosts
• hiera-eyaml configured on Puppet Server for secret encryption