Layer In place How it was tested
Edge OPNsense firewall with Suricata IDS on the WAN. Two front-door gateways on different servers share one virtual IP. Tested: Failover drilled, and fixed until it completes in 13 seconds or less. Jul 2026
Network The cluster has no route to the internet, blocked at the gateway. Apps built on the standard chart run default-deny under Cilium. Tested: A probe from inside an app pod reached its own backend and nothing else: not Vault, not another app, not the Kubernetes API. Sep 2026
Identity and TLS Keycloak single sign-on for every service, per-namespace RBAC, and an internal CA issuing TLS all the way to the pod. Tested: RBAC smoke tests. A hostile tenant probe found the real tenancy boundary is the Argo CD project, not the namespace: alone, a namespace stops nothing, because a tenant pull request could name any destination and the platform would deploy it there. Fixed, proven, and torn down. Jul–Oct 2026
Secrets HA Vault, with the unseal key on a separate transit VM and the Shamir keys kept offline. External Secrets delivers secrets to pods so none live in git, and gitleaks fails any build that tries. Tested: Vault raft snapshots are in the encrypted nightly backups and have been restored in a test. After the total power loss, Vault was unsealed by hand with the offline keys, as designed. Jul–Aug 2026
Supply chain Every image is mirrored internally and pinned by digest. Trivy gates CI and scans the registry into Prometheus on a schedule. Tested: An agent reported an 80% drop in critical CVEs. The rescan said 28%, and 28% is the number on record. The purple-team exercise found an expired registry certificate blocking every image pull, renewed from the internal CA the same night. Aug 2026 · public record → Workloads Falco watches syscalls on the nodes, and Talon quarantines a pod automatically when a rule fires. Pod Security labels report baseline violations. Tested: Purple-team exercise: an agent dropped and executed a binary in a test pod, Falco caught it, and the pod was isolated. Running the full chain exposed that the response path had been silently dead since a policy port mismatch, fixed and re-verified that night. The detection battery that followed found five Talon rules referencing Falco rule names absent from the deployed 0.44.1 ruleset: four restored from upstream and committed, one excluded on a recorded reason. Oct 2026
Hosts Kernel patching by draining each hypervisor and live-migrating its VMs back. Journals from eight hosts go to Loki, and auditd, fail2ban and AIDE run on most of them. Tested: Patch campaign across all five hypervisors with 22 of 22 VMs and 6 of 6 Kubernetes nodes up throughout. Jul 2026
Agents and automation Infrastructure credentials live on one host. Terraform runs as an identity limited to templates and networks, agents get one Vault subtree on ten-minute tokens, every change is a pull request, and Workbench holds anything that reaches outside a thread’s worktree. Tested: A sweep of the estate found no infrastructure credentials off that host apart from one documented read-only identity. Building the Terraform identity turned up a path to administrator in the first design, which was closed and probed. Aug 2026
Recovery Backups encrypted with restic before they leave the network, kept locally and in an append-only offsite archive, and restore-tested. A cold-start runbook brings everything up in dependency order. Tested: A total power loss came back in about 1h50m with no data loss. Jul 2026 · public record →