DocumentationBuild. Deploy. Operate.
DocsDeployment

Deploy on Kubernetes

On this page 9 sections

Deploy your own CDN using the QSCS Docker Hub image. A fresh deployment can run up to two nodes, one substrate and one origin without a Spook Systems account. Kubernetes supplies storage and networking; QSCS enrolls the nodes and authenticates their shared membership.

This guide uses docker.io/spooksystems/qscs:latest. You can express these settings in your own manifests, Helm chart or deployment tool. The native Linux / APT installation remains available.

Before you deploy

  • Linux amd64 workers, persistent storage and working DNS resolution.
  • Connectivity to your upstream application.
  • Outbound HTTPS (TCP 443) to spooksystems.org for enrollment and monitoring.spooksystems.org for telemetry. QSCS supplies both default service addresses.
  • A separate, stable peer address for every node, reachable by the other members on TCP 4443.
  • For public traffic, an ingress controller, a frontend hostname and a suitable TLS certificate.

A substrate is the authenticated boundary within which QSCS shares configuration and eligible state. Pointing independent deployments at the same upstream URL does not join them. No account handle, domain name or UUID is sufficient to authorize membership.

1. Configure the workload and storage

SettingValue
Imagedocker.io/spooksystems/qscs:latest
WorkloadA StatefulSet with one or two replicas
Container command and argumentsKeep the image defaults
Security contextrunAsUser: 10001, runAsGroup: 10001, and pod-level fsGroup: 10001
Persistent volumeOne writable volume per replica, mounted at /var/lib/qscs
Storage capacitySize for the application; 10 GiB per node is an example starting allocation
Peer discoveryA headless Service with publishNotReadyAddresses: true and stable per-pod DNS names
Application ServiceForward application traffic to TCP 8080
Readiness probeTCP check on port 8080
Shutdown grace periodterminationGracePeriodSeconds: 90
Image pull policyAlways to resolve latest when a container starts

The volume holds the node's identity, configuration and saved response state. Keep its association with the same replica across restarts. Never share a writable state directory between running nodes or clone an enrolled volume to create a new identity. The storage driver must support making the mount writable by group 10001; network storage may need ownership configured on the storage server. A read-only container root filesystem also needs a writable temporary mount at /tmp.

See Kubernetes' StatefulSet guide and volume permission guidance for the underlying workload settings.

2. Configure automatic enrollment

Use auto-cluster for a fresh one- or two-node deployment. One named replica initializes the account handle and substrate. The others wait for it and join through the same enrollment service using a private deployment secret. Each receives its own persistent UUID, key and authenticated peer credentials.

Generate a secret once for this deployment:

openssl rand -hex 32

Store the result as a Kubernetes Secret and mount it read-only in each replica. It must contain 64 lowercase hexadecimal characters; a trailing newline is accepted. Use permissions readable by the node user, such as 0440 with group 10001. Keep it out of source control and application responses. It authorizes new enrollment while the deployment's anonymous admission grant remains active; it is not the peer encryption key.

VariableValue
QSCS_BOOTSTRAP_MODEauto-cluster
QSCS_INITIAL_REPLICAThe exact initializer pod name, for example qscs-0
QSCS_REPLICA_NAMEThe current pod's metadata.name, supplied by the downward API
QSCS_DEPLOYMENT_SECRET_FILEAbsolute path to the mounted secret, for example /run/qscs-bootstrap/deployment-secret
QSCS_ADVERTISED_PEERThis pod's unique peer hostname, reachable on TCP 4443

For a StatefulSet named qscs, a headless Service named qscs-peer and namespace qscs, the relevant environment fragment is:

env:
  - name: QSCS_REPLICA_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
  - name: QSCS_ADVERTISED_PEER
    value: "$(QSCS_REPLICA_NAME).qscs-peer.qscs.svc.cluster.local"

Declare the replica name before the address so Kubernetes expands the reference. Adjust the namespace, Service name and cluster DNS suffix to your environment. Keep the initializer and replica names stable when reusing their volumes.

Start with one replica or two. A joiner that starts first waits instead of creating a second deployment. While the deployment remains unclaimed, scaling from one replica to two enrolls the second node automatically. The initializer is not the only node through which clients can enter the application.

For separately managed workloads, the same shared-secret enrollment also accepts QSCS_BOOTSTRAP_MODE=create on one initializer and QSCS_BOOTSTRAP_MODE=join on the other node, with the same secret file and distinct advertised addresses. These explicit modes do not need the initializer-name variables. Use a new secret for each independent deployment.

3. Set the upstream and frontend

VariableUse
ORIGINUpstream HTTP or HTTPS URL, such as https://origin.example.com
FRONTENDRequired with ORIGIN: the hostname visitors use, such as app.example.com, without a URL scheme
UPSTREAMOptional upstream HTTP Host header; defaults to the authority in ORIGIN
QSCS_INITIAL_CONFIGOptional absolute path to a read-only initial QSCS configuration file

ORIGIN may include a port, but not credentials, a path prefix, a query or a fragment. QSCS forwards the incoming request path and query. Changing UPSTREAM changes the HTTP Host header; HTTPS certificate verification and SNI still use the hostname in ORIGIN.

Private upstreams are disabled by default. For an internal application, set AllowPrivateBackends = true in the [QSCS Daemon] section of the initial configuration before first boot. Remember that localhost refers to the pod's network namespace.

Only the initializer provisions the route from these variables. Joining nodes receive the substrate's signed configuration. Before claiming, changing these variables on the initializer updates its route; clearing them does not delete the route. After claiming, manage origins through the control panel.

4. Route traffic and connect peers

Connect an application Service to port 8080 and configure your ingress for FRONTEND, preserving the Host header. Terminate public HTTPS at the ingress. The QSCS container does not need a public port 443 listener for this arrangement.

Allow DNS, outbound HTTPS to the enrollment and monitoring services, access to the upstream, and TCP 4443 between peers in both directions. The advertised peer address must identify one node, not a load-balanced application Service. Cross-cluster deployments need their own routable peer addresses and network connectivity; cluster-local Service DNS does not provide that automatically.

Use your Kubernetes scheduling rules to place replicas in the intended zones or regions, taking storage locality into account. QSCS enrollment does not select Kubernetes locations or provision global routing. Configuring an origin also does not establish ownership of its hostname. Spook-managed DNS and certificates require registration and domain verification.

5. Verify the deployment

The examples below assume namespace qscs and pods qscs-0 and qscs-1. Adapt the names to your workload:

kubectl -n qscs get pods
kubectl -n qscs logs qscs-0
kubectl -n qscs exec qscs-0 -- qscs status
# For a two-node deployment:
kubectl -n qscs exec qscs-1 -- qscs status

Confirm matching account handles and substrate IDs, distinct UUIDs and the expected peer membership. Listener readiness is not proof that every saved response has replicated. Check replication separately after requesting representative application pages.

Test through your ingress, or keep this port-forward running in one terminal:

kubectl -n qscs port-forward pod/qscs-0 8080:8080

Then request the configured frontend from another terminal:

curl -i -H "Host: app.example.com" http://127.0.0.1:8080/

Confirm the application's expected response. HTTP 421 usually means the request Host does not match the configured frontend. HEADLESS operation needs eligible content already retained by the serving node; QSCS does not replicate the application's database.

Claim, license and add more nodes

You can keep operating within the free allowance without registration. To manage the deployment in the portal, follow Claim an existing deployment. Use the account handle from qscs status, approve the portal challenge on the unclaimed administrator node, and confirm the whole deployment. In a container shell use qscs without sudo; administrator membership is what authorizes the approval.

Claiming preserves node identities, configuration, substrate membership and retained monitoring data. It also closes admission through the old deployment secret and invalidates unused pre-claim invitations. Keep existing bootstrap inputs and volumes for restarts, but use a fresh administrator-issued invitation for each new node after claiming. Increasing the original replica count does not enroll fresh identities after the claim.

For an invitation-based member, use QSCS_BOOTSTRAP_MODE=join, QSCS_INVITE_SOURCE pointing to the read-only mounted invitation JSON, and its own QSCS_ADVERTISED_PEER. Omit the deployment secret, initializer variables and environment-based origin provisioning. Give the member its own workload definition and volume. No separate substrate-name setting or second claim is needed.

Confirm available account capacity before joining. The registered account's allowance applies across its substrates and handles; claiming does not add another free allocation. Upgrade through Billing when you need more capacity. An ordinary invitation lasts ten minutes and admits one node; issuing a new ordinary invitation replaces previous unused invitations for that substrate.

Monitoring, upgrades and removal

Nodes report operational telemetry from first enrollment. Unclaimed reports have a 30-day retention period and include identity, version, health and operational counters, excluding request bodies, credentials, visitor addresses and request URLs. After claiming, retained monitoring data is available in SpookVis. If monitoring is unreachable, reporting retries in the background without blocking application serving; enrollment and master-lease renewal have separate requirements.

latest is resolved when Kubernetes starts a container with imagePullPolicy: Always; an image publication does not restart existing pods. Use a controlled update, preserve PVCs and allow graceful shutdown. Record the image digest before upgrading so you can select a compatible previous image if needed. See container upgrades.

Stopping a pod does not release its license or necessarily delete its volume. To retire a claimed node, stop it and revoke its license in the control panel. Review PVC and storage retention before deleting workloads. Deleting state discards that node's identity and cache; reinstalling is not a way to reset the account's allowance.

Troubleshooting

SymptomCheck
Waiting for the initializerExact initializer pod name, matching secret and enrollment connectivity
deployment_already_existsRestore the original initializer volume; the secret is already associated with a deployment
deployment_unavailableThe anonymous grant was claimed, revoked or invalidated; use a fresh invitation for a post-claim addition
node_limitAvailable account allowance, including stopped nodes whose licenses remain active
administrator_requiredRun the action on the administrator of the correct substrate; root privileges do not grant that role
Permission denied under /var/lib/qscsVolume mount, UID/GID 10001 and storage support for group ownership
Peers cannot connectPer-node DNS, bidirectional TCP 4443 and network policies; do not advertise the shared application Service
Need a hand with your deployment?Contact support ↗Back to top ↑