Deploy on Kubernetes
On this page 9 sections
Deploy your own CDN using the QSCS Docker Hub image. A fresh deployment can run up to two nodes, one substrate and one origin without a Spook Systems account. Kubernetes supplies storage and networking; QSCS enrolls the nodes and authenticates their shared membership.
This guide uses docker.io/spooksystems/qscs:latest. You can express these settings in your own manifests, Helm chart or deployment tool. The native Linux / APT installation remains available.
Before you deploy
- Linux amd64 workers, persistent storage and working DNS resolution.
- Connectivity to your upstream application.
- Outbound HTTPS (TCP 443) to
spooksystems.orgfor enrollment andmonitoring.spooksystems.orgfor telemetry. QSCS supplies both default service addresses. - A separate, stable peer address for every node, reachable by the other members on TCP 4443.
- For public traffic, an ingress controller, a frontend hostname and a suitable TLS certificate.
A substrate is the authenticated boundary within which QSCS shares configuration and eligible state. Pointing independent deployments at the same upstream URL does not join them. No account handle, domain name or UUID is sufficient to authorize membership.
1. Configure the workload and storage
| Setting | Value |
|---|---|
| Image | docker.io/spooksystems/qscs:latest |
| Workload | A StatefulSet with one or two replicas |
| Container command and arguments | Keep the image defaults |
| Security context | runAsUser: 10001, runAsGroup: 10001, and pod-level fsGroup: 10001 |
| Persistent volume | One writable volume per replica, mounted at /var/lib/qscs |
| Storage capacity | Size for the application; 10 GiB per node is an example starting allocation |
| Peer discovery | A headless Service with publishNotReadyAddresses: true and stable per-pod DNS names |
| Application Service | Forward application traffic to TCP 8080 |
| Readiness probe | TCP check on port 8080 |
| Shutdown grace period | terminationGracePeriodSeconds: 90 |
| Image pull policy | Always to resolve latest when a container starts |
The volume holds the node's identity, configuration and saved response state. Keep its association with the same replica across restarts. Never share a writable state directory between running nodes or clone an enrolled volume to create a new identity. The storage driver must support making the mount writable by group 10001; network storage may need ownership configured on the storage server. A read-only container root filesystem also needs a writable temporary mount at /tmp.
See Kubernetes' StatefulSet guide and volume permission guidance for the underlying workload settings.
2. Configure automatic enrollment
Use auto-cluster for a fresh one- or two-node deployment. One named replica initializes the account handle and substrate. The others wait for it and join through the same enrollment service using a private deployment secret. Each receives its own persistent UUID, key and authenticated peer credentials.
Generate a secret once for this deployment:
openssl rand -hex 32
Store the result as a Kubernetes Secret and mount it read-only in each replica. It must contain 64 lowercase hexadecimal characters; a trailing newline is accepted. Use permissions readable by the node user, such as 0440 with group 10001. Keep it out of source control and application responses. It authorizes new enrollment while the deployment's anonymous admission grant remains active; it is not the peer encryption key.
| Variable | Value |
|---|---|
QSCS_BOOTSTRAP_MODE | auto-cluster |
QSCS_INITIAL_REPLICA | The exact initializer pod name, for example qscs-0 |
QSCS_REPLICA_NAME | The current pod's metadata.name, supplied by the downward API |
QSCS_DEPLOYMENT_SECRET_FILE | Absolute path to the mounted secret, for example /run/qscs-bootstrap/deployment-secret |
QSCS_ADVERTISED_PEER | This pod's unique peer hostname, reachable on TCP 4443 |
For a StatefulSet named qscs, a headless Service named qscs-peer and namespace qscs, the relevant environment fragment is:
env:
- name: QSCS_REPLICA_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: QSCS_ADVERTISED_PEER
value: "$(QSCS_REPLICA_NAME).qscs-peer.qscs.svc.cluster.local"
Declare the replica name before the address so Kubernetes expands the reference. Adjust the namespace, Service name and cluster DNS suffix to your environment. Keep the initializer and replica names stable when reusing their volumes.
Start with one replica or two. A joiner that starts first waits instead of creating a second deployment. While the deployment remains unclaimed, scaling from one replica to two enrolls the second node automatically. The initializer is not the only node through which clients can enter the application.
For separately managed workloads, the same shared-secret enrollment also accepts QSCS_BOOTSTRAP_MODE=create on one initializer and QSCS_BOOTSTRAP_MODE=join on the other node, with the same secret file and distinct advertised addresses. These explicit modes do not need the initializer-name variables. Use a new secret for each independent deployment.
3. Set the upstream and frontend
| Variable | Use |
|---|---|
ORIGIN | Upstream HTTP or HTTPS URL, such as https://origin.example.com |
FRONTEND | Required with ORIGIN: the hostname visitors use, such as app.example.com, without a URL scheme |
UPSTREAM | Optional upstream HTTP Host header; defaults to the authority in ORIGIN |
QSCS_INITIAL_CONFIG | Optional absolute path to a read-only initial QSCS configuration file |
ORIGIN may include a port, but not credentials, a path prefix, a query or a fragment. QSCS forwards the incoming request path and query. Changing UPSTREAM changes the HTTP Host header; HTTPS certificate verification and SNI still use the hostname in ORIGIN.
Private upstreams are disabled by default. For an internal application, set AllowPrivateBackends = true in the [QSCS Daemon] section of the initial configuration before first boot. Remember that localhost refers to the pod's network namespace.
Only the initializer provisions the route from these variables. Joining nodes receive the substrate's signed configuration. Before claiming, changing these variables on the initializer updates its route; clearing them does not delete the route. After claiming, manage origins through the control panel.
4. Route traffic and connect peers
Connect an application Service to port 8080 and configure your ingress for FRONTEND, preserving the Host header. Terminate public HTTPS at the ingress. The QSCS container does not need a public port 443 listener for this arrangement.
Allow DNS, outbound HTTPS to the enrollment and monitoring services, access to the upstream, and TCP 4443 between peers in both directions. The advertised peer address must identify one node, not a load-balanced application Service. Cross-cluster deployments need their own routable peer addresses and network connectivity; cluster-local Service DNS does not provide that automatically.
Use your Kubernetes scheduling rules to place replicas in the intended zones or regions, taking storage locality into account. QSCS enrollment does not select Kubernetes locations or provision global routing. Configuring an origin also does not establish ownership of its hostname. Spook-managed DNS and certificates require registration and domain verification.
5. Verify the deployment
The examples below assume namespace qscs and pods qscs-0 and qscs-1. Adapt the names to your workload:
kubectl -n qscs get pods
kubectl -n qscs logs qscs-0
kubectl -n qscs exec qscs-0 -- qscs status
# For a two-node deployment:
kubectl -n qscs exec qscs-1 -- qscs status
Confirm matching account handles and substrate IDs, distinct UUIDs and the expected peer membership. Listener readiness is not proof that every saved response has replicated. Check replication separately after requesting representative application pages.
Test through your ingress, or keep this port-forward running in one terminal:
kubectl -n qscs port-forward pod/qscs-0 8080:8080
Then request the configured frontend from another terminal:
curl -i -H "Host: app.example.com" http://127.0.0.1:8080/
Confirm the application's expected response. HTTP 421 usually means the request Host does not match the configured frontend. HEADLESS operation needs eligible content already retained by the serving node; QSCS does not replicate the application's database.
Claim, license and add more nodes
You can keep operating within the free allowance without registration. To manage the deployment in the portal, follow Claim an existing deployment. Use the account handle from qscs status, approve the portal challenge on the unclaimed administrator node, and confirm the whole deployment. In a container shell use qscs without sudo; administrator membership is what authorizes the approval.
Claiming preserves node identities, configuration, substrate membership and retained monitoring data. It also closes admission through the old deployment secret and invalidates unused pre-claim invitations. Keep existing bootstrap inputs and volumes for restarts, but use a fresh administrator-issued invitation for each new node after claiming. Increasing the original replica count does not enroll fresh identities after the claim.
For an invitation-based member, use QSCS_BOOTSTRAP_MODE=join, QSCS_INVITE_SOURCE pointing to the read-only mounted invitation JSON, and its own QSCS_ADVERTISED_PEER. Omit the deployment secret, initializer variables and environment-based origin provisioning. Give the member its own workload definition and volume. No separate substrate-name setting or second claim is needed.
Confirm available account capacity before joining. The registered account's allowance applies across its substrates and handles; claiming does not add another free allocation. Upgrade through Billing when you need more capacity. An ordinary invitation lasts ten minutes and admits one node; issuing a new ordinary invitation replaces previous unused invitations for that substrate.
Monitoring, upgrades and removal
Nodes report operational telemetry from first enrollment. Unclaimed reports have a 30-day retention period and include identity, version, health and operational counters, excluding request bodies, credentials, visitor addresses and request URLs. After claiming, retained monitoring data is available in SpookVis. If monitoring is unreachable, reporting retries in the background without blocking application serving; enrollment and master-lease renewal have separate requirements.
latest is resolved when Kubernetes starts a container with imagePullPolicy: Always; an image publication does not restart existing pods. Use a controlled update, preserve PVCs and allow graceful shutdown. Record the image digest before upgrading so you can select a compatible previous image if needed. See container upgrades.
Stopping a pod does not release its license or necessarily delete its volume. To retire a claimed node, stop it and revoke its license in the control panel. Review PVC and storage retention before deleting workloads. Deleting state discards that node's identity and cache; reinstalling is not a way to reset the account's allowance.
Troubleshooting
| Symptom | Check |
|---|---|
| Waiting for the initializer | Exact initializer pod name, matching secret and enrollment connectivity |
deployment_already_exists | Restore the original initializer volume; the secret is already associated with a deployment |
deployment_unavailable | The anonymous grant was claimed, revoked or invalidated; use a fresh invitation for a post-claim addition |
node_limit | Available account allowance, including stopped nodes whose licenses remain active |
administrator_required | Run the action on the administrator of the correct substrate; root privileges do not grant that role |
Permission denied under /var/lib/qscs | Volume mount, UID/GID 10001 and storage support for group ownership |
| Peers cannot connect | Per-node DNS, bidirectional TCP 4443 and network policies; do not advertise the shared application Service |