Installation
nauthera-server has a Helm chart at deploy/helm/nauthera-server in its repository.
It deploys the server with cert-manager issuing every certificate and key it needs,
including the JWT signing keys.
Before you start. No image or chart has been published yet (#307). The chart's default image is
ghcr.io/nauthera/nauthera-server:0.1.0, and pulling it fails until a release pushes it. For now, build the image from source (the kind recipes below do this for you) or push it to a registry you control and setimage.repositoryandimage.digest.
Requirements
- Kubernetes 1.27 or newer
- Helm 3 or 4
- cert-manager, when
certManager.enabledis on (it is in every profile) - The Prometheus Operator CRDs, if you enable
metrics.serviceMonitorormetrics.prometheusRule - PostgreSQL, and a runtime role created beforehand, for
appendOnlyAudit
Local: kind
From a checkout of nauthera-server:
just kind-up # kind cluster, cert-manager, image built and loaded, chart installed
just kind-check # discovery, JWKS and the sign-in page really answerkind-up uses profiles/kind.yaml. That profile runs one replica with SQLite on a PVC,
in-pod TLS from the chart's own CA, no ingress controller, and a NodePort that kind
publishes on https://localhost:8443. Expect a certificate warning in the browser,
because the CA lives inside the cluster. Tear it down with just kind-down.
Production
profiles/production.yaml is the production shape: three replicas behind an ingress,
PostgreSQL, Redis over TLS, file-backed keys, a PodDisruptionBudget, a NetworkPolicy, and
a ServiceMonitor with alerting rules. Fill in the host and the Secret names:
# my-values.yaml
image:
repository: registry.example.com/nauthera-server
digest: sha256:… # pin a digest, not a tag
ingress:
host: auth.example.com
certManager:
issuerRef:
name: letsencrypt # governs the ingress certificate only
kind: ClusterIssuer
config:
issuer:
url: https://auth.example.com
cache:
mode: sentinel
addresses: ["redis-sentinel-0:26379", "redis-sentinel-1:26379", "redis-sentinel-2:26379"]
master_name: mymaster
security:
trusted_proxies: ["10.244.1.7/32"] # your ingress pods, not a whole range
client_ip:
mode: x-forwarded-for
secrets:
databaseDsn:
existingSecret: nauthera-pg-app # e.g. the Secret CloudNativePG publishes
existingSecretKey: uri
cachePassword:
existingSecret: nauthera-redis
mfaEncryptionKey:
existingSecret: nauthera-keys
keysEncryptionKey:
existingSecret: nauthera-keysThe Redis behind config.cache must not evict keys (maxmemory-policy noeviction; the
server refuses to start otherwise), should persist with AOF, and should be replicated:
cache.mode sentinel or cluster. The production profile ships a single address;
replace it as above. See Backups & Upgrades.
helm upgrade --install nauthera deploy/helm/nauthera-server \
--namespace nauthera --create-namespace \
-f deploy/helm/nauthera-server/profiles/production.yaml \
-f my-values.yamlWhat the chart checks for you
Most of the server's startup checks hinge on one value, the issuer URL. It must be
https with a host and no path. If the host is not loopback (localhost and
*.localhost count as loopback), the server also requires a shared cache and file-backed
signing and MFA keys. The chart runs the same checks when it renders templates, so a bad
combination fails helm install with the fix attached:
$ helm install nauthera . --set config.issuer.url=https://auth.example.com
Error: nauthera-server: issuer "https://auth.example.com" is a real, fronted
deployment (host "auth.example.com" is not loopback), and the server refuses to
start in this configuration:
* cache.enabled is false. Authorization codes and refresh tokens would fall
back to process memory, so a code issued by the replica that served
/authorize does not exist for the replica that serves /token …
Fix: set config.cache.enabled=true and config.cache.address.
…The chart also refuses a trusted_proxies entry that covers every address, a
client_ip.mode other than socket without trusted_proxies, SQLite or a
ReadWriteOnce volume with more than one replica, rate_limit.backend: redis without a
cache, and the gRPC listener without either mTLS or allow_insecure.
values.schema.json rejects unknown keys in the chart's own settings and in the whole
config tree. Kubernetes objects it passes through unchanged, such as resources,
affinity and the security contexts, are not checked. The schema's config half is
generated from the server's Go configuration structs, so it cannot drift from what the
server reads.
Configuration and secrets
configmirrors the server's configuration tree one-to-one. It is rendered to a YAML file and mounted read-only. Set only what you mean to change. See Configuration.- Nine keys under
secretshold the server's secret material:databaseDsn,databasePassword,cachePassword,cacheSentinelPassword,mfaEncryptionKey,keysEncryptionKey,smtpPassword,mfaRetiredKeysandkeysRetiredKeys. A tenth,ownerDatabaseDsn, is used only by the database-maintenance Job and CronJob (below). Each takes either an inlinevalue(development only) or anexistingSecretplusexistingSecretKey. secrets.delivery: file(the default) mounts seven of them as files (0440) and setsNAUTHERA_SERVER_<KEY>_FILE. The two retired-key lists always arrive as environment variables throughsecretKeyRef, because the server cannot read them from a file yet (#340).envuses one explicitsecretKeyRefper variable.
Certificates
| Certificate | Purpose | Issued by |
|---|---|---|
| CA | Self-signed CA in the release namespace | itself |
| Signing key (EC P-256) | JWT signing key (ES256) | the chart's CA |
| Signing key (RSA) | RS256 key, which OIDC Discovery requires to be advertised | the chart's CA |
| Ingress | Browser-facing certificate | certManager.issuerRef, or the chart's CA |
| HTTP (optional) | In-pod HTTPS | the chart's CA |
| gRPC server and client (optional) | mTLS for the AdminService | a CA of the control plane's own |
The signing-key certificates use rotationPolicy: Never, because re-keying would
invalidate every token in flight. The server reads its TLS certificates once at startup,
so restart the pods after cert-manager renews an in-pod or gRPC certificate. Reloading
without a restart is tracked in #443 for the gRPC certificate and client CA, and in #329
for the in-pod HTTPS certificate. The gRPC client certificate lives for 24 hours by
default (certManager.grpcTLS.client.duration); the operator re-reads it on every call,
so a short lifetime costs nothing.
Sign-in alerts, password resets, email verification and invitations need an outbound
relay. Configure the deployment's own under config.mail (smtp.host, smtp.port,
smtp.tls, from.address) with the password in secrets.smtpPassword. An organization
can send through a relay of its own, listed under mailOrganizations:
mailOrganizations:
- slug: acme
smtp:
host: smtp.acme.example
port: 587
tls: starttls # starttls | implicit | none
from:
address: no-reply@acme.example
name: Acme
credentials:
existingSecret: acme-smtp # keys username and passwordThe credentials are read from an existing Secret, mounted as files and re-read at every
delivery, so rotating the Secret needs no upgrade or restart. They never reach the
ConfigMap or the database. The connection uses TLS (starttls on 587, implicit on
465). none sends in the clear; with credentials it also needs
smtp.allowPlaintextAuth: true, and the server warns about it at startup.
IP geolocation
A new-sign-in alert can name the city and country a sign-in came from. Mount a
GeoLite2-City or GeoIP2-City database through geoip.volume (any volume source, such as
a PVC that geoipupdate writes to) and name the file in geoip.fileName. The server
re-reads the file when it changes.
Alerting
metrics.prometheusRule.enabled renders the server's alerting rules as a
PrometheusRule. They come in groups that can be switched off one by one under
metrics.prometheusRule.rules: availability, errors, latency, dependencies,
audit, security, logout, controlPlane, certificates and kubernetes (the only
one that reads kube-state-metrics). Leave out single alerts under disabled, and tune an
alert's severity, for and threshold under alerts:
metrics:
prometheusRule:
enabled: true
labels: {release: kube-prometheus-stack}
additionalRuleLabels: {team: identity}
runbookUrl: https://runbooks.example.com/nauthera
rules: {kubernetes: false}
disabled: {NautheraRateLimitingHigh: true}
alerts:
NautheraServerHighLatency: {threshold: 0.5, for: 30m}The error-budget alerts burn against availabilityObjective (99.9 by default). Every
rule selects the server's series with job="<fullname>" and the release namespace, as
the chart's ServiceMonitor produces; set selector when Prometheus scrapes the server
another way. The chart README lists every alert and its default.
Migrations and database maintenance
By default the replicas maintain the schema themselves: set config.database.migrate: true and they migrate on start. On PostgreSQL the runner takes a session-level advisory
lock, so replicas run migrations one after another instead of racing. Every replica also
creates and drops audit partitions. All of this needs the role that owns the tables, so
the server's credential can also rewrite the audit log.
On PostgreSQL, appendOnlyAudit moves that work and that credential out of the server's
pods:
appendOnlyAudit:
enabled: true
runtimeRole: nauthera_runtime # create it once, before the first install
secrets:
databaseDsn: # the server, logging in as nauthera_runtime
existingSecret: nauthera-db-runtime
existingSecretKey: uri
ownerDatabaseDsn: # the maintenance Job and CronJob: the owning role
existingSecret: nauthera-db-owner
existingSecretKey: uriA pre-install,pre-upgrade hook Job runs nauthera-server maintain before every install
and upgrade, and a CronJob runs it on appendOnlyAudit.maintenance.schedule (daily at
03:17 UTC by default). Only those pods hold ownerDatabaseDsn. Each run grants the
runtime role read and write on every table but the audit log, and only SELECT and
INSERT on that. Remove config.database.migrate: true from the production profile when
you turn it on; the chart refuses both together. Never give the runtime role table
privileges schema-wide or through ALTER DEFAULT PRIVILEGES: that would reach the audit
partitions. The chart README has the full setup, including CloudNativePG.
Helm waits for the hook Job within --timeout. A release whose migration adds an audit
index takes much longer on a large log; raise --timeout and
appendOnlyAudit.maintenance.activeDeadlineSeconds together for it. See
Backups & Upgrades.
The gRPC AdminService
The production profile turns the gRPC listener off
(config.server.grpc.enabled: false), because nothing in it calls the AdminService.
To use it, set config.server.grpc.enabled: true and certManager.grpcTLS.enabled: true.
The chart issues the gRPC server and client certificates from a CA of the control
plane's own, even when certManager.issuerRef names an ACME issuer, because mTLS needs
ca.crt in the issued Secret and an ACME issuer does not fill it in.
certManager.grpcTLS.allowedClients lists who may call the AdminService and what each
may call. An identity is typed by the client-certificate SAN it matches, dns:<name> or
uri:<uri>; the common name never matches. Operations are organizations, clients
and brandings. The default admits the chart's own client certificate:
certManager:
grpcTLS:
enabled: true
allowedClients:
- identity: dns:nauthera-operator
operations: [clients, brandings]The chart refuses to render gRPC without either mTLS or an explicit allow_insecure. The
server honours allow_insecure only on a loopback address, so the chart then binds the
listener to 127.0.0.1 inside the pod, reachable only through kubectl port-forward, as
in the kind profile.
nauthera-cli presents a client certificate when given --tls-ca-file,
--tls-cert-file and --tls-key-file; without them it connects without TLS, which only
an allow_insecure listener answers.
The operator
operator.enabled: true deploys the Kubernetes operator from the
same release, with its permissions and the OIDCClient and Branding resource
definitions. It needs certManager.grpcTLS.enabled and, by default, uses the client
certificate the chart issues:
helm upgrade --install nauthera deploy/helm/nauthera-server \
-f deploy/helm/nauthera-server/profiles/production.yaml \
--set ingress.host=auth.example.com \
--set config.server.grpc.enabled=true \
--set certManager.grpcTLS.enabled=true \
--set operator.enabled=true \
--set operator.networkPolicy.enabled=true \
--set 'operator.networkPolicy.apiServer.cidrs={10.96.0.1/32}'profiles/operator.yaml is the kind profile with the operator on, and
profiles/operator-cilium.yaml the same with a CiliumNetworkPolicy. See
Operator for its values.
Exposure
The primary Service is always ClusterIP. If you set service.type to anything else,
the chart adds a second Service that carries only the public HTTP port. That keeps
the admin listener and the AdminService (which can create clients and mint their
secrets) off the network. A headless Service carries the gRPC control plane.
Without Helm
Plain manifests live in deploy/k8s/: a Namespace, the ConfigMaps, the Deployment, the
Services, a NetworkPolicy, a PodDisruptionBudget, a ServiceMonitor, a PrometheusRule, an
example Secret and migration-job.yaml, a one-shot Job that runs
nauthera-server maintain before a rollout. They run the AdminService with mTLS, from a
Secret nauthera-server-grpc-tls holding tls.crt, tls.key and ca.crt that you
issue yourself.
The image is FROM scratch, runs as non-root (UID 65532), and is built for
linux/amd64 and linux/arm64. Wire probes to the admin port (/healthz, /readyz,
/startupz), as the chart does. grpc.health.v1 shares the AdminService listener on
:9091, which takes mTLS (or, with allow_insecure, listens on loopback only), so the
kubelet's native gRPC probe cannot reach it. Set GOMEMLIMIT from the container's memory
limit.