Skip to main content

Running in production

Whatever hosts your manifest — systemd, Docker Compose, Kubernetes, ECS — it is supervising one telo <manifest> process. This page describes how that process behaves: when it stays up, how it stops, what it exits with, and what you can probe and configure from outside.

Process lifecycle​

telo ./manifest.yaml runs four phases and then waits:

  1. Load — parse the manifest and its imports, resolve controllers, bind variables: / secrets: / ports: from the host environment.
  2. Init — the multi-pass loop constructs every resource in dependency order. init() performs no observable side effects.
  3. Run targets — everything listed in targets: is dispatched.
  4. Wait for idle — the process stays alive while any hold is outstanding, then tears down and exits.

A hold is what a long-lived resource takes to keep the app up. Http.Server acquires one when it starts listening and releases it on teardown, which is why an application whose target is a server stays up until it is signalled, while one whose targets are all Telo.Runnable runs to completion and exits on its own. Nothing needs to be configured for either shape — it follows from what the manifest declares.

Shutdown and signals​

SIGINT and SIGTERM are both handled, identically:

  1. The boot run is cancelled with reason interrupted. Targets that have not started are refused at the dispatch gate; long-lived runnables and in-flight invoke trees observe ctx.cancellation and stop early.
  2. The idle wait is released even though holds are still outstanding.
  3. Teardown cascades: child contexts first, then each resource in reverse order, with log sinks pinned last so anything logged during shutdown is still flushed. Tearing a resource down runs the inverses its init() and run() returned, newest first; one that refuses is recorded and the cascade continues — a single failure cannot abandon the rest.
  4. The process exits with the exit code the run accumulated.

Two properties matter when you configure a supervisor:

  • There is no shutdown timeout. Teardown runs to completion. Give the orchestrator a grace period longer than your slowest teardown (terminationGracePeriodSeconds on Kubernetes, stop_grace_period in Compose) or it will SIGKILL mid-drain.
  • The handlers fire once. A second SIGTERM is not caught, so it kills the process immediately with Node's default behaviour. That is the intended escape hatch for a teardown that hangs.

Cancellation is cooperative: a controller that never checks its cancellation token runs to its natural end. See Invoke cancellation.

Exit codes​

CodeMeaning
0The app went idle (or was signalled) and tore down cleanly.
1Load, init, or a target run failed — the error is printed with the manifest location that caused it. Also the code a failed assertion sets.
nWhatever a controller passed to ctx.requestExit(n). The highest requested code wins; Assert.* uses 1.

An init failure is reported by root cause, not by count: the kernel classifies which resources actually failed and which were merely blocked behind them, so the message names the one you have to fix. See Resource lifecycle.

Health checks​

Telo has no built-in probe endpoint — a health check is a route you declare, which is what lets it mean something specific to your app.

Readiness is already implied by the socket. Http.Server binds its port in run(), after every resource has initialized. A connection is refused until init has completed, so a TCP or HTTP probe against the port is a truthful readiness signal with nothing extra declared.

For an HTTP probe endpoint, declare a route like any other:

kind: Telo.Application
metadata:
  name: MyApp
  version: 1.0.0
imports:
  Http: oci://ghcr.io/telorun/http-server@0.38.0
  Run: oci://ghcr.io/telorun/run@0.28.0
targets:
  - !ref Server
ports:
  http:
    env: PORT
    default: 8080
---
kind: Http.Server
metadata:
  name: Server
listen:
  port: !cel "ports.http"
mounts:
  - path: /
    mount: !ref Health
---
kind: Http.Api
metadata:
  name: Health
routes:
  - request:
      path: /healthz
      method: GET
    handler: !ref HealthValue
    returns:
      - status: 200
        content:
          application/json:
            schema:
              type: object
              properties:
                status: { type: string }
              required: [status]
            body:
              status: !cel "result.status"
---
kind: Run.Value
metadata:
  name: HealthValue
value:
  status: ok

A liveness probe should stay this shallow — it answers "is the process still serving?". If you want a readiness check that also proves a dependency is reachable, point the route's handler: at a resource that exercises it (a Sql.Query running SELECT 1, for instance) and let the route's catches: render a non-2xx on failure.

Kubernetes​

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels: { app: my-app }
  template:
    metadata:
      labels: { app: my-app }
    spec:
      # Longer than the slowest teardown in the manifest — there is no
      # in-process shutdown timeout to fall back on.
      terminationGracePeriodSeconds: 60
      containers:
        - name: app
          image: my-registry/my-app:1.0.0
          ports:
            - containerPort: 8080
          env:
            - name: PORT
              value: "8080"
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef: { name: my-app, key: database-url }
          readinessProbe:
            httpGet: { path: /healthz, port: 8080 }
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz, port: 8080 }
            periodSeconds: 10
          resources:
            requests: { cpu: 100m, memory: 128Mi }

Notes specific to Telo:

  • The image should be built with a warmed cache. telo install in the build stage puts every controller and imported manifest under the .telo/ cache root, so boot does no network I/O. In a monorepo that root is beside telo-workspace.yaml, not beside the manifest — see where the cache lands.
  • The .telo/ tree is read-only at run time and identical in every replica, so it needs no volume and creates no shared state. Replicas are independent processes; nothing in the kernel coordinates between them. Coordination you want is declared — see the lease, idempotency, and rate-limit modules in the standard library, and durable execution for work that must survive a replica dying.
  • Pass --no-cache-write when the container filesystem is read-only. The run still reads the baked cache; it just never persists derived entries.
  • One telo process is one Node.js process. Scale with replicas.
  • Boot may touch your database. A declared schema (Sql.Schema on a backend) reconciles tables at boot, a durable journal creates its own tables, and both run in every replica that starts. Each is written to tolerate concurrent boots. The PostgreSQL journal can be told not to create its tables (createTable: false) where your migrations own them or the runtime user holds no DDL grant; a declared schema always reconciles, so its connection needs the DDL grant — see each module's own docs on the hub.
  • Pin the kernel image AND the module pins together. A module published against a newer Telo declares a floor the pinned image may not meet; see Upgrades & version skew.

Observability​

Logging is fully declared in the manifest — level, sinks, redaction and sampling. Records are structured, carry the resource that emitted them, and carry trace and span identifiers when emitted inside a dispatch. There is no TELO_LOG_LEVEL: bind a variable to your env var and reference it, so the configuration stays visible to telo check and the editor. See Logging.

Records can be shipped to an OpenTelemetry collector with the otlp module's sink, and spans with its OTLP.TraceSink declared in tracing.sinks — see Tracing. There is no metrics signal: counters and latency distributions are aggregations over exported spans, computed by the backend.

Runtime configuration​

Everything the application itself needs is declared on Telo.Application as variables:, secrets: and ports:, each bound to a host environment variable — see Application environment variables and Application ports. The kernel reads the host environment directly for exactly the following:

VariableRead byEffect
TELO_CACHE_DIRrun, install, checkRelocates the .telo cache root. Without it the root is .telo/ beside telo-workspace.yaml when one sits above the manifest, otherwise .telo/ beside the manifest — see where the cache lands. Used for the manifest cache, compiled validators, the analysis stamp, and the npm install root.
TELO_PKG_MANAGERinstallPackage manager invoked for the controllers still delivered over npm. Defaults to npm.
TELO_HUB_URLsearchHub used by telo search. Overridden by --hub-url.
TELO_EGRESSrun, install, checkSet to public-only to refuse fetches from private, loopback or link-local hosts. See Security & supply chain.

A manifest-declared name is bound by the kernel, not read ad-hoc: once an Application declares PORT in its ports: block, that variable is served through ports.http and reading process.env.PORT from inside a controller returns undefined by design, so a declared binding cannot be bypassed.

Bare CLI under a supervisor​

Docker is the recommended packaging, but a plain process works. The unit below relies on the same signal behaviour described above — systemd's default KillSignal is SIGTERM, which is exactly what triggers graceful teardown:

[Unit]
Description=My Telo app
After=network-online.target

[Service]
Type=simple
User=telo
WorkingDirectory=/srv/my-app
ExecStart=/usr/bin/telo /srv/my-app/telo.yaml
Environment=PORT=8080
EnvironmentFile=/etc/my-app.env
Restart=always
RestartSec=2
# Must exceed the slowest teardown; systemd SIGKILLs after this.
TimeoutStopSec=60

[Install]
WantedBy=multi-user.target

Run telo install /srv/my-app/telo.yaml as part of deployment, before starting the unit, so the first boot does not depend on the network.

See also​