The architecture, subsystem by subsystem

Enclave is a control plane for running untrusted, agent-generated workloads inside isolated, credential-scoped, ephemeral sessions. The whole system is a pnpm/TypeScript monorepo organized around one principle: data flows in one direction toward a single hub. Every package depends on shared for the wire format; the control plane is the hub everything else talks to; the SDK, MCP server, and UI are thin clients of its REST API; the console-api is the identity authority the control plane trusts for who.

Three structural decisions do most of the work:

• One backend interface. The control plane talks to exactly one SessionBackend interface (launch / teardown / health? / exec?). Everything above it is backend-agnostic, so the same logic drives the simulator, Docker, Kubernetes/gVisor, and Firecracker paths identically — and the entire control plane is developable and provable on a laptop against the in-process simulator, with no cluster.
• Secret withholding is structural. No brokered secret is ever injected into the workload, and the Session type has no secret field; publicView() is structuredClone(session), so nothing secret can leak onto the wire. Service-binding secrets are injected at the egress-proxy boundary; a private-repo git credential is mounted only on the clone init-container.
• Security is enforced server-side. Auth, RBAC, and org-scoped tenancy live in the control plane and run on every route (never UI/BFF-only). Cross-org access returns a uniform 404. Egress is default-deny on every backend.

The platform's six components

Reading it: callers reach the control plane only through the SDK/MCP/UI over the authenticated REST + SSE surface. Every request passes the auth hook (which sets req.principal), then RBAC- and org-scoped routes, then the orchestrator. The orchestrator admits the work through the queue and hands a LaunchSpec to whichever SessionBackend is configured (minting a scoped git credential for the clone init-container if the source is a private repo). The backend stands up the session boundary; the workload runs on the dangerous side; egress is default-deny and any allowed traffic flows through the per-session proxy that injects service-binding secrets the sandbox never sees. Everything the orchestrator does flows through the applyEffects chokepoint that feeds audit, SSE, events, and billing — so all four backends inherit identical behaviour.

The orchestrator's applyEffects is the single chokepoint (invariant 2): every state transition there fans out to the audit log, the SSE stream, the webhook dispatcher, and the billing meter at once. Because it sits above the backend interface, the simulator, Docker, Kubernetes, and Firecracker paths all produce identical audit, streaming, event, and billing behaviour.

shared/ is the keystone: changing a contract there is what keeps every package in lock-step. It holds the domain wire types (Session, SessionSpec, StreamFrame, SessionResult, EgressPolicy, …), the auth contract (Principal, Role, Scope, JWT/API-key verify helpers in shared/src/auth.ts), the billing types, and the environment types. It has no runtime — pure types and small pure helpers.

enclave/
├── shared/          wire-format types + auth contract (one source of truth)
├── control-plane/   the hub — Fastify API + orchestrator + backends
│   └── src/backends/  simulator · docker · kubernetes · firecracker
├── console-api/     identity authority (login, orgs, RBAC, API keys)
├── sdk/             typed TS client
├── mcp/             MCP server (agent tools)
├── ui/              React web console
├── runner/          in-sandbox harness image (python + node)
├── deploy/          k8s manifests (RuntimeClass, NetworkPolicy, RBAC, …)
└── demo/            self-contained adversarial demo

The hub. Boot is src/index.ts → routes in src/server.ts → the src/orchestrator.ts engine, with backends in src/backends/ and feature modules in their own folders.

Key seams

• config.ts resolves the deployment profile. In hosted, auth + RBAC + tenant isolation are forced on and the dev bypass (ENCLAVE_AUTH_DISABLED=1) is refused, along with default/empty signing/JWT/internal secrets.
• backends/factory.ts picks the backend from ENCLAVE_BACKEND (simulator | docker | kubernetes | firecracker).
• queue/factory.ts picks the admission backend from ENCLAVE_QUEUE_BACKEND (in-process | redis).
• billing/factory.ts picks an in-memory or console-api-persisted ledger.
• clock.ts is the injectable clock every time-dependent module (cron, retry backoff, idle-TTL, metering) reads — so tests are deterministic.

The whole system's testability hinges on this one interface (backends/backend.ts):

launch(spec: LaunchSpec, sink: BackendEventSink): Promise<void>   // runs async, emits via sink
teardown(sessionId): Promise<void>                               // idempotent resource reclaim
health?(): Promise<{ ok, detail? }>                              // readiness probe
exec?(sessionId, turn, req): Promise<ExecTurnResult>             // interactive (warm) turns

All four go through sentinel.ts / sentinel-stream.tsto parse the runner's sentinel-line protocol (stdout/stderr, structured result, declared artifacts, egress observations) and workload-model.tsfor the simulator's modelling.

The sandbox holds no brokered secret of its own. Two protections keep secrets out of the workload: the broker mints a private-repo git credential injected only on the clone init-container; service bindings ensure a service secret never enters the sandbox at all.

• Git-clone credential — for a private-repo source, credentials.ts mints a signed, short-lived, scoped git credential. It is mounted only on the clone init-container (a private HOME volume), never the workload, and never returned by the API. The Session type has no secret field; publicView() is structuredClone — leaking is structurally impossible (invariant 1).
• Service bindings — a tenant-scoped ServiceBinding defines an upstream, an injection scheme (bearer/header/basic), and a write-only secret the API never returns. At launch the control plane provisions a per-session egress proxy and injects a per-binding base-URL env var (ENCLAVE_SVC_<NAME>) — a URL, not a secret. The NetworkPolicy permits egress only to the proxy, which matches the binding, injects the secret, forwards to the real upstream, and audits each call. On k8s the proxy runs as a native sidecar fed a Secret mounted only on it.
• Bound services (bound-services/) are the test/demo upstreams — a real Deployment+Service+Secret (k8s) or container (Docker), or the in-process echo service — that bindings point at, with a runtime.ts abstraction over Docker and Kubernetes.

• One-shot (execMode: "oneshot") — run a program/script/repo-entrypoint or snippet to completion with a timeout; capture {stdout, stderr, exit, json} via the enclave.result() structured-result contract, plus declared output artifacts streamed as artifact sentinels (utf8 verbatim / base64, trusted-side byte-capped).
• Interactive (execMode: "interactive") — the Code-Interpreter pattern. The sandbox stays warm and accepts a sequence of exec turns sharing one persistent namespace (interpreter globals + filesystem + installed packages). runner_kernel.py becomes a small kernel server; on k8s the session is a long-lived Pod (not a Job) driven over the k8s exec stream, reaped on idle-TTL/teardown with an activeDeadlineSeconds crash backstop. In-order, one exec at a time; bounded by per-turn wall-clock, idle-TTL, and max-lifetime.
• Egress is observed, never self-enforced — the runner emits sentinel lines for egress attempts, but the enforcement is at the network layer (NetworkPolicy / --network none / no microVM tap). The workload can never vouch for itself.

• Queue— a concurrency-bounded admission gate between the API and the backend so bursts can't overwhelm the cluster. Default in-process (FIFO, resource-free, single-replica). A Redis-backed backend (ENCLAVE_QUEUE_BACKEND=redis) makes max-concurrency global across replicas via a fair, lease-based, crash-safe semaphore behind the same AdmissionQueue interface — orchestrator and backends unchanged. (This is distributed admission control, not session-ownership failover.)
• Fleets — submit N workloads as a named fleet; fan out through the queue; track aggregate phase + per-member results. One layer above sessions, not a DAG.
• Triggers — bind a workload template (inline or git) to an event: inbound webhook, HMAC-verified GitHub webhook (github.ts), or cron (cron.ts). Firing enqueues a session/fleet.

Two services with a clean split: console-api owns who (identities, orgs, keys); the control plane enforces what they may do (authz + org-scoping) on every route.

• Auth — every caller-facing route requires a bearer credential: a user JWT (HS256, minted by console-api) or an API key (ek_<id>_<secret>, resolved via console-api /internal/introspect). Verified in the control plane, not just a BFF. The hosted profile requires a hashed console credential and refuses the placeholder.
• RBAC — fixed roles enforced server-side: owner (full), admin (members/keys), developer (sessions/fleets/bindings/triggers/webhooks), viewer (read). A cross-tenant platformAdmin claim is server-minted into the JWT and never re-derived.
• Multi-tenancy — every session/fleet/trigger/audit/binding record belongs to an org; all access is org-scoped server-side (cross-org → uniform 404, invariant 5). Orgs can be suspended (enforced on create + exec). On k8s, each org's session objects live in a per-tenant namespace, reaped when empty.

• Audit — immutable, secret-free, tenant-attributed per-session log.
• Streaming — SSE with bounded backlog + replay for late subscribers.
• Per-session result webhook (events.ts) — fires session.completed to a sink with HMAC-SHA256 signing and bounded clock-driven retry/backoff. Documented boundary: the signing secret is deployment-wide.
• Webhook subscriptions (webhooks/) — org-scoped endpoints subscribe to 7 event types (session.created/running/completed/failed/killed/torn_down, egress.denied), each signed with its own per-endpoint whsec_ secret (Svix/Stripe-style timestamp-in-signature replay defence). Dispatched fire-and-forget from the applyEffects chokepoint. Destination URLs are SSRF-guarded at store time and delivery time, plus an egress NetworkPolicy backstop on the control-plane pod. The SDK ships verifyWebhookSignature(...).
• No session replay — deliberately out of scope.

• Billing — minute-resolution metering, per-org prepaid balance, a 402 admission gate on insufficient credits, lazy + recurring grants, finalize-debit; in-memory or console-api-persisted ledger. Enforced on session create + exec.
• Environments — a secret-free EnvironmentSpec (org-scoped + global built-ins) builds to an OCI image via a generated Dockerfile; every Dockerfile unconditionally appends the /app runner harness + a non-root USER (root refused at authoring). Base registries (incl. inline FROM) must be allowlisted. A content digest is stamped into the audit ("what exactly executed?"). On k8s the backend runs the resolved custom image; on Firecracker it's converted to a content-addressed rootfs.ext4. Also imports a safe subset of .devcontainer (Features/host-hooks rejected + audited).

• Self-host (Built) — deploy/base/ composed by deploy/kustomization.yaml: Namespace, RuntimeClass (runsc), least-privilege RBAC, LimitRange, control-plane Deployment + Service, plus the control-plane egress NetworkPolicy.
• Hosted / multi-tenant (Partial) — the same control-plane binary in the hosted profile (ENCLAVE_PROFILE=hosted): auth + RBAC + tenant isolation enforced, dev bypass refused. deploy/hosted/ flips the profile; per-tenant namespaces are provisioned at runtime. console-api runs as a SAM Lambda (DynamoDB repo) or locally as Express (in-memory repo).
• Backends by host: simulator + Docker run on a laptop (no Linux/cluster); gVisor needs Linux + k3s/k3d; Firecracker needs a KVM host (/dev/kvm).