# Agent-friendly Remote Development

> Persistent remote workspaces for directing coding agents across devices, repositories, and harnesses

- HTML version: https://robbiepalmer.me/projects/agent-friendly-remote-development
- Status: in_progress
- Started: 2026-09-06
- Updated: 2026-10-14
- Technologies: Terraform, NixOS, Docker, Doppler, Flux Schema, K3s, Hetzner Cloud, Tailscale, Healthchecks.io
- Built on: Build and runtime
- Platform technologies: Nix, NixOS, Tailscale
- Platform policies: Artifact coordinates and digests are independent of the blob provider, Declare services private-only or govern their public ingress separately, Repository declarations are authoritative; catalog records are derived

# Vision

Give each person a private development computer that remains available from a
phone or laptop. Coding agents continue working when the client disconnects,
and the person can return to the same files, processes, and context.

The workspace belongs to a person. It can hold related first-party
repositories, third-party code checked out for inspection or forking,
worktrees, local services, and multiple agent harnesses. T3 Code is the first
interface for directing Codex, Claude Code, OpenCode, Grok, terminals, browser
tools, and requests that need human attention.

# Problem

Local development ties active work to one device. Hosted task runners solve a
narrower problem: they execute a job in a fresh environment and return a
result. This project needs a durable working home where a person can move
between devices, supervise multiple agents, and leave long-running work in
progress.

Running that home on a remote server creates its own burden. Someone must
provision the machine, keep it patched, isolate users, preserve data, manage
access, and recover from failure. A useful service must make those jobs
repeatable enough for coding agents to inspect and help operate.

# Target user

The first users are developers who already direct coding agents for daily
work and want the environment available throughout the week. They are
comfortable bringing their own GitHub and model-provider accounts. They value
persistent multi-repository state and unattended execution enough to prefer a
hosted service over maintaining a server themselves.

The initial cohort is small and trusted. The product does not yet claim a
security boundary suitable for mutually hostile tenants or regulated
production data.

# Product requirements

Each user receives:

* a private, durable home directory and one or more working directories;
* access to T3 Code and the supported coding harnesses from a phone or laptop;
* work that continues through client disconnects;
* their own source-control and model-provider identities;
* resource and placement limits that prevent an accidental workload from
  exhausting another user's workspace node;
* encrypted backups with a tested restore path;
* a documented way to export their files and leave the service; and
* an operator who can provision, update, recover, and remove the workspace from
  versioned definitions.

Users must not share a home directory, provider login, private endpoint, or
ordinary service traffic. The workspace must not receive the host filesystem,
container-runtime socket, or cluster administrator credentials.

# Rollout boundary

The first deployment is now one workspace for one person. The attempted
two-workspace design consumed recovery headroom on a 4 vCPU, 8 GiB host before
the second user was onboarded. My brother's pilot is deferred until it can run
on dedicated compute without competing with the operator workspace.

This changes the rollout order, not the product goal. The current VPS proves
the persistent-workspace workflow and remains the rollback source while a
multi-node design is evaluated. No migration may erase the only copy of a T3
session, worktree, branch, login, or workspace backup.

## Workspace telemetry notice

The operator workspace records infrastructure health and a limited set of T3
Code operational events. Project-owned PostHog receives provider and terminal
lifecycle traces plus warning and error markers. Netdata receives aggregate T3
Code counters and host, storage, and Kubernetes health metrics. Records use the
workspace ID `operator` and may include provider type, operation, outcome,
runtime mode, trace ID, and timing.

Prompts, responses, source files, diffs, terminal output, environment
variables, credentials, provider payloads, personal identifiers, and working
directory paths are not exported. Warning and error text is replaced with a
fixed marker before export. A more detailed rotating trace remains only on the
workspace host for local diagnosis. This telemetry must be disclosed and
reviewed again before enabling another person's workspace.

# Current milestone

The current NixOS and K3s definitions now describe only the operator workspace.
Its durable-data ceiling is 55 GiB, its rebuildable cache has a separate 30 GiB
ceiling, and the filesystem keeps about 13 GiB outside both quotas. K3s state
and container images remain on the root disk, so a full cache cannot also remove
the control plane's restart capacity.

The next milestone is a parallel Kube-Hetzner evaluation. The existing VPS
stays available while a candidate cluster proves node replacement, volume
reattachment, encrypted workspace restore, control plane recovery, and T3
session continuity. Any conversion waits until those checks pass on dedicated
workspace compute; second-user onboarding waits too.

Rendered workspace definitions now pass Flux Schema validation against pinned
Kubernetes 1.37 and Doppler Operator schemas. The check rejects missing schemas
and evaluates the Doppler custom resource's CEL admission rule before a live
cluster dry run.

# Success measures

The next rollout stage succeeds when:

1. the current workspace has an automated encrypted backup and a tested restore;
2. a candidate cluster can lose and replace a worker without losing workspace
   data;
3. a restored copy opens the expected repositories, T3 threads, and provider
   sessions before any cutover;
4. one user's compute, storage, and lifecycle are independent from another's;
5. a failed migration can return to the untouched VPS without reverse data
   reconstruction; and
6. normal operation leaves no undocumented provider-console or host changes.

# Scope boundaries

The next rollout does not include hostile-tenant isolation, an SLA, metered
model usage, automatic billing, per-user infrastructure spend caps, or a
general credential broker. Users continue to buy their own model access.

The project may later connect to the Work Graph and other knowledge services.
A repository and ordinary provider accounts must remain sufficient to use the
workspace.

# Recorded decisions

The implementation choices and their trade-offs live in ADRs:

* [ADR 000: Hetzner Cloud](/projects/agent-friendly-remote-development/adrs/000-hetzner-cloud)
* [ADR 001: NixOS workspace hosts](/projects/agent-friendly-remote-development/adrs/001-nixos-host)
* [ADR 002: K3s for workspace workloads](/projects/agent-friendly-remote-development/adrs/002-k3s-workloads)
* [ADR 003: Portable per-user agent workspaces](/projects/agent-friendly-remote-development/adrs/003-portable-per-user-workspaces)
* [ADR 004: Coder Community trial](/projects/agent-friendly-remote-development/adrs/004-coder-community)
* [ADR 005: Early-cohort pricing](/projects/agent-friendly-remote-development/adrs/005-small-cohort-pricing)
* [ADR 006: Delegated resource access](/projects/agent-friendly-remote-development/adrs/006-delegated-resource-access)
* [ADR 007: Per-workspace Doppler secrets](/projects/agent-friendly-remote-development/adrs/007-doppler-secrets)
* [ADR 008: Shared-host resource containment](/projects/agent-friendly-remote-development/adrs/008-shared-host-resource-containment)
* [ADR 009: Pre-apply manifest validation with Flux Schema](/projects/agent-friendly-remote-development/adrs/009-flux-schema-validation)
* [ADR 010: Layered observability for agent workspaces](/projects/agent-friendly-remote-development/adrs/010-layered-workspace-observability)
* [ADR 011: Single-user boundary for the first host](/projects/agent-friendly-remote-development/adrs/011-single-user-host-boundary)
* [ADR 012: Kube-Hetzner multi-node evaluation](/projects/agent-friendly-remote-development/adrs/012-kube-hetzner-evaluation)
* [ADR 013: Managed heartbeat monitoring with Healthchecks.io](/projects/agent-friendly-remote-development/adrs/013-managed-heartbeat-monitoring)

# Open questions

* What is the smallest Kube-Hetzner topology that survives a node failure
  without turning routine operation into cluster administration?
* Which workspace backup target and schedule meet the required recovery point
  without treating a Hetzner Volume as a backup?
* Should durable workspace nodes remain static while only disposable build
  workers autoscale?
* Does a workspace control plane remove enough onboarding and lifecycle work to
  justify operating it?
* Which files and sessions belong in an export, and which credentials must be
  revoked or excluded?
* How much operator time remains after onboarding and recovery are automated?
* Will invited users pay for continuity and maintenance when self-hosting stays
  available?

## Initiatives

- [Semi-autonomous Software Development](https://robbiepalmer.me/initiatives/semi-autonomous-software-development.md): A connected system that keeps useful work moving, asks for human judgment when it matters, and learns how to need less intervention next time

## Architecture Decision Records

- [ADR 000: Hetzner Cloud](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/000-hetzner-cloud.md) — Accepted, 2026-09-07
- [ADR 001: NixOS workspace hosts](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/001-nixos-host.md) — Accepted, 2026-09-06
- [ADR 002: K3s for workspace workloads](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/002-k3s-workloads.md) — Proposed, 2026-09-06
- [ADR 003: Portable per-user agent workspaces](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/003-portable-per-user-workspaces.md) — Proposed, 2026-09-06
- [ADR 004: Coder Community trial](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/004-coder-community.md) — Proposed, 2026-09-06
- [ADR 005: Early-cohort pricing](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/005-small-cohort-pricing.md) — Proposed, 2026-09-06
- [ADR 006: Delegated resource access](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/006-delegated-resource-access.md) — Proposed, 2026-09-06
- [ADR 007: Per-workspace Doppler secrets](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/007-doppler-secrets.md) — Accepted, 2026-09-09
- [ADR 008: Shared-host resource containment](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/008-shared-host-resource-containment.md) — Accepted, 2026-09-09
- [ADR 009: Pre-apply manifest validation with Flux Schema](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/009-flux-schema-validation.md) — Accepted, 2026-09-09
- [ADR 010: Layered observability for agent workspaces](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/010-layered-workspace-observability.md) — Proposed, 2026-09-12
- [ADR 011: Single-user boundary for the first host](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/011-single-user-host-boundary.md) — Accepted, 2026-09-14
- [ADR 012: Kube-Hetzner multi-node evaluation](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/012-kube-hetzner-evaluation.md) — Proposed, 2026-09-14
- [ADR 013: Managed heartbeat monitoring with Healthchecks.io](https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/013-managed-heartbeat-monitoring.md) — Accepted, 2026-09-29

---

Markdown index of this site: https://robbiepalmer.me/llms.txt