# ADR 000: Hetzner Cloud

- HTML version: https://robbiepalmer.me/projects/agent-friendly-remote-development/adrs/000-hetzner-cloud
- Project: Agent-friendly Remote Development (https://robbiepalmer.me/projects/agent-friendly-remote-development.md)
- Status: Accepted
- Date: 2026-09-07
- Initiatives: Semi-autonomous Software Development (https://robbiepalmer.me/initiatives/semi-autonomous-software-development.md)

# Context

The project needs a general-purpose Linux server for its first two persistent
workspaces. The initial target is one always-on machine with at least 8 GB of
memory. It must support private access, encrypted backups, and later replacement
from the repository.

Infrastructure as code is a selection requirement. Coding agents need a
versioned description they can inspect and change. The same description must
create a test environment, detect drift, replace a failed server, and remove
resources. Routine provider-console work would leave part of the recovery
procedure outside the repository.

Each candidate has a concrete reason for appearing here. Google Cloud already
hosts the recipe site's Google sign-in configuration. AWS and Azure provide a
useful baseline for mature cloud APIs and Terraform support. Hetzner Cloud,
Hostinger, netcup, and OVHcloud sell low-cost virtual machines. Oracle Cloud's
free Arm allowance makes it an obvious price outlier to check.

The provider search uses a satisficing rule. Stop once a provider passes the
acceptance test, fits the first-host budget, and has no missing capability that
another candidate would supply. A market-wide ranking would go stale quickly.
DigitalOcean, Akamai Linode, Vultr, Scaleway, and other credible providers remain
eligible for a later comparison if their price or capabilities could change the
decision. Omitted providers have no recorded outcome.

# Decision

Use a Hetzner Cloud CX33 for the first host. Manage the provider layer with a
pinned version of the official `hetznercloud/hcloud` Terraform provider.
Within Hetzner, use the available region closest to the operator's usual
location.

The declared provider resources must include the server, SSH keys, primary IP,
firewall, network, and any attached volume. Host configuration starts after
Terraform has created those resources. An emergency console change must be
recorded in code or reverted once service is stable.

Keep encrypted workspace backups independent of Hetzner. Provider snapshots or
backups may shorten recovery, while the required restore path starts from a new
server and the versioned project definitions.

# Selection criteria

Apply the criteria in this order:

1. **Declarative lifecycle.** An actively maintained provider can create,
   update, import, and destroy the VM and its network, firewall, IP, SSH-key,
   and storage resources without routine console steps.
2. **Reproducible host bootstrap.** The API and rescue or user-data facilities
   can install the declared operating system and reach an idempotent second
   plan.
3. **Operational fit.** The provider has documented APIs, predictable deletion
   and billing, and a credible recovery path.
4. **Continuity.** Paid capacity has no idle-use reclamation policy.
5. **Cost.** Compare the full recurring cost among candidates that pass the
   earlier criteria.

# Provider comparison

The comparison reflects published capabilities on 6 September 2026.

| Provider        | Assessment                                                                                                                                                                                                                                                                                                                                      | Outcome                                                                                                                                                                                                                             |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Hetzner Cloud   | The official provider covers servers, IPs, networks, firewalls, volumes, snapshots, load balancers, and their attachments. Rescue mode and user data support automated host installation.                                                                                                                                                       | Selected. It covers every resource in the baseline through a simple operating model with a low recurring cost.                                                                                                                      |
| Google Cloud    | The Google Terraform provider has mature coverage for projects, APIs, Compute Engine, Virtual Private Cloud networks, firewalls, addresses, and disks. The recipe site's Google Cloud project holds OAuth configuration, so an account already exists. A separate project and billing setup would keep this host isolated from the recipe site. | Rejected for the first host. It passes the infrastructure-as-code requirements, while its resource hierarchy, IAM setup, separate disk and network charges, and higher pay-as-you-go cost bring no needed capability at this scale. |
| AWS             | The HashiCorp AWS provider covers the complete EC2 lifecycle, including the instance, Virtual Private Cloud, subnet, security group, address, and EBS volume.                                                                                                                                                                                   | Rejected for the first host. It passes the infrastructure-as-code requirements, while the extra account, IAM, networking, and itemised billing work buys no required capability for one server.                                     |
| Microsoft Azure | The HashiCorp Azure provider covers resource groups, virtual networks, subnets, security groups, public addresses, managed disks, and Linux virtual machines.                                                                                                                                                                                   | Rejected for the first host. It passes the infrastructure-as-code requirements, while its subscription and resource hierarchy plus higher pay-as-you-go cost are a poor fit for the pilot.                                          |
| OVHcloud        | The official provider can order and reinstall a VPS. VPS coverage remains narrower than the provider's wider public-cloud coverage. Open comprehensive-coverage work describes about 15% of the VPS API as implemented and proposes another 20 resources and 40 data sources.                                                                   | Rejected for this project. Important VPS lifecycle operations still depend on incomplete provider coverage or lower-level API and console work.                                                                                     |
| Hostinger       | The official provider can create and delete a VPS subscription, inject an SSH key, and run a post-install script. The provider remains in its 0.1 release series, and Hostinger does not accept custom operating-system images.                                                                                                                 | Rejected. The provider and custom-host installation path are less mature than Hetzner's. Comparable renewal terms also cost more.                                                                                                   |
| netcup          | netcup documents Server Control Panel and REST APIs, custom images, and hourly VPS billing. The community Terraform provider manages parts of an existing server. netcup's public API cannot create or delete the server contract itself.                                                                                                       | Rejected. The lifecycle begins and ends outside Terraform, which prevents a complete create-and-destroy test.                                                                                                                       |
| Oracle Cloud    | Oracle maintains a broad official Terraform provider. Its free Arm capacity may be unavailable in the account's home region, and Oracle may reclaim it after sustained low use. A deployment also requires compartments, virtual cloud networks, subnets, routes, security rules, and IAM.                                                      | Rejected for the pilot. Free capacity cannot support an availability promise. A paid deployment adds administration that this one-host service does not need.                                                                       |

Sources: [Hetzner Terraform server resource](https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/server),
[Hetzner provider releases](https://github.com/hetznercloud/terraform-provider-hcloud/releases),
[Google Cloud Terraform documentation](https://cloud.google.com/docs/terraform),
[Google Compute Engine pricing](https://cloud.google.com/products/compute/pricing),
[Google disk pricing](https://cloud.google.com/compute/disks-image-pricing),
[Terraform AWS tutorial](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/aws-create),
[AWS EC2 pricing](https://aws.amazon.com/ec2/pricing/on-demand/),
[Microsoft's Terraform virtual-machine example](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-terraform),
[Azure Linux virtual-machine pricing](https://azure.microsoft.com/en-gb/pricing/details/virtual-machines/linux/),
[recipe-site Google OIDC decision](/projects/recipe-site/adrs/036-google-oidc-login),
[OVHcloud Terraform VPS resource](https://registry.terraform.io/providers/ovh/ovh/latest/docs/resources/vps),
[OVHcloud provider changelog](https://github.com/ovh/terraform-provider-ovh/blob/master/CHANGELOG.md),
[OVHcloud VPS coverage issue](https://github.com/ovh/terraform-provider-ovh/issues/1322),
[Hostinger Terraform VPS resource](https://registry.terraform.io/providers/hostinger/hostinger/latest/docs/resources/vps),
[Hostinger VPS prices](https://www.hostinger.com/vps-hosting),
[Hostinger custom-image policy](https://www.hostinger.com/support/8852324-can-you-add-your-own-os-image-to-vps-at-hostinger/),
[netcup server API documentation](https://www.netcup.com/en/helpcenter/documentation/server),
[netcup VPS terms](https://www.netcup.com/en/server/vps),
[community netcup Terraform provider](https://registry.terraform.io/providers/hornc-greedy/netcup/latest/docs),
[Oracle Terraform provider](https://registry.terraform.io/providers/oracle/oci/latest/docs),
and [Oracle Always Free limits](https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier_topic-Always_Free_Resources.htm).

## Promotional credits and free allowances

Public sign-up offers can fund a short proof of concept. Their expiry leaves
the project paying standard rates or migrating persistent workspaces. Check
eligibility and the remaining balance in the provider's billing system before
assigning any value to a credit offer.

The estimates below use a Linux VM with 2 vCPUs and 8 GB of memory, running for
730 hours per month. AWS and Google use Frankfurt prices; Azure uses West Europe.
Those locations set the price used in the calculation. The quoted prices cover
VM compute only. Boot storage, public addresses, traffic, and tax can shorten
the quoted duration. Calculate monthly compute by multiplying the published
hourly price by 730. Funded time equals the credit divided by that hourly price,
capped at the offer's expiry date.

The selected CX33 and IPv4 cost €8.99 per month. The
[European Central Bank reference rate for 4 September 2026](https://data-api.ecb.europa.eu/service/data/EXR/D.GBP.EUR.SP00.A?startPeriod=2026-09-04\&endPeriod=2026-09-04\&format=csvdata)
converts that to £7.72, or about £0.254 per day. The last column shows the
Hetzner bill avoided while each offer funds the comparison VM. Add-on charges
that shorten the funded time also reduce this saving.

The offers and on-demand prices checked on 7 September 2026 give these results:

| Provider        | Comparison VM and compute price                                                          | Public offer                                                                                                                              | Time funded for one always-on VM                                                                                                                                                    |                                    Hetzner cost avoided |
| --------------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------: |
| AWS             | `m7i-flex.large`, 2 vCPU and 8 GiB, about $83.74 per month                               | New customers receive $100 and can earn another $100. The Free account plan lasts six months; credits expire after 12 months.             | The initial $100 covers about 36 days. Earning the full $200 raises that to about 73 days. Storage, addresses, and traffic use the same credit and would bring those dates forward. | About £9 initially or £18 after earning the full credit |
| Google Cloud    | `e2-standard-2`, 2 vCPU and 8 GiB, about $63.00 per month                                | Eligible new customers receive $300 for 90 days.                                                                                          | The 90-day expiry caps the run at 90 days, using about $186 of the credit for compute. The continuing free `e2-micro` has only 1 GB of memory.                                      |                                               About £23 |
| Microsoft Azure | `B2as_v2`, 2 vCPU and 8 GiB, about $63.07 per month                                      | Eligible new customers receive $200 for 30 days. New customers also receive 750 monthly hours of `B2pts_v2` and `B2ats_v2` for 12 months. | The 30-day expiry caps the run at 30 days, using about $62 of the credit for compute. Both 12-month VM sizes have only 1 GiB of memory.                                             |                                                About £8 |
| Oracle Cloud    | `VM.Standard.A1.Flex`, 2 Arm OCPUs and 8 GB, $23.36 per month at published overage rates | New accounts advertise a $300, 30-day trial. Always Free includes 3,000 A1 OCPU-hours and 18,000 GB-hours of memory each month.           | The VM fits inside the monthly Always Free allocation and can continue past the trial while its capacity remains allocated. Oracle may reclaim an idle instance.                    | £7.72 for each month that the free VM remains available |

The recipe site's OAuth project proves that a Google Cloud account exists.
Google determines trial eligibility from the account's billing and trial
history, so the OAuth project alone says nothing about whether the $300 remains
available. Verify that in Cloud Billing before reconsidering Google Cloud.

Startup programmes offer much larger awards. AWS Activate advertises $1,000 for
self-funded founders and may award selected applicants up to $5,000.
Provider-backed companies can apply for as much as $200,000. Google advertises
$2,000 for qualifying pre-funded startups and $200,000 for qualifying
venture-backed startups, rising to $350,000 for qualifying AI companies.
Microsoft's Investor Network advertises up to $150,000 for eligible
partner-backed startups. Approval, company status, funding, prior credits, and
expiry determine the usable amount.

For this project, savings equal avoided Hetzner spend. One year on the selected
host costs about £93. Extra credit has value when the project would otherwise
buy more infrastructure before the award expires.

Compare providers at their recurring post-credit cost. Record an awarded credit
balance separately as temporary runway. An award that covers the same host for
at least 12 months triggers a fresh provider comparison before buying the next
server.

Sources: [AWS Free Tier](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/free-tier.html),
[AWS EC2 Free Tier eligibility](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-free-tier-usage.html),
[AWS M7i Flex specifications](https://aws.amazon.com/ec2/instance-types/general-purpose/),
[AWS Price List documentation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/using-the-aws-price-list-bulk-api.html),
[Google Cloud Free Program](https://cloud.google.com/free/docs/free-cloud-features),
[Google E2 machine types](https://cloud.google.com/compute/docs/general-purpose-machines#e2_machine_types),
[Google Compute Engine pricing](https://cloud.google.com/products/compute/pricing),
[Azure free services](https://azure.microsoft.com/en-gb/pricing/free-services/),
[Azure Arm B-series sizes](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/bpsv2-series),
[Azure AMD B-series sizes](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/basv2-series),
[Azure Retail Prices API](https://learn.microsoft.com/en-us/rest/api/cost-management/retail-prices/azure-retail-prices),
[Oracle Cloud Free Tier](https://www.oracle.com/cloud/free/),
[Oracle Ampere A1 pricing](https://www.oracle.com/cloud/compute/arm/pricing/),
[AWS Activate](https://aws.amazon.com/startups/credits/),
[Google for Startups benefits](https://cloud.google.com/startup/benefits), and
[Microsoft Investor Network benefits](https://learn.microsoft.com/en-us/startups/microsoft-for-startups/unlock-more-benefits).

## OVHcloud detail

OVHcloud has an official Terraform provider. Its `ovh_vps` resource appeared in
April 2024, followed by SSH-key and image-installation fields in July 2025.
Terraform can order and install a VPS.

The remaining gaps affect operations needed for reproducible recovery. The
open comprehensive-coverage work lists image discovery, reinstallation,
snapshots, automated backups, networking, power operations, and existing
creation fields. A recent fix also addressed a VPS becoming tainted immediately
after creation. Hetzner exposes the required lifecycle through an established,
integrated set of resources.

OVHcloud's VPS-2 lists a lower monthly price than a Hetzner CX33 with IPv4 and
includes a daily backup. That saving does not cover the engineering and
recovery risk created by the provider gaps. OVHcloud becomes a candidate again
when comprehensive VPS coverage has landed and passes the acceptance test.

Sources: [OVHcloud VPS prices](https://www.ovhcloud.com/en-ie/vps/unmetered-vps/),
[Hetzner server prices](https://docs.hetzner.com/general/infrastructure-and-availability/price-adjustment/),
[Hetzner IP prices](https://docs.hetzner.com/cloud/servers/primary-ips/overview/),
[initial OVHcloud VPS resource](https://github.com/ovh/terraform-provider-ovh/pull/594),
[OVHcloud SSH-key and image-installation fields](https://github.com/ovh/terraform-provider-ovh/pull/1015),
[OVHcloud comprehensive VPS pull request](https://github.com/ovh/terraform-provider-ovh/pull/1323),
and [OVHcloud post-create state fix](https://github.com/ovh/terraform-provider-ovh/pull/1267).

# Provider acceptance test

A replacement provider must pass this test in a disposable project:

1. Create the network, firewall, SSH key, IP, server, and volume from versioned
   Terraform or OpenTofu configuration.
2. Bootstrap the declared host without a provider-console step.
3. Run a second plan with no unexplained changes.
4. Restore an encrypted workspace backup and run the standard build and private
   access checks.
5. Replace the server while preserving the documented durable state.
6. Destroy every test resource from code and reconcile the final invoice.

# Consequences

The provider lifecycle is reviewable by people and coding agents, and the first
host has a low fixed cost. Hetzner-specific Terraform remains necessary at the
lowest layer. The project also owns host installation, workload operation,
backup, and recovery. Hetzner charges separately for backups, and a comparable
OVHcloud VPS may have a lower list price.

# Review point

Review this decision when the provider fails a recovery exercise, awarded cloud
credits cover the same 8 GB host for at least 12 months, or another provider
passes the complete acceptance test with materially better cost or reliability.
Record a provider change in a new ADR after collecting that evidence.

---

Markdown index of this site: https://robbiepalmer.me/llms.txt
