Using Pulumi with Hetzner
Consolidating domains and moving cloud providers has been a "TODO" on my org list for quite a while.
I'm currently hosting on both Bluehost and Digital Ocean, and both cost significantly more than I would like to be paying. Hetzner got on my radar a few months ago, and the cost differential (a monthly reduction of 80%) was highly appealing.
While I haven't finished this process (goal is by the end of the year), I did at least start working on the first part of better managing static (and SPA) sites with Pulumi. In the interest of not borking any of my existing systems, I took a domain I already owned (spitedriven.dev), but had no actual site to see how hard it would be to set up Pulumi to manage my infrastructure.
Why Pulumi?
I've used Terraform for work, but I've been interested in Pulumi for a while as an open source solution with Python support (among others). Pulumi also had an Hcloud Provider available, so it looked like this could be pretty easy to set up.
What I Did
The first step was to install the Pulumi CLI (and upgrade XCode). While Pulumi has a bunch of templates, it didn't have one for Hetzner Cloud (yet), so I opted to start a project just using
pulumi new python
I was initially hoping that I could even manage my DNS with Hetzner DNS and Pulumi, but that proved to be an unsuccesful detour. While there is a Github repo for pulumi-hetznerdns, it doesn't look like it's being maintained. The PyPI package has vanished, and attempts to build the site package locally failed. I might revisit this later when I have more time, but it was only a minimal addition of brain space to add the A records to Porkbun to point to the right server.
One thing that was a bit annoying to troubleshoot was that I decided to use uv for the first time to manage the venv for Pulumi. I kept having issues with pulumi up wiping out the site packages, which then resulted in ImportError: No module named hetzner_hcloud. I needed to add
dependencies:
- requirements.txt
to my Pulumi.yaml file
I also had to manually recreate all the secrets for my Pulumi.dev.yaml via
pulumi config set --secret dev:${VAR} ${VAR_VALUE}
because I created a new Pulumi stack of dev.
Pulumi __main__.py
My final __main__.py file for the Pulumi project looked like this:
import pulumi
from pulumi import ResourceOptions
from pulumi_hcloud import Provider, Server
import pulumi_command as command
config = pulumi.Config()
hcloud = Provider("hcloud", token=config.require_secret("hetzner_token"))
server = Server.get(
"existing-server",
config.require("server_id"),
opts=ResourceOptions(provider=hcloud),
)
certbot_install = command.remote.Command(
"install-certbot",
connection={
"host": server.ipv4_address,
"user": "root",
"private_key": config.require("ssh_private_key"),
},
create="apt update && apt install -y certbot python3-certbot-nginx",
)
site_dir = command.remote.Command(
"create-dir",
connection={
"host": server.ipv4_address,
"user": "root",
"private_key": config.require("ssh_private_key"),
},
create=pulumi.Output.concat(
"mkdir -p /var/www/",
config.require("domain"),
" && ",
"chown -R www-data:www-data /var/www/",
config.require("domain"),
),
opts=ResourceOptions(depends_on=[certbot_install]),
)
site_sync = command.local.Command(
"sync-site",
create=pulumi.Output.concat(
"rsync -avz -e 'ssh -i ",
config.require("ssh_private_key_path"),
"' ",
config.require("site_path"),
"/* root@",
server.ipv4_address,
":/var/www/",
config.require("domain"),
"/",
),
opts=ResourceOptions(depends_on=[site_dir]),
)
nginx_config = command.remote.Command(
"nginx-config",
connection={
"host": server.ipv4_address,
"user": "root",
"private_key": config.require("ssh_private_key"),
},
create=pulumi.Output.concat(
"cat > /etc/nginx/sites-available/",
config.require("domain"),
" << 'EOL'\n",
"server {\n",
" listen 80;\n",
" listen [::]:80;\n",
" server_name ",
config.require("domain"),
" www.",
config.require("domain"),
";\n",
" root /var/www/",
config.require("domain"),
";\n",
" index index.html;\n",
"}\n",
"EOL\n",
"ln -sf /etc/nginx/sites-available/",
config.require("domain"),
" /etc/nginx/sites-enabled/",
" && nginx -t && systemctl reload nginx",
),
opts=ResourceOptions(depends_on=[site_sync]),
)
ssl_cert = command.remote.Command(
"ssl-cert",
connection={
"host": server.ipv4_address,
"user": "root",
"private_key": config.require("ssh_private_key"),
},
create=pulumi.Output.concat(
"certbot --nginx -d ",
config.require("domain"),
" -d www.",
config.require("domain"),
" --non-interactive --agree-tos -m ",
config.require("email"),
),
opts=ResourceOptions(depends_on=[nginx_config]),
)
pulumi.export("server_ip", server.ipv4_address)
which handles almost everything I've previously been handling manually, including setting SSL certs, NGINX configuration, and site syncing from my local machine. Since this is just a static site, I anticipate some additional complications with porting over some applications (including bridge.watch) which is running off of containers with docker-compose.
The only other hiccup I had with this setup had to do with my SSH keys in Hetzner Cloud. While I had created a key with ssh-keygen and copied the public key to Hetzner Cloud, I was still being prompted for a password on both the server itself at the CLI, and when trying to run ssh root@${MY_SERVER}. It turned out that I needed to either run:
ssh-add ~/.ssh/${MY_HETZNER_KEY}
for adding it to the ssh-agent
or explicitly passing it in the command line:
ssh -v -i ~/.ssh/${MY_HETZNER_KEY} root@${MY_HETZNER_SERVER}
pulumi up gave me a preview of what would happen on my server, and then gave me a readout of the status for each command I'd specified in the project file:
✦ ❯ pulumi up
Previewing update (dev)
Type Name Plan
+ pulumi:pulumi:Stack dev-dev create
+ ├─ pulumi:providers:hcloud hcloud create
+ ├─ command:remote:Command install-certbot create
+ ├─ command:remote:Command create-dir create
+ ├─ command:local:Command sync-site create
+ ├─ command:remote:Command nginx-config create
+ └─ command:remote:Command ssl-cert create
Resources:
+ 7 to create
Do you want to perform this update? yes
Updating (dev)
Type Name Status
+ pulumi:pulumi:Stack dev-dev created (16s)
+ ├─ pulumi:providers:hcloud hcloud created (0.26s)
+ ├─ command:remote:Command install-certbot created (7s)
+ ├─ command:remote:Command create-dir created (0.64s)
+ ├─ command:local:Command sync-site created (0.93s)
+ ├─ command:remote:Command nginx-config created (0.72s)
+ └─ command:remote:Command ssl-cert created (3s)
Resources:
+ 7 created
Duration: 21s
And with that, I have the scaffold for adding all my other static sites to Hetnzer!
