arthur@homelab:~$ cat ~/posts/leahs-blog-south-africa-and-home.md
Leah's Blog Went to South Africa and Came Home
This spring I spun up a Ghost container at home as a demo for Leah. The idea was simple: give her a real place to write about her Africa trip, see whether Ghost felt comfortable, and avoid turning a small experiment into an infrastructure project before it had earned one.
The demo worked. The first post appeared on April 2, and the disposable-looking container stopped being disposable as soon as it held something worth keeping.
What followed was a useful little infrastructure round trip. The blog began in my homelab, moved to an EC2 instance in AWS’s Cape Town region for the active trip, accumulated a summer of writing and photographs, and returned home in August. It is now back in the original Incus container, but with the current database and all of the content created while it was away.
The first home
The original deployment was a Debian Incus container running Docker Compose. Ghost 6 handled the site, MySQL 8 held its database, and the content directory lived in a named Docker volume. Tailscale gave the container its own stable identity, while Caddy handled the public hostname.
That was enough for a demo, and deliberately so. I did not want to design a grand publishing platform for one person before she had even decided whether she liked the editor.
By early June, the container had proved the application but not necessarily
the location. The blog was about a trip in Africa, and I began moving its active
deployment to EC2 in AWS’s af-south-1 region in Cape Town. The software shape
stayed familiar—Ghost, MySQL, Docker Compose, and Caddy—but the public origin
would now live in South Africa.
The first AWS deployment disappeared
My first attempt used a Spot instance. It looked like an economical fit for a small blog, but I had not designed the application around the central fact of Spot capacity: the machine can disappear.
In my initial configuration, reaching the default maximum-CPU condition led to the Spot instance being terminated. Ghost’s database and uploaded content were local to that machine. When it went away, so did work that had been added after the move.
This was not a theoretical warning from an architecture diagram. Leah had already written posts and sent photographs through a site I had presented as ready to use. I had made the infrastructure inexpensive by making its state disposable, without treating the state as disposable anywhere else.
Recovery was partly technical and partly human. I brought the original on-prem deployment back into the process and used it as the base for a new AWS site. Leah searched through sent email and her photo library, then resent the words and images that existed only on the terminated instance. We reconstructed the missing entries from those copies and deployed the blog again on persistent EC2 capacity.
The replacement kept the familiar Ghost, MySQL, Docker Compose, and Caddy shape, but it no longer depended on an interruptible machine. The incident also changed the meaning of the old home container. It was no longer merely a demo I had forgotten to delete; it was the only intact baseline after the first cloud deployment vanished.
The old container remained at home, powered off. That turned out to be a better rollback plan than I appreciated at the time.
A season in Cape Town
The rebuilt AWS site did its job quietly. Leah published nine more entries after the April demo, with posts appearing through July 22. Ghost accumulated not only the database records but also hundreds of megabytes of images, generated assets, theme files, and other content.
Nothing about the deployment was especially exotic. That was the point. A small VM with conventional containers is easy to understand, and for a travel blog the most important feature is that it keeps accepting words and pictures.
Eventually the trip ended and my own public website moved back toward the homelab. Leah’s blog was an obvious candidate to follow. The original container was still there, the external Caddy server already had a route for it, and its Tailscale address had survived months of being powered off.
The danger was treating the old container as though it were the current site. It was not. Its RSS feed held one April entry; AWS held ten. Turning it on was easy. Moving the state was the actual migration.
Finding the right machine
The first obstacle was almost embarrassingly ordinary: SSH appeared to reject the correct private key.
There were two different addresses in play. One hostname now led to my public Caddy edge, while the blog itself still resolved to its EC2 address. After identifying the actual Cape Town instance through AWS, the key still seemed to fail.
The key was valid. Its fingerprint matched. The problem was the SSH command.
I had loaded the key into an agent and also forced IdentitiesOnly=yes. In this
combination, SSH did exactly what it was told and declined to offer the agent
identity because it had not been named as an identity file. Removing that
contradictory option made the original login work.
Before finding that mistake, I had already prepared a conservative recovery path: a snapshot of the 12 GiB root volume, a temporary Amazon Linux recovery instance, and an SSH-only security group restricted to one address. Once direct access worked, none of those resources was needed. I removed them after the migration rather than leaving a small archaeology site in the AWS account.
Moving Ghost without guessing
The live AWS stack had three containers: Caddy, Ghost 6, and MySQL 8. Ghost’s content and MySQL’s data each lived in named Docker volumes. The on-prem target used the same Ghost and MySQL major versions, which kept the restore path predictable.
I took two backups from the running AWS host:
- A transaction-consistent
mysqldumpcontaining the Ghost schema and data. - A compressed archive of the complete Ghost content volume.
The database dump compressed to only 85 KiB. The photographs and other content compressed to about 652 MiB. Both received SHA-256 checksums on AWS, after transfer, and again inside the on-prem container. A file was not considered a backup merely because a command had created it; the bytes had to match at every boundary.
I also backed up the old on-prem database and content, then created an Incus snapshot of the entire destination container. That gave the migration two independent rollback paths before it replaced anything.
The transfer itself was less graceful than the plan. Long copy sessions were interrupted, and one attempted resume overlapped a transfer that was still finishing. The staged archive became larger than its source—an excellent sign that it should go nowhere near the live volume. I discarded only that staging copy and retransmitted the archive in fixed byte ranges at explicit offsets. The final checksum matched AWS exactly.
Only then did I stop Ghost, extract the content, recreate the MySQL database, and stream the verified SQL dump into it. MySQL stayed running while Ghost was offline, preventing the application from racing its own startup migrations against the restore.
Running out of disk at the worst useful moment
The first restore attempt filled the container’s root filesystem.
The destination had an old content tree, a rollback backup, the newly extracted
content, and another 652 MiB copy of the transfer archive. MySQL dropped the old
database but could not create the replacement schema. Ghost reported the real
problem clearly: No space left on device.
This was inconvenient but safe. Public traffic still went to AWS, the Incus snapshot still existed, and the source archives were preserved elsewhere. The large staged content archive inside the container had already been extracted and existed in two verified copies outside it, so removing that one redundant file recovered about 612 MiB.
The second database import progressed slowly over Ceph-backed storage, table by table, but completed. Ghost then passed through its normal database and URL service initialization and finally reported that it had booted. The restored homepage returned HTTP 200, its RSS feed contained all ten entries, and a representative image loaded through the public Caddy path.
The container was healthy, although 94 percent disk usage was a clear follow-up item rather than a number to admire. Pruning unused Docker images and package caches later returned about 2.3 GiB without touching application data.
The path home
The active request path is now:
reader
-> Route 53
-> caddy-external
-> Tailscale
-> leah-blog Incus container
-> Ghost 6 and MySQL 8
Public DNS changed from the Cape Town EC2 address to the external Caddy edge.
The split BIND zone changed separately so tailnet clients reach that Caddy
server over its Tailscale address. I validated the Caddyfile before cutover,
validated the staged BIND zone before and after installation, incremented the
zone serial, restarted only named, and checked the site through both paths.
The AWS change reached INSYNC, external resolvers returned the new address,
and the home origin served the same ten RSS items that I had measured on AWS.
The temporary AWS recovery resources were deleted. I kept the original EC2
instance running during the settling period, then stopped it and repeated the
public homepage, RSS, and image checks. After creating a fresh on-prem backup
and post-migration Incus snapshot, I terminated the instance and removed its
retained volume, Elastic IP, security group, and key pair. Route 53 remains,
but the blog no longer has a compute or storage footprint in AWS.
What this small migration reinforced
This was not a large site, but small sites still contain irreplaceable state. The useful lessons were the same ones that apply to much larger systems:
- A powered-off old server is context, not proof that its data is current.
- Spot instances are disposable compute; a stateful application needs its data to survive independently of them.
- Compare source and destination content before changing traffic.
- Checksums turn an interrupted transfer from a hunch into a fact.
- Keep the public source online until the replacement passes end-to-end tests.
- Snapshot the destination before destructive work, even when application backups already exist.
- Treat split DNS as part of the deployment, not as a private afterthought.
- Read the exact SSH behavior before deciding that a key is wrong.
- Leave an obvious rollback path and write down how to use it.
There is also a less technical lesson. The original home container was created as a demo and preserved because deleting it offered no immediate benefit. Four months later, that modest decision made the return trip much easier. The container already had the right application shape, tailnet identity, and proxy route. It only needed the story it had missed while it was asleep.
Leah’s blog went to South Africa, did what it was built to do, and came home with all ten entries intact.