On spending too long setting up one webserver

First, let’s step back a bit.

After running The Diary of Samuel Pepys for ten years I had the bright idea of rewriting the site and doing it again. So in 2013 the new version, written using Django, appeared. The website was hosted on Heroku, one of whose benefits is simplifying the task of running a webserver. It can become expensive but for my simple needs it cost US$9 per month (for a database).

Last year Heroku re-jigged their pricing, understandably, and so I’d now have to pay for not only the database but also serving the site itself. The monthly cost would rise to $16 per month. It’s not a huge amount but it was a nudge and I wondered about alternatives.

As a comparison all of my other self-hosted sites are hosted on a Webfaction “shared hosting” account for a total of $10 per month. I could probably have moved Pepys there but this seemed like a good chance to learn something new.

I decided to try setting up a virtual server at DigitalOcean. This is a bit more like a “real” webserver, without Heroku hiding the grubby bits beneath the hood. Consequently it requires more work to get going. But it’s cheaper: the smallest option at DigitalOcean only costs $5 per month. It’s more time consuming and there’s more scope for things to go wrong. But this was a chance to learn more about part of the “full stack” of web development that I’ve usually avoided.

Thankfully, there are lots of instructions for doing this kind of thing, and I was able to follow these to set up a server and get my Django site working. It took about a day of carefully following along and making notes but, almost uniquely in my experience, the process worked.

I was amazed. I’d learned useful new things. I’d saved $11 per month. Imagine the celebrations.

However.

While I had a new, live, production website, I also needed to write new code on a different version of the site. Django makes it easy to run a local version of your code for development, which is fine but not ideal — my Mac is a different environment to a DigitalOcean virtual server. It’d probably be OK for my own decreasingly popular personal projects but I like to do things “properly”.

Them: You have an interesting CV, Phil. Now then, what’s your biggest weakness?

Me: Well… I like to do things properly.

I always like to try and find the very best, the most “proper” way to do something.

When I’m doing work for clients this tendency meets its natural enemy: deadlines. Working out how to do a new thing the best way can take a while. And while I, of course, like to do client work well, the pressures of time always mean balancing “properly” with “good enough”. This is the real world and the reason anything gets done.

When I’m doing work for myself I am not in the real world. I rarely have set deadlines — I’ll never finish all of my personal projects anyway, so I may as well take the time to find the best way to do things. I may as well try and do things properly!

This is satisfying, and involves learning a lot more, but it takes longer and the end, visible, result is often little different to a “good enough”, “it seems to work” solution.

For a development version of Pepys I, of course, wanted to do things properly — I wanted to mimic the live server as closely as possible. I’d used Vagrant before — it lets you run virtual computers on your own computer — so I just needed to use it to set up a copy of my live server. Which would mean manually going through the same instructions that took a day to set up the live server. And I’d have to repeat this every time I needed a new server for the same or a different project. That seemed like it could be a lot of boring, repetitive days.

Of course, people write code to avoid boring, repetitive days, and there are many ways to automate the setting up of servers. Chef, Puppet, Ansible, SaltStack… all with their own metaphors. I liked the sound of Ansible — it didn’t sound too complicated, and I liked that you run its scripts on one server (such as your laptop) to configure a remote server.

I decided to try using Ansible to set up a Vagrant server, for local development, and to replace the new live server with a new one set up using the same script. I could even make a staging server on DigitalOcean too. Soon, with a single command I could set up as many identical servers as I liked!

Well, not that soon.

I like to do things properly.

Although DigitalOcean isn’t terribly expensive, it seemed like a waste to have an entire virtual server for Pepys. It rarely uses more than 10% of the CPU. And I imagined I might build Django sites in the future that would be smaller and even less worth their own server.

Ansible, and its ilk, are usually used to set up a single website spread over several different servers (to handle the load of a lot of traffic). I wanted to go the other way — set up several websites on a single server. This can’t be that hard, I thought. This must be possible.

I can confirm that it’s possible, and my new DigitalOcean server is live, and I have a matching Vagrant virtual machine. It’s easy to create new ones. It’s great! But, somehow, it’s now the end of March and this is almost all I’ve achieved so far this year.

This is “doing things properly”.

If I’d wanted to use Ansible to set up a server with a single Django app, that’s been done. I would, being me, have built up my own version, copying bits from existing solutions, which would have taken a while, but not too long. It would have been closer to the “it’ll probably take a couple of weeks to get to grips with Ansible” scenario that I initially imagined.

This has taken me far, far too long. Simply to replicate one webserver. If it was going to take me a whole day to manually create each new server I could have done that for many future servers and still saved time. I’ve spent several weeks inching forward, getting despondent, producing nothing visible. But, now, if I make a new Django app it can go on the same server as Pepys. So long as it’s not really popular. By spending weeks typing and swearing I’ve saved myself a hypothetical $5 per month in the future! More celebrations!

Here is my Ansible playbook which I don’t really expect anyone else to use. It has a lot of documentation because I quickly forget how to do things, so always write it down in detail.

Why was this so difficult?

Part of the problem was that I’m not that familiar with servers. So I was learning how to automate something that I know little about. I’d end up spending a lot of time puzzling over basic server administration issues. I am, now, a bit more familiar with this stuff, so that’s good.

A lot of my time was spent trying to make things work for multiple Django websites. Lots of tasks that are quite simple to automate with Ansible become trickier when wanting to do it for one or more websites. Little was impossible, but a lot was harder. A lot.

More time was taken up with making the playbook easy to reuse. More of that “doing things properly”. I wanted it to be easy to add a new Django website to the server, simply by adding a few configuration variables. Which it is, mostly. But making things easily reusable — smoothing out rough edges, thinking through how to structure things — takes a while.

A good few days early on were lost to figuring out the differences between setting up a “real” webserver (like on DigitalOcean) versus a Vagrant one. Although Vagrant mostly behaves like an actual server, there are enough differences that some Ansible things that work fine for proper servers don’t work the same, if at all, on Vagrant.

There were probably a few days sunk solely into how best to handle different Python environments. I can’t quite believe this is so difficult, but there we go. I wanted it to be possible for each Django site to have its own version of Python, its own Python modules and its own environment variables. This is not a solved problem.

(To get a little more technical for a couple of paragraphs… I’ve ended up using: pyenv to manage different python versions; virtualenv and pyenv-virtualenv to manage the discrete environments for installing modules; and autoenv for handling environment variables. It works, but it feels like a mess.

Autoenv sets environment variables OK, but doesn’t have a way of unsetting them, which makes me uneasy. I also looked at direnv and python-dotenv which had their own problems. I spent a long time struggling to get virtualenvwrapper and pyenv-virtualenvwrapper working but failed, not helped by confusing documentation. I feel like I’m missing something.)

Whenever I’m learning some new technical thing it’s a case of running blindly from one seemingly insoluble problem to another. This was no exception, probably because it involves so many different moving parts. There’s Ansible, and there’s knowing general UNIX-y stuff. And then there are the wrinkles of every piece of software that’s being installed, configured and used. Git, Nginx, Gunicorn, Supervisor, Postgresql, Memcached, Django, all that Python gubbins… I can’t pretend to have mastered that lot.

My struggles with learning any new development things are chronicled in my desperate questions on StackOverflow. I usually spend a day or so exhausting all my Googling and experimentation before I give in and compose a question in the hope an American will come along overnight and provide the answer. About 50% of the time I end up answering my own questions eventually, as if I’m continually breaking new and exciting ground like a trailblazing scientist, rather than an increasingly despairing web developer. This project has resulted in these “questions”:

• Using pyenv in Ansible
• Using a variable from one Ansible var file in a second var file
• In Ansible, how to combine variables from separate files into one array?
• Using Ansible set_fact to create a dictionary from register results
• User’s home directory owned by root when syncing a folder with Vagrant and Ansible
• In Ansible, run only one item in a loop based on a variable

There we go. I’ve no idea how much time I’ve actually spent on this. If it was full-time, five-days-a-week it must be over a month. Six weeks? Whatever, it seems ridiculous. It’s been a grind, without even the usual, eventual, reward of being able to say “TA-DA! Look at this beautiful and/or useful thing what I made!” I guess this blog post is the replacement. “TA-DA! Look at what I’ve been through, even if everything looks exactly like it did before!”

On the bright side, I’ve learned a lot! I’ve filled in some gaps in my knowledge and have a bit more idea about how to serve the websites I might build. My sysadmin skills have probably advanced from the “completely useless” level to the “almost knows enough to be dangerous” level. Which is progress.

On the other hand — and I always have a pessimistic third hand ready — I can’t even keep up with front-end development these days, never mind starting to keep up with a completely new part of the stack. I already had spinning plates crashing down around me and now I’ve found a completely new set of crockery wobbling, wobbling, wobbling away.

Commenting is disabled on posts once they’re 30 days old.