How to Self-Host your own Node.js Website

Amazon, Microsoft and Google control too much of the Internet. If you're an employed web engineer chances are almost 100% that you're renting web hosting services from one or more of these companies at work. You will own nothing and be happy (at work)! But at home you have choice over where you host your web presence. If you have personal sites or side projects, you may want to consider self-hosting these on your home computer. It's easy, free, and fun. You will learn a lot by being your own sysadmin, and I'll show you how.

In this guide I'll explain how to deploy your Node.js web app to a web server virtual machine (VM) running on a computer that you control. Our methodology will be configuring the Server VM to a "blank slate" status with network access, and then using Ansible over ssh to completely automate the process of installing and configuring your web project. For this to work you just need the following:

An Internet Service Provider that can assign you a Static IP address
A router that can do port forwarding
A computer that stays awake and connected to the Internet

If this sounds intriguing, read on and I'll show you how deep the rabbit-hole goes.

Step 1: Get a static IP address from your Internet Service Provider

Most ISPs can assign you a Static IP address if you ask nicely or pay them extra for "Business Grade Internet." This usually involves a sizable upcharge, but often brings several other benefits as well.

Over the years I've purchased Business Internet from several ISPs (Comcast, AT&T and Spectrum) as I've moved around, and one seemingly universal benefit is they don't play games with the pricing. The price is the price, not a six month promotional price after which they jack it way up and you have to call them begging to lower it or threatening to cancel.

Another benefit isn't universally applicable, but many ISPs are capping monthly bandwidth for consumer accounts and charging overage fees if you download beyond some arbitrary number of gigabytes or terabytes. None of the Business Internet plans I've seen have any bandwidth limitations.

So, yeah, it's annoying paying more for Business Internet, but isn't it also annoying paying rent to Amazon, the perverted company that spied on women through their home security systems? I'll let you be the judge of that, but get yourself a Static WAN IP and we can continue.

Step 2: Install Ubuntu Server in a Virtual Machine

OK you're free to use any Linux you want but this guide assumes you'll have the apt package manager. Really it's not such a big deal though.

If you're on Windows or Linux you can use the free Oracle Virtualbox to host your Virtual Machines. On a Mac you can rent the privilege of using Parallels for Virtual Machining (get the "Pro" subscription.)

Once you have your Virtual Machine host installed, download Ubuntu Server 22.04 LTS (or get the ARM Version if you're on an M1/M2 Mac). Create a new virtual machine for your server, and I'd recommend the following settings before proceeding with installation:

IMPORTANT: Use a bridged network adapter to give the VM its own IP address on your local network.
Assign up to half the CPU cores of your computer (so if you have an 8 core processor, give 4 cores to your server VM)
Assign 8gb of memory if you have 32gb+, otherwise 4gb
Initialize your storage with 64gb of disk space (or more)

NOTE: Assigning half your CPU cores to the Virtual Machine won't make them unusable in your Host operating system - your OS scheduler will distribute work across all your cores, regardless of whether the work comes from the VM or your Host OS. Assigning half just prevents the server from biting too much into your Host OS if things get busy.

NOTE 2: On the other hand, assigning memory to the VM typically takes it away from the Host OS, so if you're doing other stuff on this computer make sure to only assign as much memory as you can tolerate living without in your day-to-day usage. But obviously you'll need to give the VM enough RAM to host your web app, so download more RAM if you need to.

Step 3: Configure your Server VM to a "blank slate" status

Now we'll configure your server to a minimal required configuration to be able to safely connect to it over SSH. We will not install the packages to run your web app at this point; rather, we will use this "blank slate" configuration as a starting point to allow Ansible to fully automate the process of installing your dependencies and running your web app.

First we will want to install the "Guest Additions" software for better integration between your Virtual Machine Guest and the Host OS:

Linux or Windows: Install VirtualBox Guest Additions
1. If your installation DVD is still mounted to the virtual CD-ROM drive, enter eject /mnt/cdrom.
2. Using the Devices menu, select "Insert Guest Additions CD Image..."
3. sudo mount /dev/cdrom /mnt/cdrom (you may first need to sudo mkdir /mnt/cdrom if it doesn't exist).
4. cd /mnt/cdrom
5. sudo ./autorun.sh (go through the installation)
6. sudo reboot
MacOS: Install Parallels Tools
1. If your installation DVD is still mounted to the virtual CD-ROM drive, enter eject /dev/cdrom.
2. Using the menu, connect the Parallels Tools ISO to the Virtual Machine.
3. sudo mount /dev/cdrom /media/cdrom (you may first need to sudo mkdir /media/cdrom if it doesn't exist).
4. cd /media/cdrom
5. sudo ./install (go through the installation)
6. sudo reboot (may have to do this twice)

NOTE: If you're using Windows as your Host OS you should really install Linux on Windows with WSL. Using WSL will allow us to run the same Linux tools in Windows (such as ssh or Ansible) that we rely on in MacOS and Linux.

Next, we'll configure the SSH daemon on your Server VM to have better security than the default configuration. By default the SSH daemon allows password authentication, which exposes your server to brute force password attacks if you choose to forward its SSH port on the open Internet (this is a very bad idea unless you make the changes I'm about to explain). We will turn off password authentication and instead use SSH public key authentication, which is much more secure. I'm assuming you already have an SSH public key. If you don't, you can use this guide to configure SSH on your Host computer, but not already knowing this could also mean that you're in over your head and running a web server on your computer might be a bad idea.

To configure your Server VM's SSH daemon properly, follow these steps:

(On your VM) in terminal, enter ip addr, note the IP address for your bridged network interface.
(On your Host) Copy your SSH public key contents to clipboard (eg. cat ~/.ssh/id_rsa.pub, highlight the text, and copy). Then SSH into your Server VM, eg. ssh USERNAME@[BRIDGED IP ADDRESS] — enter your password for your Server user account when prompted.
(In your SSH session)
- mkdir ~/.ssh (if needed)
- nano ~/.ssh/authorized_keys to open a text editor
- Paste the contents of your Host's SSH public key into its own line in the nano editor
- Save the file (CTRL+O) and exit (CTRL+X)
- Now edit the Server's SSH Daemon config: sudo nano /etc/ssh/sshd_config
- Use the arrow keys to scroll down the file until you see a bit about "PasswordAuthentication". Change it such that this line will not be commented out and the value should be set to no. Eg:
```
# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no
#PermitEmptyPasswords no
```
- Extra credit: Change to a non-standard SSH port! (that's out of scope for this guide ;)
- Save the file (CTRL+O) and exit (CTRL+X).
- Restart the SSH Daemon: sudo /etc/init.d/ssh restart

If all goes well your ass gets booted out of your SSH session and the next time you try to SSH into the Server VM you get prompted for your SSH key passphrase (Success!)

Next, we'll configure a static LAN IP address so your router doesn't reassign your local IP address to some other device on your network.

You should pick a LAN IP address on the same subnet, preferably towards the end of the DHCP assignment range (or beyond). So for example, if your router's IP address is 192.168.1.1 it might make sense to pick 192.168.1.249 as your server's static LAN IP.

To set your static LAN IP, follow these steps on your Server VM:

cd /etc/netplan
ls (take note of the filename of your LAN config)
sudo nano FILENAME (replace FILENAME with the actual filename)
Take note of the name of the network adapter listed under ethernets:, eg. enp0s5.
Delete everything out of the file. Hold the Delete key or hit CTRL+K a bunch of times.

Paste the following, be sure to replace {NETWORK ADAPTER} with the name of the network adapter, {DESIRED LAN IP} with the IP address you want to use, and {ROUTER IP} with your router's IP:

# This is the network config written by 'subiquity'
network:
renderer: networkd
ethernets:
 {NETWORK ADAPTER}:
   dhcp4: no
   addresses: [{DESIRED LAN IP}/24]
   gateway4: {ROUTER IP}
   nameservers:
     addresses: [8.8.8.8,8.8.4.4]

Save the file and exit with CTRL+O and CTRL+X
sudo netplan apply (note that this may kick you off SSH again)
wget https://www.google.com to test that you have Internet access with the new settings.

Finally, just to be safe we'll set up a firewall.

Ubuntu and most modern Linux distros come with ufw, the Uncomplicated Firewall. With this, we can make sure only the ports you want open are available to the local network (and eventually the Internet), and it's a breeze to set up. Just enter the following commands on your Server VM:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22 (if you chose a non-standard SSH port, substitute it here)
sudo ufw allow 443
sudo ufw allow 80
sudo ufw enable

A quick review...

We've installed Ubuntu Server (or your Linux distro of choice), installed the Guest Additions packages from your VM Host, set up some sane SSH daemon settings, picked a static LAN IP address, and turned on the firewall. Now we're just about ready to install your web app on this VM, but first...

Step 4: Take a VM snapshot of your "blank slate" server config

In the next steps, we'll be using Ansible from your Host OS to install all of the packages and config files necessary to make your site run. But initially this is a process of trial and error. By taking a snapshot of your "blank slate" server config, you can easily nuke and rollback your changes in the event that you screw something up along the way, thus wiping out any potentially destructive environment changes.

The nice thing about using a Virtual Machine is that basically every VM Host supports snapshots, and rolling back to a previous good snapshot can be invaluable if you screw something up later on.

Every VM host handles snapshots slightly differently, but they all make it easy, so I leave it to you, the reader, to make a snapshot of this clean install prior to the messing around we're about to embark on.

Step 5: Install and configure Ansible

NOTE: Please refer to the example playbooks at https://gitlab.com/obsessivefacts/obsessivefacts-ansible/

Ansible is the tool we'll use to deploy your web project to your empty virtual machine. Ansible allows you to create scripts known as playbooks that define a series of commands you want to run on your server, over SSH. It can copy files, access databases, manage filesystem permissions, and all sorts of other cool things. With Ansible playbooks you can bring your server VM from its "clean slate" status to a fully functioning web server with your app hosted on it, with minimal manual intervention.

To install Ansible, simply:

sudo apt install ansible in Linux
...or brew install ansible on MacOS (assuming you've already installed the Homebrew Package Manager).

Once Ansible is installed, we will create a hosts configuration file to point it at your server.

sudo mkdir /etc/ansible (if it doesn't exist — and for MacOS Homebrew this would likely be mkdir /usr/local/etc/ansible).
Within this folder create and edit a file called hosts, and make it look like this:
```
[server]
192.168.1.249:22

[mycoolwebapp]
192.168.1.249:22
```

^ Keep in mind that the IP address and port should be the actual LAN IP of your server and the port you're running the SSH daemon on. With this configuration we've created two Ansible groups: one called server which we will use to set up some basic system-wide packages for the server (things like nginx or PostgreSQL, etc); the other group is mycoolwebapp which is a placeholder for whatever cool web app you want to install on this server.

Step 6: Create Ansible playbooks to install and host your web app

Alright, here's where things get a bit hand-wavey. At the beginning of this article I promised to show how to install a Node.js app, but really you can use Ansible to configure any kind of server: Ruby, PHP, Macromedia ColdFusion, whatever. Since every app is going to be different, this is where your trial and error should begin. But at a high level, in order to host a Node.js web app on a domain like www.mycoolwebapp.com, we will do something like the following:

Install nginx web server to listen for incoming connections to www.mycoolwebapp.com
Install pm2 to run your Node.js app on multiple CPU cores
Configure nginx reverse proxy to pass those incoming connections to your Node.js app on pm2
(Maybe) Set up a database while we're at it.

All of this can be completely automated through the use of Ansible playbooks. You should probably take the time to read the Ansible playbooks documentation, but I'll try to give a focused and brief summary.

NOTE: I've created some example Ansible playbooks that you could use to install the Obsessive Facts web site on another server in the event that I get Epstein'ed. So look these over on GitLab if you get confused about anything.

In the previous section, we created two groups, server and mycoolwebapp. Now we can create playbooks for both of these groups to actually do useful stuff. You can put your playbooks anywhere on your computer where it's convenient to run scripts. Ansible playbooks have an innate file and directory structure, and it looks a bit like this:

# Example Ansible Playbooks directory structure!

- .env

- mycoolwebapp.yml
> mycoolwebapp
  > files
    - goatse.jpg
    - tubgirl.jpg
  > handlers
    - main.yml
  > tasks
    - main.yml
  > templates
    - mycoolwebapp.config.js.j2

- server.yml
> server
  > files
    - nginx.conf
  > handlers
  > tasks
    - main.yml
  > templates

In the top directory you have .yml files for each of the groups we defined in the Ansible hosts file (in this example server.yml and mycoolwebapp.yml), and for each of these we have a named subdirectory with (up to) all of the following four sub-sub directories: files, handlers, tasks, and templates.

The subdirectories for each group handler are as follows:

files: files you want copied over to your server verbatim
handlers: (see the docs) functionality to only be run if Ansible makes a change
tasks: the "meat" of your Ansible playbook. The main.yml task file will perform a number of tasks on your Server VM and access the files, templates, and handlers as needed.
templates: like files, but they can have environment-specific variable data substituted into them via the jinja2 templating system. So if your environment has export ROOT_PASSWORD=abcd1234, you could create a mycoolwebapp.config.js.j2 file with const rootPassword = '{{ lookup('env','ROOT_PASSWORD') }}'; and Ansible will substitute in the environment data for this ultra-secure root password as needed.

But first you will need a .env file which (likely) contains secret information that you don't want to bake directly into your playbooks, like API keys, database passwords, etc. This file will look a bit like this:

# make sure this file is executable (`chmod +x .env`)
# and then run it before you run your playbooks (`source .env`)

export MYCOOLWEBAPP_POSTGRESQL_DATABASE=mycoolwebapp
export MYCOOLWEBAPP_POSTGRESQL_USER=username
export MYCOOLWEBAPP_POSTGRESQL_PASSWORD=abcd1234
export MYCOOLWEBAPP_WWW_DOMAIN=www.mycoolwebapp.com
export MYCOOLWEBAPP_LETSENCRYPT_ACCOUNT_ID=123456790abcdef

If you decide to put your playbooks in a public Git repository then definitely make sure your .env file is added to .gitignore and kept private!

Create an Ansible playbook for your server's main configuration.

This is a good practice if your server is hosting multiple web apps that each have common dependencies. You can pre-install global dependencies in your server playbook and then manage app-specific dependencies in separate playbooks for those particular apps.

Make a file called server.yml. It should refer to the server group you created in the Ansible hosts configuration, eg:
```
---
- hosts: server
  become: true
  roles:
   - server
```
Create a directory called server; this will contain the "guts" of the server playbook.
Within your server directory create a subdirectory called files
Within your server directory create a subdirectory called tasks
Within server/tasks/ create a file called main.yml. This is where the magic happens.

Obviously your server config is going to be different from mine, but you can take a look at my example main.yml on GitLab and get an idea of what might make sense for you. You should probably read the Ansible docs, but I'll call out some useful bits and pieces below.

Update apt and install nginx and PostgreSQL — read more about Ansible's apt module

- name: Update apt
  apt: update_cache=yes

- name: Ensure package nginx exists
  apt: name=nginx state=present

- name: Ensure PostgreSQL exists
  apt: name=postgresql state=present

Installing a specific version of Node.js — see NodeSource on Github for more info.

- name: Add Nodesource GPG key
  ansible.builtin.apt_key:
    url: https://deb.nodesource.com/gpgkey/nodesource.gpg.key
    state: present

- name: Install Node 18 Sources 
  ansible.builtin.apt_repository:
    repo: deb https://deb.nodesource.com/node_18.x jammy main
    state: present

- name: Install Node 18 Dev Sources
  ansible.builtin.apt_repository:
    repo: deb-src https://deb.nodesource.com/node_18.x jammy main
    state: present

Copy nginx.conf from the files directory you definitely created — read more about Ansible's copy module

- name: Copy nginx.conf from files and set permissions
  copy: src=nginx.conf dest=/etc/nginx/nginx.conf mode=700 owner=root group=root

Make sure the /etc/nginx/ssl directory exists — read more about Ansible's file module

- name: Ensure /etc/nginx/ssl directory exists
  file: dest=/etc/nginx/ssl mode=700 owner=root group=root state=directory

PROTIP: Install the ACL apt package to prevent ansible errors when su'ing to other user accounts in Ubuntu 22.04+

- name: Ensure package ACL is installed
  apt: name=acl state=present

These are all just some examples of stuff you can do to get you started. One great thing about Ansible's execution behavior is that it's "idempotent", meaning it won't make changes if it doesn't need to. So in the example above, if Ansible detects that your specific nginx.conf already exists on the server, it won't copy again or do anything destructive — the playbook will simply skip over that step.

Create an Ansible playbook for your specific app.

Once you've determined what system-wide packages to install, you can create a playbook specific to your app. It's the same process, but I'll touch on Ansible's templating functionality, which allows you to pull stuff like database passwords out of your environment variables and bake them into config files as they get copied.

In keeping with the example configurations above, create a file called mycoolwebapp.yml in your playbooks directory. Again it will refer to the mycoolwebapp group you created in the Ansible hosts configuration, eg:
```
---
- hosts: mycoolwebapp
  become: true
  roles:
   - mycoolwebapp
```
Create a directory called mycoolwebapp
Create a directory called mycoolwebapp/templates; here we'll store files that Ansbile can modify in-transit using the jinja2 templating system
Create a directory called mycoolwebapp/tasks
Within mycoolwebapp/tasks/ create your main.yml file.
Within mycoolwebapp/templates/ create an empty file mycoolwebapp.config.js.j2 — we'll get to this in a second.

Configure main.yml to use your template file — read more about Ansible templating

- name: Copy templated mycoolwebapp.config.js and set permissions
  template: src=mycoolwebapp.config.js.j2 dest=/htdocs/mycoolwebapp.config.js owner=www-data group=www-data mode=700

Use jinja2 templating syntax to pull data from your environment variables into mycoolwebapp.config.js.j2

// mycoolwebapp secret config file
module.exports = {
    db: {
        db: '{{ lookup('env','MYCOOLWEBAPP_POSTGRESQL_DATABASE') }}',
        user: '{{ lookup('env','MYCOOLWEBAPP_POSTGRESQL_USER') }}',
        pass: '{{ lookup('env','MYCOOLWEBAPP_POSTGRESQL_PASSWORD') }}',
        options: {
            host: 'localhost',
            dialect: 'postgres',
            logging: false,
            operatorsAliases: 0
        }
    }

You can also perform template substitutions within your tasks/main.yml file!

- name: Ensure database exists
  become: yes
  become_user: postgres
  postgresql_db: name={{ lookup('env','MYCOOLWEBAPP_POSTGRESQL_DATABASE') }}

Okay, now just make your playbook copy all required files, install the code for your web site using Ansible's built-in git module, perform necessary changes to your server config, and you're done!

Really we're at a point where you need to figure out what works for your specific app. If you feel lost, take a look at the playbook for the Obsessive Facts web site. Pick and choose and adapt the stuff in there for your use, and if there's something else you need, chances are good that Ansible already provides it, either via one if its built-in modules, or an add-on.

Step 7: Run your Ansible playbooks and your server installs itself!

Assuming you created your playbooks perfectly the first time and nothing goes wrong, you would execute the following commands and your empty virtual machine will become a fully functioning web server with your app running on it!

source .env — to pull your secret stuff into the environment
ansible-playbook server.yml --ask-become-pass — install your server
ansible-playbook mycoolwebapp.yml --ask-become-pass — install your app

The --ask-become-pass argument allows Ansible to prompt you for your account password on the Server VM so that it can use sudo and su to execute commands on behalf of other user accounts used by daemons on your server (eg the www-data user for nginx)

Step 8: If something goes wrong, nuke your server VM from orbit.

If this is your first time working with Ansible, chances are good that something will go wrong. A lot of things will go wrong. Maybe you ran the playbooks in the wrong order and ended up installing the wrong version of Node.js. Maybe you accidentally the whole thing. No worries, no judgement.

If your Server VM gets to a broken or "dirty" state, nuke it and restore the "clean slate" snapshot you took right after doing the initial configuration. Then, sort out the problems in your Ansible playbooks and keep trying until things go perfectly.

The whole point of Ansible is to allow you to configure your server quickly from a clean starting point. Setting things up this way allows you to easily move your app to a new server if and when the need arises. So if you have to do too much manual futzing to fix things on the server that your playbooks missed, then you're kind of missing the point. Of course some minimal futzing will still be inevitable, and we'll get to that in a bit.

Step 9: Forward ports on your router to connect the Internet!

Assuming you got things running properly, your server is ready to accept incoming connections from THE INTERNET. Now all you need to do is forward the relevant ports from your router to your server's LAN IP address. Normally this would be Port 80 for HTTP and Port 443 for HTTPS. You might also consider forwarding your SSH port, but even with SSH password authentication turned off that's like asking for trouble. If you really want to SSH in from outside your LAN, please run the SSH daemon on a non-standard port.

Once you have your ports forwarded you should be able to visit your web server by simply entering your Static IP address (from your ISP) into the URL bar of any web browser.

Step 10: Point the DNS A record(s) for your domain to your server

Now you can point the A record(s) for your domain to your Server. You will want to point the domain root (@) record at your IP address, along with any subdomains (such as www) that you want to host on your server. This is also necessary in order to set up SSL with Certbot, which we'll cover shortly.

Step 11: Enable SSL and make it persist to future Ansible deploys

Once your server is listening to the Internet and your domain is pointing at your server you can enable SSL by generating a certificate with Certbot. To do this, you'll have to SSH into your server, install Certbot and then NOT follow the instructions on the Certbot website. Because there is kind of a chicken-and-egg problem here.

The thing is, if you follow the default path and allow Certbot to directly edit your nginx config after the fact, then you'll have to manually fix SSL any time you deploy changes to your nginx config (since the nginx templates you have in Ansible won't have the stuff that Certbot adds to your configuration). You could copy the Certbot-modified nginx configs from your server into your Ansible playbook and use them going forward, but you'd still lose your certificate if you ever do a "clean slate" install again.

So this is kind of a hack, but what we can do is run Certbot manually to generate certificates for your domain and subdomains, copy all of Certbot's generated files and configs in to our Ansible files, and then use Ansible to deploy these files back out to the server, along with modified nginx configs enabling SSL. Basically we'll use Ansible to forcibly recreate Certbot's state if it gets lost, which means we won't have to manually configure SSL again on a "clean slate" install. This is a bit weird but #yolo.

So to actually do this:

SSH into your server
Install Certbot
STOP your running nginx server (sudo service nginx stop) to free up Port 80. Certbot will temporarily run a web server on this port to do the certificate signing challenge.
Run sudo certbot certonly. You will go through a series of prompts similar to this (be sure to select option 1):

Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): obsessivefacts@protonmail.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
Account registered.
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): obsessivefacts.com,www.obsessivefacts.com
Requesting a certificate for obsessivefacts.com and www.obsessivefacts.com

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/obsessivefacts.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/obsessivefacts.com/privkey.pem
This certificate expires on 2023-09-07.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot
  can automatically renew the certificate in the background, but you
  may need to take steps to enable that functionality.
  See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

(At this point can turn nginx back on for now: sudo service nginx start)

So we've generated a certificate for your domain, but one thing we didn't do here is choose the specific options to automatically mangle our nginx configs. We'll want to do that manually, so we can incorporate it into our Ansible process. But because of this "going solo" route, Certbot didn't write two key files that we'll need in order to use this certificate with nginx. We will now grab these files directly from Certbot's GitHub and place them in /etc/letsencrypt:

cd /etc/letsencrypt

Get options-ssl-nginx.conf:

sudo wget https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf

Get ssl-dhparams.pem:

sudo wget https://raw.githubusercontent.com/certbot/certbot/master/certbot/certbot/ssl-dhparams.pem

So now let's poke around and look at our directory structure for /etc/letsencrypt:

# /etc/letsencrypt

> accounts
  > acme-v02.api.letsencrypt.org
    > directory
      > 1234567890abcdef             # (actually your unique Let's Encrypt account)
        - meta.json
        - private_key.json
        - regr.json
> archive
  > YOURDOMAIN.COM                   # (this would actually be your domain)
    - cert1.pem
    - chain1.pem
    - fullchain1.pem
    - privkey1.pem
> csr
  - 0000_csr-certbot.pem
> keys
  - 0000_key-certbot.pem
> live
  > YOURDOMAIN.COM
    ◦ cert.pem                       # (these...
    ◦ chain.pem                      # ...pem files...
    ◦ fullchain.pem                  # ...are symbolic links...
    ◦ privkey.pem                    # ...to ../../archive)
  - README
- options-ssl-nginx.com
> renewal
  - YOURDOMAIN.COM.conf
> renewal-hooks
  > deploy
  > post
  > pre
- ssl-dhparams.pem

So this is going to be totally super annoying, but we'll have to copy all of these files to the files directory of your web project's Ansible playbook:

meta.json
private_key.json
regr.json
cert1.pem
chain1.pem
fullchain1.pem
privkey1.pem
0000_csr-certbot.pem
0000_key-certbot.pem
options-ssl-nginx.com
YOURDOMAIN.COM.conf (located in the renewal directory)
ssl-dhparams.pem

Then we'll instruct Ansible to ensure that Certbot is installed and if needed recreate this entire directory structure while preserving all permissions and maintaining symbolic links.

So in your web project's Ansible tasks/main.yml, we will add something like this:

- name: Ensure letsencrypt is installed (along with Certbot)
  apt: name=letsencrypt state=present

# INSTALL LET'S ENCRYPT CERT -- BY FORCE!

- name: Ensure /etc/letsencrypt directory exists
  file: dest=/etc/letsencrypt state=directory owner=root group=root mode=755

- name: Ensure /etc/letsencrypt/accounts directory exists
  file: dest=/etc/letsencrypt/accounts state=directory owner=root group=root mode=700

- name: Ensure /etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org directory exists
  file: dest=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org state=directory owner=root group=root mode=700

- name: Ensure /etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory directory exists
  file: dest=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory state=directory owner=root group=root mode=700

- name: Ensure /etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/LETSENCRYPT_ACCOUNT directory exists
  file: dest=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/{{ lookup('env','OBSESSIVEFACTS_LETSENCRYPT_ACCOUNT_ID') }} state=directory owner=root group=root mode=700

- name: Copy meta.json to our letsencrypt account directory
  copy: src=meta.json dest=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/{{ lookup('env','OBSESSIVEFACTS_LETSENCRYPT_ACCOUNT_ID') }} owner=root group=root mode=644

- name: Copy private_key.json to our letsencrypt account directory
  copy: src=private_key.json dest=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/{{ lookup('env','OBSESSIVEFACTS_LETSENCRYPT_ACCOUNT_ID') }} owner=root group=root mode=400

- name: Copy regr.json to our letsencrypt account directory
  copy: src=regr.json dest=/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory/{{ lookup('env','OBSESSIVEFACTS_LETSENCRYPT_ACCOUNT_ID') }} owner=root group=root mode=644

- name: Ensure /etc/letsencrypt/archive directory exists
  file: dest=/etc/letsencrypt/archive state=directory owner=root group=root mode=700

- name: Ensure /etc/letsencrypt/archive/obsessivefacts.com directory exists
  file: dest=/etc/letsencrypt/archive/obsessivefacts.com state=directory owner=root group=root mode=755

- name: Copy cert1.pem to /etc/letsencrypt/archive/obsessivefacts.com
  copy: src=cert1.pem dest=/etc/letsencrypt/archive/obsessivefacts.com owner=root group=root mode=644

- name: Copy chain1.pem to /etc/letsencrypt/archive/obsessivefacts.com
  copy: src=chain1.pem dest=/etc/letsencrypt/archive/obsessivefacts.com owner=root group=root mode=644

- name: Copy fullchain1.pem to /etc/letsencrypt/archive/obsessivefacts.com
  copy: src=fullchain1.pem dest=/etc/letsencrypt/archive/obsessivefacts.com owner=root group=root mode=644

- name: Copy privkey1.pem to /etc/letsencrypt/archive/obsessivefacts.com
  copy: src=privkey1.pem dest=/etc/letsencrypt/archive/obsessivefacts.com owner=root group=root mode=600

- name: Ensure /etc/letsencrypt/csr directory exists
  file: dest=/etc/letsencrypt/csr state=directory owner=root group=root mode=755

- name: Copy 0000_csr-certbot.pem to /etc/letsencrypt/csr
  copy: src=0000_csr-certbot.pem dest=/etc/letsencrypt/csr owner=root group=root mode=644

- name: Ensure /etc/letsencrypt/keys directory exists
  file: dest=/etc/letsencrypt/keys state=directory owner=root group=root mode=700

- name: Copy 0000_key-certbot.pem to /etc/letsencrypt/keys
  copy: src=0000_key-certbot.pem dest=/etc/letsencrypt/keys owner=root group=root mode=600

- name: Ensure /etc/letsencrypt/live directory exists
  file: dest=/etc/letsencrypt/live state=directory owner=root group=root mode=700

- name: Ensure /etc/letsencrypt/live/obsessivefacts.com directory exists
  file: dest=/etc/letsencrypt/live/obsessivefacts.com state=directory owner=root group=root mode=755

- name: Create symbolic link to cert1.pem
  file: src=/etc/letsencrypt/archive/obsessivefacts.com/cert1.pem dest=/etc/letsencrypt/live/obsessivefacts.com/cert.pem state=link

- name: Create symbolic link to chain1.pem
  file: src=/etc/letsencrypt/archive/obsessivefacts.com/chain1.pem dest=/etc/letsencrypt/live/obsessivefacts.com/chain.pem state=link

- name: Create symbolic link to fullchain.pem
  file: src=/etc/letsencrypt/archive/obsessivefacts.com/fullchain1.pem dest=/etc/letsencrypt/live/obsessivefacts.com/fullchain.pem state=link

- name: Create symbolic link to privkey.pem
  file: src=/etc/letsencrypt/archive/obsessivefacts.com/privkey1.pem dest=/etc/letsencrypt/live/obsessivefacts.com/privkey.pem state=link

- name: Copy options-ssl-nginx.conf
  copy: src=options-ssl-nginx.conf dest=/etc/letsencrypt owner=root group=root mode=755

- name: Ensure /etc/letsencrypt/renewal directory exists
  file: dest=/etc/letsencrypt/renewal state=directory owner=root group=root mode=755

- name: Copy obsessivefacts.com.conf to /etc/letsencrypt/renewal
  copy: src=obsessivefacts.com.conf dest=/etc/letsencrypt/renewal owner=root group=root mode=644

- name: Ensure /etc/letsencrypt/renewal-hooks directory exists
  file: dest=/etc/letsencrypt/renewal-hooks state=directory owner=root group=root mode=755

- name: Ensure /etc/letsencrypt/renewal-hooks/deploy directory exists
  file: dest=/etc/letsencrypt/renewal-hooks/deploy state=directory owner=root group=root mode=755

- name: Ensure /etc/letsencrypt/renewal-hooks/post directory exists
  file: dest=/etc/letsencrypt/renewal-hooks/post state=directory owner=root group=root mode=755

- name: Ensure /etc/letsencrypt/renewal-hooks/pre directory exists
  file: dest=/etc/letsencrypt/renewal-hooks/pre state=directory owner=root group=root mode=755

- name: Copy ssl-dhparams.pem
  copy: src=ssl-dhparams.pem dest=/etc/letsencrypt owner=root group=root mode=755

Umm, well that's a lot (and you will have to tweak it to use your own domain and Lets Encrypt Account ID environment variable). But assuming you copied the files properly into your Ansible playbook, it will work. The next time you run your playbook, Ansible will forcibly install and configure your Let's Encrypt certificate, even on a "clean slate" server install.

Alright, now that the certificate is installed, you will want to add it to the nginx config for your site. Mine looks a bit like this:

server {
    # Enforce the use of HTTPS (root domain)
    listen 80;
    server_name obsessivefacts.com;
    server_tokens off;

    listen 443 ssl;
    ssl_certificate /etc/letsencrypt/live/obsessivefacts.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/obsessivefacts.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    return 302 https://www.obsessivefacts.com$request_uri;
}

server {
    # Redirect HTTP www subdomain to HTTPS
    listen 80;
    server_name www.obsessivefacts.com;
    server_tokens off;
    return 302 https://www.obsessivefacts.com$request_uri;
}

server {
    server_name  www.obsessivefacts.com;

    access_log  /var/log/nginx/www.obsessivefacts.com-access.log;
    error_log /var/log/nginx/www.obsessivefacts.com-error.log;

    charset utf-8;
    server_tokens off;

    client_max_body_size 50M;

    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header  X-Real-IP $remote_addr;
        proxy_set_header  X-Forwarded-For $remote_addr;
        proxy_pass http://localhost:9001;
     }

    listen 443 ssl;
    ssl_certificate /etc/letsencrypt/live/obsessivefacts.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/obsessivefacts.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}

Okay, so you've got Ansible deploying your Let's Encrypt SSL certificate and using the certificate to enable HTTPS on your web app. The only remaining thing to do is to set up a cron job that renews your certificate if it's expiring soon. We can do this with two tiny shell scripts:

/etc/cron.daily/renew-certbot-if-needed.sh

#!/bin/sh
certbot renew -q

/etc/letsencrypt/renewal-hooks/post/reload-nginx.sh

#!/bin/sh
service nginx reload

This will perform a daily check to see if it's time to renew your SSL cert, and if so, Certbot will automatically perform the renewal and call the reload-nginx.sh script to tell nginx to reload with your new certificate.

Put these files in your Ansible playbook, and deploy them like so:

- name: Enable cron job to renew letsencrypt cert when needed
  copy: src=renew-certbot-if-needed.sh dest=/etc/cron.daily owner=root group=root mode=755

- name: Enable post-renewal hook to reload nginx when the certificate finished updating
  copy: src=reload-nginx.sh dest=/etc/letsencrypt/renewal-hooks/post owner=root group=root mode=755

Note: Once your certificate actually renews, you will want to copy the files for the new certificate from /etc/letsencrypt into your Ansible playbook. So maybe you'll want to set up a cron job to scp the updated keys from your server back to your development computer. For now I'll leave that as an exercise for you, the reader.

Wrapping up...

Welp, this post got a lot longer than I expected, and I made some changes to my own Ansible playbooks in the process of writing. Especially that mess about SSL. But just to keep myself honest, I actually nuked the Obsessive Facts Server and ran through all of my playbooks to make sure everything still works. (If you're reading this, it probably still works.)

I'm not claiming to be an Ansible expert; this is just what works for me, based on habits I've established, and I know there are better ways to do some of the stuff here (like using Ansible Variables instead of environment variables). Please do let me know in the comments if I got something wrong or dangerously bad!

If you got this far, you're well on your way to hosting your own web app and taking a small slice of the Internet back from Big Tech!

Further reading:

←blog