From 942fef16d99eb9ff46d4c1d119613a50961291ae Mon Sep 17 00:00:00 2001 From: Maarten de Waard Date: Tue, 15 Aug 2017 18:25:16 +0200 Subject: [PATCH] reduce readme size --- CONTRIBUTING.rst | 110 +++++++++++++++++++ FULL_INSTALL.rst | 163 ++++++++++++++++++++++++++++ README.rst | 271 ++++------------------------------------------- 3 files changed, 292 insertions(+), 252 deletions(-) create mode 100644 CONTRIBUTING.rst create mode 100644 FULL_INSTALL.rst diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst new file mode 100644 index 0000000..45592b5 --- /dev/null +++ b/CONTRIBUTING.rst @@ -0,0 +1,110 @@ +Development: Getting started +----------------------------- + +In order to run tests against the Let's Encrypt API we will run a Boulder +server, which is the exact same server Let's Encrypt is running. The server is +started in Virtual Box using Vagrant. To prevent the installation of any +components and dependencies from cluttering up your computer there is also a +client Virtual Box instance. Both of these machines can be setup and started by +running the ``dev_start.sh`` script. This sets up a local boulder server and the +letsencrypt client, so don't worry if it takes more than an hour. + +Vagrant machines +================ +The ``dev_start.sh`` script boots two virtual machines. The first is named +'boulder' and runs a development instance of the boulder server. The second is +'lehaproxy' and runs the client. To test if the machines are setup correctly, +you can SSH into the 'lehaproxy' machine, by running ``vagrant ssh +lehaproxy``. Next, go to the /lehaproxy directory and run +``./tests/boulder-integration.sh``. This runs a modified version of certbot's +boulder-integration test, which tests the HAProxy plugin. If the test succeeds, +your development environment is setup correctly. + +Development: Running locally without sudo +----------------------------------------- + +You can't run certbot without root privileges because it needs to access +``/etc/letsencrypt``, however you can tell it not to use ``/etc/`` and use some +other path in your home directory. + +.. code:: bash + + mkdir ~/projects/certbot-haproxy/working + mkdir ~/projects/certbot-haproxy/working/config + mkdir ~/projects/certbot-haproxy/working/logs + cat <> ~/.config/letsencrypt/cli.ini + work-dir=~/projects/certbot-haproxy/working/ + logs-dir=~/projects/certbot-haproxy/working/logs/ + config-dir=~/projects/certbot-haproxy/working/config + EOF + +Now you can run Certbot without root privileges. + +Further time savers during development.. +---------------------------------------- +The following options can be saved in the ``cli.ini`` file for the following +reasons. + +- ``agree-tos``: During each request for a certificate you need to agree to the + terms of service of Let's Encrypt, automatically accept them every time. +- ``no-self-upgrade``: Tell LE to not upgrade itself. Could be very annoying + when stuff starts to suddenly break, that worked just fine before. +- ``register-unsafely-without-email``: Tell LE that you don't want to be + notified by e-mail when certificates are about to expire or when the TOS + changes, if you don't you will need to enter a valid e-mail address for + every test run. +- ``text``: Disable the curses UI, and use the plain CLI version instead. +- ``domain example.org``: Enter a default domain name to request a certificate + for, so you don't have to specify it every time. +- ``configurator certbot-haproxy:haproxy``: Test with the HAProxy plugin every + time. + +.. code:: bash + + cat <> ~/.config/letsencrypt/cli.ini + agree-tos=True + no-self-upgrade=True + register-unsafely-without-email=True + text=True + domain=example.org + authenticator=certbot-haproxy:haproxy-authenticator + installer=certbot-haproxy:haproxy-installer + EOF + +Setuptools version conflict +--------------------------- + +Most likely the ``python-setuptools`` version in your os's repositories is +quite outdated. You will need to install a newer version, to do this you can +run: + +.. code:: bash + + pip install --upgrade setuptools + +Since pip is part of ``python-setuptools``, you need to have it installed before +you can update. + +Making a `.deb` debian package +------------------------------ + +Requirements: + +- python stdeb: pip install --upgrade stdeb +- dh clean: apt-get install dh-make + +Run the following commands in your vagrant machine: + +.. code:: bash + + apt-file update + python setup.py sdist + # py2dsc has a problem with vbox mounted folders + mv dist/certbot-haproxy-.tar.gz ~ + cd ~ + py2dsc certbot-haproxy-.tar.gz + cd deb_dist/certbot-haproxy- + # NOTE: Not signed, no signed changes (with -uc and -us) + # NOTE: Add the package to the ghtools repo + dpkg-buildpackage -rfakeroot -uc -us + diff --git a/FULL_INSTALL.rst b/FULL_INSTALL.rst new file mode 100644 index 0000000..a24b59f --- /dev/null +++ b/FULL_INSTALL.rst @@ -0,0 +1,163 @@ +Full server setup +================= + +This document describes how to set up a server running HAProxy with certbot and +the certbot-haproxy plugin. The installation below assumes you are running +Debian Jessie but it should be almost entirely the same process on Ubuntu. + +First add the backports repo for Jessie to your apt sources. + +.. note:: + + This will not work for Ubuntu, you will need to use another source, + check which version comes with your version of Ubuntu, if it is a version + below 0.8, you need to find a back port PPA or download certbot from source. + +.. code:: bash + + echo "deb http://ftp.debian.org/debian jessie-backports main" >> \ + /etc/apt/sources.list.d/jessie-backports.list + +Now update, upgrade and install some requirements: + +- **Some utilities:** ``sudo`` ``tcpdump`` ``ufw`` ``git`` ``curl`` ``wget`` +- **OpenSSL and CA certificates:** ``openssl`` ``ca-certificates`` +- **Build dependencies:** ``build-essential`` ``libffi-dev`` ``libssl-dev`` ``python-dev`` +- **Python and related:** ``python`` ``python-setuptools`` +- **HAProxy:** ``haproxy`` +- **Python dependency managing:** ``pip`` + +.. code:: bash + + apt-get update + apt-get upgrade -y + apt-get install -y \ + sudo tcpdump ufw git curl wget \ + openssl ca-certificates \ + build-essential libffi-dev libssl-dev python-dev \ + python python-setuptools \ + haproxy + + easy_install pip + pip install --upgrade setuptools + +We also installed a simple firewall above, but it is not yet configured, let's +do that now: + +.. code:: bash + + ufw allow ssh + ufw allow http + ufw allow https + ufw default deny incoming + ufw --force enable + +.. warning:: + + You probably want a little more protection for a production proxy + than just this simple firewall, but it's out of the scope of this readme. + +Now that we have all dependencies, it's time to start a process that may take +quite some time to complete. HAProxy comes with a DH parameters file that is +considered weak. We need to generate a new dhparams.pem file with a prime of at +least ``2048`` bit length, you can also opt for ``3072`` or ``4096``. This can +take hours on lower specification hardware, but will still take minutes on +faster hardware, especially with ``4096`` bit primes. Run this is in a separate +ssh session or use ``screen`` of ``tmux`` to allow this to run in the +background. + +.. code:: bash + + openssl dhparam -out /opt/certbot/dhparams.pem 2048 + +Now set a hostname. + +.. code:: bash + + echo "[INSERT YOUR HOSTNAME HERE]" > /etc/hostname + hostname -F /etc/hostname + +Run as unprivileged user +++++++++++++++++++++++++ + +If you want to run Certbot in an unprivileged mode, keep reading, otherwise, +skip to the installation of Certbot. + +Certbot normally requires access to the ``/etc/`` directory, which is owned by +root and therefore, Certbot needs to run as root. However, we don't like it +when processes run as root, most especially when they are opening ports on a +public network interface.. + +In order to let Certbot run as an unprivileged user, we will: + +- Create a ``certbot`` user with a home directory on the system so the + automatic renewal of certificates can be run by this user. +- Tell Certbot that the working directories are located in ``certbot``'s home + directory. +- Optionally: add your own user account to the Certbot user's group so you can + run Certbot manually. +- Allow HAProxy to access the certificates that are generated by Certbot. +- Allow the certbot user to restart the HAProxy server. + +Lastly, to do automatic renewal of certificates, we will create a systemd timer +and a service to start at every boot and every 12 hours, at a random time off +the day, in order to not collectively DDOS Let's Encrypts service. + +.. code:: bash + + useradd -s /bin/bash -m -d /opt/certbot certbot + usermod -a -G certbot haproxy # Allow HAProxy access to the certbot certs + mkdir -p /opt/certbot/logs + mkdir -p /opt/certbot/config + mkdir -p /opt/certbot/.config/letsencrypt + +If you need to use Certbot from your user account, or if you have a daemon +running on your proxy server, that configures domains on your proxy, e.g.: in a +web hosting environment - you can add those users to the ``certbot`` group. + +.. code:: bash + + usermod -a -G certbot [ADD YOUR USER HERE] + +You will also need to tell your user what the working directory of your Certbot +setup is (``/opt/certbot/``). Certbot allows you to create a configuration file +with default settings in the users' home dir: +``opt/certbot/.config/letsencrypt/cli.ini``. + +Besides the working directory. + +.. code:: bash + + mkdir -p /opt/certbot/.config/letsencrypt + cat < /opt/certbot/.config/letsencrypt/cli.ini + work-dir=/opt/certbot/ + logs-dir=/opt/certbot/logs/ + config-dir=/opt/certbot/config + EOF + +Next time you run Certbot, it will use our new working directory. + +Now to allow the certbot user to restart HAProxy, put the following in the +sudoers file: + +.. code:: bash + + cat <> /etc/sudoers + %certbot ALL=NOPASSWD: /bin/systemctl restart haproxy + EOF + +Installing certbot-haproxy +++++++++++++++++++++++++++ + +Now we haven't done one very essential thing yet, install ``certbot-haproxy``. +Since our plugin is in an alpha stage, we did not package it yet. You will need +to get it from our Gitlab server. + +.. code:: bash + + git clone https://code.greenhost.net/open/certbot-haproxy.git + cd ./certbot-haproxy/ + sudo pip install ./ + +Continue reading `README.rst`_ after the quick installation instructions, at +:ref:`haproxy_config` diff --git a/README.rst b/README.rst index b362260..a3a316b 100644 --- a/README.rst +++ b/README.rst @@ -45,153 +45,19 @@ take additional steps during the installation. Thus, the requirements are: - HAProxy 1.6+ (we will configure SNI, which is not strictly required) - Certbot 0.8+ -Installing: Getting started ---------------------------- +Installing: +----------- -The installation below assumes you are running Debian Jessie but it should be -almost entirely the same process on Ubuntu. +If you need to set up a server, follow the instructions in the +`FULL_INSTALL.rst`_ document. If you only need to install the certbot-haproxy +plugin and already have HAProxy running on a server, keep reading. -First add the backports repo for Jessie to your apt sources. +Quick installation +++++++++++++++++++ -.. note:: - - This will not work for Ubuntu, you will need to use another source, - check which version comes with your version of Ubuntu, if it is a version - below 0.8, you need to find a back port PPA or download certbot from source. - -.. code:: bash - - echo "deb http://ftp.debian.org/debian jessie-backports main" >> \ - /etc/apt/sources.list.d/jessie-backports.list - -Now update, upgrade and install some requirements: - -- **Some utilities:** ``sudo`` ``tcpdump`` ``ufw`` ``git`` ``curl`` ``wget`` -- **OpenSSL and CA certificates:** ``openssl`` ``ca-certificates`` -- **Build dependencies:** ``build-essential`` ``libffi-dev`` ``libssl-dev`` ``python-dev`` -- **Python and related:** ``python`` ``python-setuptools`` -- **HAProxy:** ``haproxy`` -- **Python dependency managing:** ``pip`` - -.. code:: bash - - apt-get update - apt-get upgrade -y - apt-get install -y \ - sudo tcpdump ufw git curl wget \ - openssl ca-certificates \ - build-essential libffi-dev libssl-dev python-dev \ - python python-setuptools \ - haproxy - - easy_install pip - pip install --upgrade setuptools - -We also installed a simple firewall above, but it is not yet configured, let's -do that now: - -.. code:: bash - - ufw allow ssh - ufw allow http - ufw allow https - ufw default deny incoming - ufw --force enable - -.. warning:: - - You probably want a little more protection for a production proxy - than just this simple firewall, but it's out of the scope of this readme. - -Now that we have all dependencies, it's time to start a process that may take -quite some time to complete. HAProxy comes with a DH parameters file that is -considered weak. We need to generate a new dhparams.pem file with a prime of at -least ``2048`` bit length, you can also opt for ``3072`` or ``4096``. This can -take hours on lower specification hardware, but will still take minutes on -faster hardware, especially with ``4096`` bit primes. Run this is in a separate -ssh session or use ``screen`` of ``tmux`` to allow this to run in the -background. - -.. code:: bash - - openssl dhparam -out /opt/certbot/dhparams.pem 2048 - -Now set a hostname. - -.. code:: bash - - echo "[INSERT YOUR HOSTNAME HERE]" > /etc/hostname - hostname -F /etc/hostname - -If you want to run Certbot in an unprivileged mode, keep reading, otherwise, -skip to the installation of Certbot. - -Certbot normally requires access to the ``/etc/`` directory, which is owned by -root and therefore, Certbot needs to run as root. However, we don't like it -when processes run as root, most especially when they are opening ports on a -public network interface.. - -In order to let Certbot run as an unprivileged user, we will: - -- Create a ``certbot`` user with a home directory on the system so the - automatic renewal of certificates can be run by this user. -- Tell Certbot that the working directories are located in ``certbot``'s home - directory. -- Optionally: add your own user account to the Certbot user's group so you can - run Certbot manually. -- Allow HAProxy to access the certificates that are generated by Certbot. -- Allow the certbot user to restart the HAProxy server. - -Lastly, to do automatic renewal of certificates, we will create a systemd timer -and a service to start at every boot and every 12 hours, at a random time off -the day, in order to not collectively DDOS Let's Encrypts service. - -.. code:: bash - - useradd -s /bin/bash -m -d /opt/certbot certbot - usermod -a -G certbot haproxy # Allow HAProxy access to the certbot certs - mkdir -p /opt/certbot/logs - mkdir -p /opt/certbot/config - mkdir -p /opt/certbot/.config/letsencrypt - -If you need to use Certbot from your user account, or if you have a daemon -running on your proxy server, that configures domains on your proxy, e.g.: in a -web hosting environment - you can add those users to the ``certbot`` group. - -.. code:: bash - - usermod -a -G certbot [ADD YOUR USER HERE] - -You will also need to tell your user what the working directory of your Certbot -setup is (``/opt/certbot/``). Certbot allows you to create a configuration file -with default settings in the users' home dir: -``opt/certbot/.config/letsencrypt/cli.ini``. - -Besides the working directory. - -.. code:: bash - - mkdir -p /opt/certbot/.config/letsencrypt - cat < /opt/certbot/.config/letsencrypt/cli.ini - work-dir=/opt/certbot/ - logs-dir=/opt/certbot/logs/ - config-dir=/opt/certbot/config - EOF - -Next time you run Certbot, it will use our new working directory. - -Now to allow the certbot user to restart HAProxy, put the following in the -sudoers file: - -.. code:: bash - - cat <> /etc/sudoers - %certbot ALL=NOPASSWD: /bin/systemctl restart haproxy - EOF - -Now we haven't done one very essential thing yet, install ``certbot-haproxy``. -Since our plugin is in an alpha stage, we did not package it yet. You will need -to get it from our Gitlab server. +If you already have a server running HAProxy, first install certbot following +their `installation instructions `_. +Then follow these steps to install certbot-haproxy: .. code:: bash @@ -199,6 +65,9 @@ to get it from our Gitlab server. cd ./certbot-haproxy/ sudo pip install ./ +.. _haproxy_config: +Configuring HAProxy to work with certbot-haproxy +------------------------------------------------ Let's Encrypt's CA server will try to contact your proxy on port 80, which is most likely in use for your and/or your customers' websites. So we have @@ -302,6 +171,9 @@ together a configuration that works for you. systemctl restart haproxy +Running certbot-haproxy +----------------------- + Now you can try to run Certbot with the plugin as the Authenticator and Installer, if you already have websites configured in your HAProxy setup, you may try to install a certificate now. @@ -376,112 +248,7 @@ after the server has been offline for a long time. systemctl start letsencrypt.timer -Development: Getting started ------------------------------ +Development +----------- -In order to run tests against the Let's Encrypt API we will run a Boulder -server, which is the exact same server Let's Encrypt is running. The server is -started in Virtual Box using Vagrant. To prevent the installation of any -components and dependencies from cluttering up your computer there is also a -client Virtual Box instance. Both of these machines can be setup and started by -running the ``dev_start.sh`` script. This sets up a local boulder server and the -letsencrypt client, so don't worry if it takes more than an hour. - -Vagrant machines -================ -The ``dev_start.sh`` script boots two virtual machines. The first is named -'boulder' and runs a development instance of the boulder server. The second is -'lehaproxy' and runs the client. To test if the machines are setup correctly, -you can SSH into the 'lehaproxy' machine, by running ``vagrant ssh -lehaproxy``. Next, go to the /lehaproxy directory and run -``./tests/boulder-integration.sh``. This runs a modified version of certbot's -boulder-integration test, which tests the HAProxy plugin. If the test succeeds, -your development environment is setup correctly. - -Development: Running locally without sudo ------------------------------------------ - -You can't run certbot without root privileges because it needs to access -``/etc/letsencrypt``, however you can tell it not to use ``/etc/`` and use some -other path in your home directory. - -.. code:: bash - - mkdir ~/projects/certbot-haproxy/working - mkdir ~/projects/certbot-haproxy/working/config - mkdir ~/projects/certbot-haproxy/working/logs - cat <> ~/.config/letsencrypt/cli.ini - work-dir=~/projects/certbot-haproxy/working/ - logs-dir=~/projects/certbot-haproxy/working/logs/ - config-dir=~/projects/certbot-haproxy/working/config - EOF - -Now you can run Certbot without root privileges. - -Further time savers during development.. ----------------------------------------- -The following options can be saved in the ``cli.ini`` file for the following -reasons. - -- ``agree-tos``: During each request for a certificate you need to agree to the - terms of service of Let's Encrypt, automatically accept them every time. -- ``no-self-upgrade``: Tell LE to not upgrade itself. Could be very annoying - when stuff starts to suddenly break, that worked just fine before. -- ``register-unsafely-without-email``: Tell LE that you don't want to be - notified by e-mail when certificates are about to expire or when the TOS - changes, if you don't you will need to enter a valid e-mail address for - every test run. -- ``text``: Disable the curses UI, and use the plain CLI version instead. -- ``domain example.org``: Enter a default domain name to request a certificate - for, so you don't have to specify it every time. -- ``configurator certbot-haproxy:haproxy``: Test with the HAProxy plugin every - time. - -.. code:: bash - - cat <> ~/.config/letsencrypt/cli.ini - agree-tos=True - no-self-upgrade=True - register-unsafely-without-email=True - text=True - domain=example.org - authenticator=certbot-haproxy:haproxy-authenticator - installer=certbot-haproxy:haproxy-installer - EOF - -Setuptools version conflict ---------------------------- - -Most likely the ``python-setuptools`` version in your os's repositories is -quite outdated. You will need to install a newer version, to do this you can -run: - -.. code:: bash - - pip install --upgrade setuptools - -Since pip is part of ``python-setuptools``, you need to have it installed before -you can update. - -Making a `.deb` debian package ------------------------------- - -Requirements: - -- python stdeb: pip install --upgrade stdeb -- dh clean: apt-get install dh-make - -Run the following commands in your vagrant machine: - -.. code:: bash - - apt-file update - python setup.py sdist - # py2dsc has a problem with vbox mounted folders - mv dist/certbot-haproxy-.tar.gz ~ - cd ~ - py2dsc certbot-haproxy-.tar.gz - cd deb_dist/certbot-haproxy- - # NOTE: Not signed, no signed changes (with -uc and -us) - # NOTE: Add the package to the ghtools repo - dpkg-buildpackage -rfakeroot -uc -us +For development guidelines, check `CONTRIBUTING.rst`_