Merge pull request #4948 from certbot/challenge_docs

Challenge docs

5 files changed, 140 insertions, 159 deletions

diff --git a/docs/challenges.rst b/docs/challenges.rst
index c00cd3bdf..25d190147 100644
--- a/docs/challenges.rst
+++ b/docs/challenges.rst
@@ -1,159 +1,85 @@
 Challenges
 ==========
 
-Digital certificates can only be issued to people who are entitled to them. For example, assuming you don't run google.com, you're not entitled to a certificate for it. Nor is someone else entitled to receive a certificate for your web site.
-
-In order to receive a certificate from Let's Encrypt certificate authority (CA), you have to prove your control over each of the domain names that will be listed in the certificate. You can do so by making certain publicly-visible changes, proving that the person who's requested a particular certificate is the same person who controls the site(s) that the certificate will refer to.
-
-Let’s Encrypt specifies three different ways to prove your control over a domain (each of which Certbot may be able to do for you). These are called "challenges," because you are being challenged to perform tasks that only someone who controls the domain should be able to accomplish.
-
-When you use Certbot, it will attempt to help you prove control over your domains automatically in a way that's acceptable to the CA. Especially if this doesn't work the way you expected, it can be helpful to understand what Certbot is trying to do in each case.
-
-The three ways to prove your control over a domain for the Let’s Encrypt CA are:
-
-* Posting a specified file on a web site
-
-This method is called the HTTP-01 challenge.  In this challenge, the certificate authority will expect a specified file to be posted in a specified location on a web site.  The file will be downloaded using an HTTP request on TCP port 80.  Since part of what this challenge shows is the ability to create a file at an arbitrary location, you cannot choose a different location or port number.
-
-* Offering a specified certificate on a web site
-
-This method is called the TLS-SNI-01 challenge.  In this challenge, the certificate authority will expect a specified digital certificate to be provided by the web server in response to an HTTPS request using a particular made-up domain name.  The request will be made using HTTPS on TCP port 443.  You cannot choose a different port number.
-
-This certificate is a self-signed certificate created by Certbot.  You use it only temporarily to prove your control over a domain name.  It’s not the same as the certificate for your site that will later be issued by Let's Encrypt once you've proven that you control the site.
-
-* Posting a specified DNS record in the domain name system
-
-This method is called the DNS-01 challenge.  In this challenge, the certificate authority will expect a specified DNS record to be present in your DNS zone when queried for.  The record will be a TXT record for a specific subdomain of the name you're proving your control over.
-
-For each kind of challenge, the challenge can potentially be completed *automatically* (Certbot directly makes the necessary changes itself, or runs another program that does so), or *manually* (Certbot tells you to make a certain change, and you edit a configuration file of some kind in order to accomplish it).  Certbot's design emphasizes performing challenges *automatically*, and this is the normal case for most uses of Certbot.
-
-Some Certbot *plugins* offer the functionality of an *authenticator*, which simply means that they can satisfy challenges. Different plugins can satisfy different kinds of challenges, as follows:
-
-apache plugin: Can only use TLS-SNI-01.  Tries to edit your Apache configuration files in order to temporarily serve a specified Certbot-generated certificate for a specified name.  This can work when you're running Certbot on a web server with an existing installation of Apache that is able to listen on port 443. This makes certain assumptions about your Apache configuration.
-
-nginx plugin: Can only use TLS-SNI-01.  Tries to edit your nginx configuration files in order to temporarily serve a specified Certbot-generated certificate for a specified name.  This can work when you're running Certbot on a web server with an existing installation of nginx that is able to listen on port 443. This makes certain assumptions about your nginx configuration.
-
-webroot plugin: Can only use HTTP-01.  Tries to place a file into an appropriate place in order for that file to be served over HTTP on port 80 by an existing web server running on your system.  This can work when you're running Certbot on a web server with any existing server application that already listens to web requests on port 80, and that serves files from disk in response.
-
-standalone plugin: Can use either TLS-SNI-01 or HTTP-01.  (You can choose with the `--preferred-challenges` option.)  Tries to run its own temporary web server which will speak either HTTP on port 80 (for HTTP-01) or HTTPS on port 443 (for TLS-SNI-01).  This can work if either of these ports is free to receive incoming connections at the moment that you run Certbot, because there's no existing program listening to them or because you've temporarily shut down any server application that was listening to them.
-
-manual plugin: Can use either DNS-01 or HTTP-01.  May tell you what changes you are expected to make to your configuration.  Or, using an external script, can update your DNS records (for DNS-01) or your webroot (for HTTP-01).  This can work if you have appropriate technical knowledge of how to make these kinds of changes yourself when asked to do so.  Note that this will prevent automated renewal of your certificate using `certbot renew`.  [Can manual also use TLS-SNI-01??]
-
-
-Common problems with passing different challenges
-
-HTTP-01 challenge:
-* (With webroot plugin) You aren't running Certbot on your web server
-
-  Most people should install and run Certbot on their web server hosting their website, not on their laptops or some other computer.  While you can use Certbot in manual mode on a laptop and then separately set up the appropriate files on your webserver, it's not likely to be the most convenient way to get a certificate for most users.
-
-* A domain name you're requesting a certificate for isn't correctly pointed at that web server
-
-  In most cases, every name you're requesting a certificate for should already exist and be pointed to the public IP address of the server where you're requesting that certificate.  (Some alternatives exist for complex network configurations, but they're the exception rather than the rule.)
-
-* A firewall is blocking access to port 80
-
-  The certificate authority needs to be able to connect to port 80 of your server in order to confirm that you satisfied the HTTP-01 challenge.  So that needs to be publicly reachable from the Internet, and not blocked by a router or firewall.
-
-* (With webroot plugin) You specified the webroot directory incorrectly
-
-  If you used `--webroot`, you need to tell Certbot where it can put
-  files in order to have them served by your existing web server.
-  If you said your webroot for example.com was /var/www/example.com,
-  then a file placed in /var/www/example.com/.well-known/acme-challenge/testfile should appear on
-  your web site at `http://example.com/.well-known/acme-challenge/testfile` (which you can test using a web browser). (A redirection to HTTPS
+To receive a certificate from Let's Encrypt certificate authority (CA), you must pass a *challenge* to 
+prove you control each of the domain names that will be listed in the certificate. A challenge is one of 
+three tasks that only someone who controls the domain should be able to accomplish:
+
+* Posting a specified file in a specified location on a web site (the HTTP-01 challenge)
+* Offering a specified temporary certificate on a web site (the TLS-SNI-01 challenge) 
+* Posting a specified DNS record in the domain name system (the DNS-01 challenge)
+
+It’s possible to complete each type of challenge *automatically* (Certbot directly makes the necessary 
+changes itself, or runs another program that does so), or *manually* (Certbot tells you to make a 
+certain change, and you edit a configuration file of some kind in order to accomplish it). Certbot's 
+design favors performing challenges automatically, and this is the normal case for most users of Certbot.
+
+Some plugins offer an *authenticator*, meaning that they can satisfy challenges:
+
+* Apache plugin: (TLS-SNI-01) Tries to edit your Apache configuration files to temporarily serve 
+  a Certbot-generated certificate for a specified name. Use the Apache plugin when you're running 
+  Certbot on a web server with Apache listening on port 443.
+* NGINX plugin: (TLS-SNI-01) Tries to edit your NGINX configuration files to temporarily serve a 
+  Certbot-generated certificate for a specified name. Use the NGINX plugin when you're running 
+  Certbot on a web server with NGINX listening on port 443.
+* Webroot plugin: (HTTP-01) Tries to place a file where it can be served over HTTP on port 80 by a
+  web server running on your system. Use the Webroot plugin when you're running Certbot on 
+  a web server with any server application listening on port 80 serving files from a folder on disk in response.
+* Standalone plugin: (TLS-SNI-01 or HTTP-01) Tries to run a temporary web server listening on either HTTP on 
+  port 80 (for HTTP-01) or HTTPS on port 443 (for TLS-SNI-01). Use the Standalone plugin if no existing program 
+  is listening to these ports. Choose TLS-SNI-01 or HTTP-01 using the `--preferred-challenges` option.
+* Manual plugin: (DNS-01 or HTTP-01) Either tells you what changes to make to your configuration or updates 
+  your DNS records using an external script (for DNS-01) or your webroot (for HTTP-01). Use the Manual 
+  plugin if you have the technical knowledge to make configuration changes yourself when asked to do so. 
+
+Tips for Challenges
+-------------------
+General tips:
+
+* Run Certbot on your web server, not on your laptop or another server. It’s usually the easiest way to get a certificate.
+* Use a tool like the DNSchecker at dnsstuff.com to check your DNS records to make sure 
+  there are no serious errors. A DNS error can prevent a certificate authority from 
+  issuing a certificate, even if it does not prevent your site from loading in a browser.
+* If you are using Apache or NGINX plugins, make sure the configuration of your Apache or NGINX server is correct.
+
+HTTP-01 Challenge
+~~~~~~~~~~~~~~~~~
+
+* Make sure the domain name exists and is already pointed to the public IP address of the server where 
+  you’re requesting the certificate.
+* Make sure port 80 is open, publicly reachable from the Internet, and not blocked by a router or firewall.
+* When using the Webroot plugin or the manual plugin, make sure the the webroot directory exists and that you
+  specify it properly. If you set the webroot directory for example.com to `/var/www/example.com`
+  then a file placed in `/var/www/example.com/.well-known/acme-challenge/testfile` should appear on
+  your web site at `http://example.com/.well-known/acme-challenge/testfile` (A redirection to HTTPS
   is OK here and should not stop the challenge from working.)
+* In some web server configurations, all pages are dynamically generated by some kind of framework, 
+  usually using a database backend. In this case, there might not be a particular directory 
+  from which the web server can serve filesdirectly. Using the Webroot plugin in this case 
+  requires making a change to your web server configuration first.
+* Make sure your web server serves files properly from the directory where the challenge 
+  file is placed (e. g. `/.well-known/acme-challenge`) to the expected location on the 
+  website without adding a header or footer.
+* When using the Standalone plugin, make sure another program is not already listening to port 80 on the server.
+* When using the Webroot plugin, make sure there is a web server listening on port 80.
+
+TLS-SNI-01 Challenge
+~~~~~~~~~~~~~~~~~~~~
+
+* The TLS-SNI-01 challenge doesn’t work with content delivery networks (CDNs) 
+  like CloudFlare and Akamai because the domain name is pointed at the CDN, not directly at your server.
+* Make sure port 443 is open, publicly reachable from the Internet, and not blocked by a router or firewall.
+* When using the Apache plugin, make sure you are running Apache and no other web server on port 443.
+* When using the NGINX plugin, make sure you are running NGINX and no other web server on port 443.
+* With either the Apache or NGINX plugin, certbot modifies your web server configuration. If you get
+  an error after successfully completing the challenge, then you have received a certificate but the
+  plugin was unable to modify your web server configuration, meaning that you'll have to install the certificate manually.
+  In that case, please file a bug to help us improve certbot!
+* When using the Standalone plugin, make sure another program is not already listening to port 443 on the server.
+
+DNS-01 Challenge
+~~~~~~~~~~~~~~~~
+
+* When using the manual plugin, make sure your DNS records are correctly updated; 
+  you must be able to make appropriate changes to your DNS zone in order to pass the challenge.
 
-  Note that you should *not* specify the .well-known/acme-challenge directory itself.  Instead, you should specify the top level directory that web content is served from.
-
-* (With webroot plugin) You don't have a webroot directory at all
-
-  In some web server configurations, all pages are dynamically generated by some kind of framework, usually using a database backend.  In this case, there might not be a particular directory that files can be directly served from by the existing web server application.  Using the webroot plugin in this case requires making a change to your web server configuration first.
-
-* (With manual plugin) You updated the webroot directory incorrectly
-
-  If you used `--manual`, you need to know where you can put files in order to have them served by your existing web server. If you think your webroot for example.com is /var/www/example.com, then a file placed in /var/www/example.com/.well-known/acme-challenge/testfile should appear on
-  your web site at `http://example.com/.well-known/acme-challenge/testfile`.  (A redirection to HTTPS
-  is OK here and should not stop the challenge from working.) You should also make sure that you don't make a typo in the name of the file when creating it.
-
-* Your existing web server's configuration refuses to serve files
-  from /.well-known/acme-challenge, or doesn't serve them at the
-  /.well-known/acme-challenge location on your site, or serves them
-  with a header or footer, or serves them with an unusual MIME type.
-
-* (With standalone plugin)
-  You tried to use `--standalone` when there was already some other
-  program on your server listening to port 80
-
-* (With webroot plugin)
-  You tried to use `--webroot` when you don't have an existing web
-  server listening on port 80
-
-* Your DNS records aren't valid
-  Try checking your DNS records with a tool like the DNSchecker at
-  http://www.dnsstuff.com/ to make sure there are no serious errors.
-  Sometimes a DNS error still allows your site to load in a web
-  browser, but prevents the certificate authority from issuing a
-  certificate.
-
-TLS-SNI-01 challenge:
-* You aren't running Certbot on your web server
-
-  Most people should install and run Certbot on their web server hosting their website, not on their laptops or some other computer.  While you can use Certbot in manual mode on a laptop and then separately set up the appropriate files on your webserver, it's not likely to be the most convenient way to get a certificate for most users.
-
-* A domain name you're requesting a certificate for isn't correctly
-  pointed at that web server
-
-  In most cases, every name you're requesting a certificate for should
-  already exist and be pointed to the server where you're requesting
-  that certificate.  (Some alternatives exist for complex network
-  configurations, but they're the exception rather than the rule.)
-
-* You're using a content delivery network (CDN)
-
-  TLS-SNI-01 doesn't work with CDNs (like CloudFlare and Akamai).  You
-  have to use a different challenge type.  (This is a special case of
-  the previous problem: the domain name is pointed at the CDN, not
-  directly at your server.)
-
-* A firewall is blocking access to port 443
-
-  The certificate authority needs to be able to connect to port 443 of
-  your server in order to confirm that you satisfied the TLS-SNI-01
-  challenge.  So that needs to be publicly reachable from the Internet,
-  and not blocked by a router or firewall.
-
-* (With apache plugin)
-  Certbot thinks you're running Apache, but you aren't running it, or
-  you're running a different server of some kind on port 443
-
-* (With nginx plugin)
-  Certbot thinks you're running nginx, but you aren't running it, or
-  you're running a different server of some kind on port 443
-
-* (With apache or nginx plugin)
-  Certbot doesn't know how to modify your web server configuration correctly
-
-* (With standalone plugin)
-  You tried to use `--standalone` when there was already some other
-  program on your server listening to port 443
-
-* Your DNS records aren't valid
-  Try checking your DNS records with a tool like the DNSchecker at
-  http://www.dnsstuff.com/ to make sure there are no serious errors.
-  Sometimes a DNS error still allows your site to load in a web
-  browser, but prevents the certificate authority from issuing a
-  certificate.
-
-DNS-01 challenge:
-
-* (With manual plugin) Your DNS records weren't correctly updated.
-  You need to be able to make appropriate changes to your DNS zone
-  in order to pass the challenge.
-
-* Your DNS records aren't valid.
-  Try checking your DNS records with a tool like the DNSchecker at
-  http://www.dnsstuff.com/ to make sure there are no serious errors.
-  Sometimes a DNS error still allows your site to load in a web
-  browser, but prevents the certificate authority from issuing a
-  certificate.
diff --git a/docs/index.rst b/docs/index.rst
index 746080864..17cde1adf 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -5,6 +5,7 @@ Welcome to the Certbot documentation!
    :maxdepth: 2
 
    intro
+   what
    install
    using
    contributing
diff --git a/docs/intro.rst b/docs/intro.rst
index 90c3761ec..2d4abdc2d 100644
--- a/docs/intro.rst
+++ b/docs/intro.rst
@@ -2,6 +2,9 @@
 Introduction
 =====================
 
+.. note::
+    To get started quickly, use the `interactive installation guide <https://certbot.eff.org>`_.
+
 .. include:: ../README.rst
     :start-after: tag:intro-begin
     :end-before: tag:intro-end
diff --git a/docs/using.rst b/docs/using.rst
index c545dc48b..aae8efbf2 100644
--- a/docs/using.rst
+++ b/docs/using.rst
@@ -61,7 +61,7 @@ manual_     Y    N    | Helps you obtain a certificate by giving you instruction
 =========== ==== ==== =============================================================== =============================
 
 Under the hood, plugins use one of several ACME protocol challenges_ to
-prove you control a domain.  The options are http-01_ (which uses port 80),
+prove you control a domain. The options are http-01_ (which uses port 80),
 tls-sni-01_ (port 443) and dns-01_ (requiring configuration of a DNS server on
 port 53, though that's often not the same machine as your webserver). A few
 plugins support more than one challenge type, in which case you can choose one
@@ -102,7 +102,7 @@ If you're getting a certificate for many domains at once, the plugin
 needs to know where each domain's files are served from, which could
 potentially be a separate directory for each domain. When requesting a
 certificate for multiple domains, each domain will use the most recently
-specified ``--webroot-path``.  So, for instance,
+specified ``--webroot-path``. So, for instance,
 
 ::
 
@@ -290,6 +290,7 @@ using the ``--cert-name`` flag to specify a particular certificate for the ``run
 
   certbot certonly --cert-name example.com
 
+.. _updating_certs:
 
 Re-creating and Updating Existing Certificates
 ----------------------------------------------
@@ -324,7 +325,24 @@ need to issue this command in normal circumstances.
 
 ``--expand`` tells Certbot to update an existing certificate with a new
 certificate that contains all of the old domains and one or more additional
-new domains.
+new domains. With the ``--expand`` option, use the ``-d`` option to specify
+all existing domains and one or more new domains.
+
+Example:
+
+.. code-block:: none
+
+  certbot --expand -d existing.com,example.com,newdomain.com
+
+If you prefer, you can specify the domains individually like this:
+
+.. code-block:: none
+
+  certbot --expand -d existing.com -d example.com -d newdomain.com
+
+Consider using ``--cert-name`` instead of ``--expand``, as it gives more control
+over which certificate is modified and it lets you remove domains as well as adding them.
+
 
 ``--allow-subset-of-names`` tells Certbot to continue with certificate generation if
 only some of the specified domain authorizations can be obtained. This may
@@ -338,8 +356,10 @@ certificate counts against several rate limits that are intended to prevent
 abuse of the ACME protocol, as described
 `here <https://community.letsencrypt.org/t/rate-limits-for-lets-encrypt/6769>`__.
 
+.. _changing:
+
 Changing a Certificate's Domains
---------------------------------
+================================
 
 The ``--cert-name`` flag can also be used to modify the domains a certificate contains,
 by specifying new domains using the ``-d`` or ``--domains`` flag. If certificate ``example.com``
@@ -803,7 +823,7 @@ changed by passing the desired number to the command line flag
 Certbot command-line options
 ============================
 
-Certbot supports a lot of command line options.  Here's the full list, from
+Certbot supports a lot of command line options. Here's the full list, from
 ``certbot --help all``:
 
 .. literalinclude:: cli-help.txt
diff --git a/docs/what.rst b/docs/what.rst
new file mode 100644
index 000000000..3d33346c2
--- /dev/null
+++ b/docs/what.rst
@@ -0,0 +1,31 @@
+======================
+What is a Certificate?
+======================
+
+A public key or digital *certificate* (formerly called an SSL certificate) uses a public key 
+and a private key to enable secure communication between a client program (web browser, email client, 
+etc.) and a server over an encrypted SSL (secure socket layer) or TLS (transport layer security) connection.
+The certificate is used both to encrypt the initial stage of communication (secure key exchange) 
+and to identify the server. The certificate
+includes information about the key, information about the server identity, and the digital signature
+of the certificate issuer. If the issuer is trusted by the software that initiates the communication,
+and the signature is valid, then the key can be used to communicate securely with the server identified by 
+the certificate. Using a certificate is a good way to prevent "man-in-the-middle" attacks, in which
+someone in between you and the server you think you are talking to is able to insert their own (harmful)
+content.
+
+You can use Certbot to easily obtain and configure a free certificate from Let's Encrypt, a
+joint project of EFF, Mozilla, and many other sponsors.
+
+Certificates and Lineages
+=========================
+
+Certbot introduces the concept of a *lineage,* which is a collection of all the versions of a certificate
+plus Certbot configuration information maintained for that certificate from
+renewal to renewal. Whenever you renew a certificate, Certbot keeps the same configuration unless
+you explicitly change it, for example by adding or removing domains. If you add domains, you can 
+either add them to an existing lineage or create
+a new one. 
+
+See also:
+:ref:`updating_certs`