Remove codecov (#7811)

After getting a +1 from everyone on the team, this PR removes the use of `codecov` from the Certbot repo because we keep having problems with it.

Two noteworthy things about this PR are:

1. I left the text at https://github.com/certbot/certbot/blob/4ea98d830bcc3d1b980a4055243c6a6a25d8dc54/.azure-pipelines/INSTALL.md#add-a-secret-variable-to-a-pipeline-like-codecov_token because I think it's useful to document how to set up a secret variable in general.
2. I'm not sure what the text "Option -e makes sure we fail fast and don't submit to codecov." in `tox.cover.py` refers to but it seems incorrect since `-e` isn't accepted or used by the script so I just deleted the line.

As part of this, I said I'd open an issue to track setting up coveralls (which seems to be the only real alternative to codecov) which is at https://github.com/certbot/certbot/issues/7810.

With my change, failure output looks something like:
```
$ tox -e py27-cover
...
Name                                                         Stmts   Miss  Cover   Missing
------------------------------------------------------------------------------------------
certbot/certbot/__init__.py                                      1      0   100%
certbot/certbot/_internal/__init__.py                            0      0   100%
certbot/certbot/_internal/account.py                           191      4    98%   62-63, 206, 337
...
certbot/tests/storage_test.py                                  530      0   100%
certbot/tests/util_test.py                                     374     29    92%   211-213, 480-484, 489-499, 504-511, 545-547, 552-554
------------------------------------------------------------------------------------------
TOTAL                                                        14451    647    96%
Command '['/path/to/certbot/dir/.tox/py27-cover/bin/python', '-m', 'coverage', 'report', '--fail-under', '100', '--include', 'certbot/*', '--show-missing']' returned non-zero exit status 2
Test coverage on certbot did not meet threshold of 100%.
ERROR: InvocationError for command /Users/bmw/Development/certbot/certbot/.tox/py27-cover/bin/python tox.cover.py (exited with code 1)
_________________________________________________________________________________________________________________________________________________________ summary _________________________________________________________________________________________________________________________________________________________
ERROR:   py27-cover: commands failed
```
I printed the exception just so we're not throwing away information.

I think it's also possible we fail for a reason other than the threshold not meeting the percentage, but I've personally never seen this, `coverage report` output is not being captured so hopefully that would inform devs if something else is going on, and saying something like "Test coverage probably did not..." seems like overkill to me personally.

* remove codecov

* remove unused variable group

* remove codecov.yml

* Improve tox.cover.py failure output.

6 files changed, 15 insertions, 46 deletions

diff --git a/.azure-pipelines/templates/tests-suite.yml b/.azure-pipelines/templates/tests-suite.yml
index 069ea94d6..d330b7954 100644
--- a/.azure-pipelines/templates/tests-suite.yml
+++ b/.azure-pipelines/templates/tests-suite.yml
@@ -25,8 +25,6 @@ jobs:
           PYTEST_ADDOPTS: --numprocesses 4
     pool:
       vmImage: $(IMAGE_NAME)
-    variables:
-    - group: certbot-common
     steps:
     - bash: brew install augeas
       condition: startswith(variables['IMAGE_NAME'], 'macOS')
@@ -39,14 +37,3 @@ jobs:
       displayName: Install dependencies
     - script: python -m tox
       displayName: Run tox
-    # We do not require codecov report upload to succeed. So to avoid to break the pipeline if
-    # something goes wrong, each command is suffixed with a command that hides any non zero exit
-    # codes and echoes an informative message instead.
-    - bash: |
-        curl -s https://codecov.io/bash -o codecov-bash || echo "Failed to download codecov-bash"
-        chmod +x codecov-bash || echo "Failed to apply execute permissions on codecov-bash"
-        ./codecov-bash -F windows  || echo "Codecov did not collect coverage reports"
-      condition: in(variables['TOXENV'], 'py37-cover', 'integration-certbot')
-      env:
-        CODECOV_TOKEN: $(codecov_token)
-      displayName: Publish coverage
diff --git a/.codecov.yml b/.codecov.yml
deleted file mode 100644
index 0a97fffe3..000000000
--- a/.codecov.yml
+++ /dev/null
@@ -1,18 +0,0 @@
-coverage:
-  status:
-    project:
-      default: off
-      linux:
-        flags: linux
-        # Fixed target instead of auto set by #7173, can
-        # be removed when flags in Codecov are added back.
-        target: 97.4
-        threshold: 0.1
-        base: auto
-      windows:
-        flags: windows
-        # Fixed target instead of auto set by #7173, can
-        # be removed when flags in Codecov are added back.
-        target: 97.4
-        threshold: 0.1
-        base: auto
diff --git a/.travis.yml b/.travis.yml
index e5354898d..d498d0305 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -247,15 +247,13 @@ addons:
 # version of virtualenv. The option "-I" is set so when CERTBOT_NO_PIN is also
 # set, pip updates dependencies it thinks are already satisfied to avoid some
 # problems with its lack of real dependency resolution.
-install: 'tools/pip_install.py -I codecov tox virtualenv'
+install: 'tools/pip_install.py -I tox virtualenv'
 # Most of the time TRAVIS_RETRY is an empty string, and has no effect on the
 # script command. It is set only to `travis_retry` during farm tests, in
 # order to trigger the Travis retry feature, and compensate the inherent
 # flakiness of these specific tests.
 script: '$TRAVIS_RETRY tox'
 
-after_success: '[ "$TOXENV" == "py27-cover" ] && codecov -F linux'
-
 notifications:
   email: false
   irc:
diff --git a/certbot/README.rst b/certbot/README.rst
index d1b1e4fe2..5ed74f247 100644
--- a/certbot/README.rst
+++ b/certbot/README.rst
@@ -71,16 +71,12 @@ ACME spec: http://ietf-wg-acme.github.io/acme/
 
 ACME working area in github: https://github.com/ietf-wg-acme/acme
 
-|build-status| |coverage| |container|
+|build-status| |container|
 
 .. |build-status| image:: https://travis-ci.com/certbot/certbot.svg?branch=master
    :target: https://travis-ci.com/certbot/certbot
    :alt: Travis CI status
 
-.. |coverage| image:: https://codecov.io/gh/certbot/certbot/branch/master/graph/badge.svg
-   :target: https://codecov.io/gh/certbot/certbot
-   :alt: Coverage status
-
 .. |container| image:: https://quay.io/repository/letsencrypt/letsencrypt/status
    :target: https://quay.io/repository/letsencrypt/letsencrypt
    :alt: Docker Repository on Quay.io
diff --git a/tools/dev_constraints.txt b/tools/dev_constraints.txt
index 7d2013c7a..cfa036435 100644
--- a/tools/dev_constraints.txt
+++ b/tools/dev_constraints.txt
@@ -18,7 +18,6 @@ boto3==1.11.7
 botocore==1.14.7
 cached-property==1.5.1
 cloudflare==2.3.1
-codecov==2.0.15
 configparser==3.7.4
 contextlib2==0.6.0.post1
 coverage==4.5.4
diff --git a/tox.cover.py b/tox.cover.py
index 3e69a14d6..4848b2740 100755
--- a/tox.cover.py
+++ b/tox.cover.py
@@ -1,4 +1,6 @@
 #!/usr/bin/env python
+from __future__ import print_function
+
 import argparse
 import os
 import subprocess
@@ -48,18 +50,23 @@ def cover(package):
 
     subprocess.check_call([sys.executable, '-m', 'pytest',
                            '--cov', pkg_dir, '--cov-append', '--cov-report=', pkg_dir])
-    subprocess.check_call([
-        sys.executable, '-m', 'coverage', 'report', '--fail-under', str(threshold), '--include',
-        '{0}/*'.format(pkg_dir), '--show-missing'])
+    try:
+        subprocess.check_call([
+            sys.executable, '-m', 'coverage', 'report', '--fail-under',
+            str(threshold), '--include', '{0}/*'.format(pkg_dir),
+            '--show-missing'])
+    except subprocess.CalledProcessError as err:
+        print(err)
+        print('Test coverage on', pkg_dir,
+              'did not meet threshold of {0}%.'.format(threshold))
+        sys.exit(1)
 
 
 def main():
     description = """
 This script is used by tox.ini (and thus by Travis CI and Azure Pipelines) in
 order to generate separate stats for each package. It should be removed once
-those packages are moved to a separate repo.
-
-Option -e makes sure we fail fast and don't submit to codecov."""
+those packages are moved to a separate repo."""
     parser = argparse.ArgumentParser(description=description)
     parser.add_argument('--packages', nargs='+')