diff options
| -rw-r--r-- | README.md | 2 | ||||
| -rw-r--r-- | doc/html_generated/commandline.html | 1 | ||||
| -rw-r--r-- | doc/html_generated/configuration.html | 8 | ||||
| -rw-r--r-- | doc/html_generated/faq.html | 3 | ||||
| -rw-r--r-- | doc/html_generated/features.html | 8 | ||||
| -rw-r--r-- | doc/html_generated/parameterized-tests.html | 4 | ||||
| -rw-r--r-- | doc/html_generated/reporters.html | 83 | ||||
| -rw-r--r-- | doc/html_generated/roadmap.html | 34 | ||||
| -rw-r--r-- | doc/html_generated/stringification.html | 4 | ||||
| -rw-r--r-- | doctest/doctest.h | 6 | ||||
| -rw-r--r-- | doctest/parts/doctest_fwd.h | 6 | ||||
| -rw-r--r-- | examples/all_features/test_output/version.txt | 2 | ||||
| -rw-r--r-- | examples/all_features/test_output/version_xml.txt | 2 | ||||
| -rw-r--r-- | scripts/version.txt | 2 |
14 files changed, 89 insertions, 76 deletions
@@ -46,7 +46,7 @@ [](https://bestpractices.coreinfrastructure.org/projects/503) [](https://lgtm.com/projects/g/onqtam/doctest/context:cpp) [](https://gitter.im/onqtam/doctest?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) -[](https://wandbox.org/permlink/nRXesFZUIjiLBQa6) +[](https://wandbox.org/permlink/yuzdXqAzeMbmYwi6) <!-- [](https://isocpp.org/) [](https://github.com/onqtam/doctest/blob/master/doc/markdown/readme.md#reference) diff --git a/doc/html_generated/commandline.html b/doc/html_generated/commandline.html index ae1ebe0e..86844fcb 100644 --- a/doc/html_generated/commandline.html +++ b/doc/html_generated/commandline.html @@ -36,6 +36,7 @@ All the options can also be set with code (defaults/overrides) if the user [**su | ```-sc``` ```--subcase=<filters>``` | Same as ```--test-case=<filters>``` but filters subcases based on their names | | ```-sce``` ```--subcase-exclude=<filters>``` | Same as ```--test-case-exclude=<filters>``` but filters based on subcase names | | ```-r``` ```--reporters=<filters>``` | List of [**reporters**](reporters.html) to use (default is ```console```) | +| ```-o``` ```--out=<string>``` | Output filename | | ```-ob``` ```--order-by=<string>``` | Test cases will be sorted before being executed either by **the file in which they are** / **the test suite they are in** / **their name** / **random**. The possible values of ```<string>``` are ```file```/```suite```/```name```/```rand```. The default is ```file``` | | ```-rs``` ```--rand-seed=<int>``` | The seed for random ordering | | ```-f``` ```--first=<int>``` | The **first** test case to execute which passes the current filters - for range-based execution - see [**the example python script**](../../examples/range_based_execution.py) | diff --git a/doc/html_generated/configuration.html b/doc/html_generated/configuration.html index 7c7bfe63..ef9a2b55 100644 --- a/doc/html_generated/configuration.html +++ b/doc/html_generated/configuration.html @@ -19,7 +19,7 @@ Defining something ```globally``` means for every source file of the binary (exe - [**```DOCTEST_CONFIG_NUM_CAPTURES_ON_STACK```**](#doctest_config_num_captures_on_stack) - [**```DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING```**](#doctest_config_treat_char_star_as_string) - [**```DOCTEST_CONFIG_SUPER_FAST_ASSERTS```**](#doctest_config_super_fast_asserts) -- [**```DOCTEST_CONFIG_USE_IOSFWD```**](#doctest_config_use_iosfwd) +- [**```DOCTEST_CONFIG_USE_STD_HEADERS```**](#doctest_config_use_std_headers) - [**```DOCTEST_CONFIG_NO_COMPARISON_WARNING_SUPPRESSION```**](#doctest_config_no_comparison_warning_suppression) - [**```DOCTEST_CONFIG_OPTIONS_PREFIX```**](#doctest_config_options_prefix) - [**```DOCTEST_CONFIG_NO_UNPREFIXED_OPTIONS```**](#doctest_config_no_unprefixed_options) @@ -106,9 +106,9 @@ It also implies [**```DOCTEST_CONFIG_NO_TRY_CATCH_IN_ASSERTS```**](#doctest_conf This can be defined both globally and in specific source files only. -### **```DOCTEST_CONFIG_USE_IOSFWD```** +### **```DOCTEST_CONFIG_USE_STD_HEADERS```** -The library by default provides a forward declaration of ```std::ostream``` in order to support the ```operator<<``` [**stringification**](stringification.html) mechanism. This is forbidden by the standard (even though it works everywhere on all tested compilers). However if the user wishes to be 100% standards compliant - then this configuration option can be used to force the inclusion of ```<iosfwd>```. +The library by default provides a forward declaration of ```std::ostream``` in order to support the ```operator<<``` [**stringification**](stringification.html) mechanism (also ```std::tuple<>``` and ```std::nullptr_t```). This is forbidden by the standard (even though it works everywhere on all tested compilers). However if the user wishes to be 100% standards compliant - then this configuration option can be used to force the inclusion of the relevant standard headers. Also it is possible that some STL implementation of a compiler with niche usage defines them differently - then there will be compilation errors in STL headers and using this option should fix the problem. @@ -154,7 +154,7 @@ This can be defined both globally and in specific source files only. ### **```DOCTEST_CONFIG_NO_EXCEPTIONS```** -This will remove everything that uses exceptions from the framework - it is also auto detectable for some compilers (GCC, Clang) if exceptions are disabled with ```-fno-exceptions```. For MSVC ```_HAS_EXCEPTIONS``` cannot be used for auto detecting because it is defined in system headers instead of as a project define - and doctest will not include a header just for that. +This will remove everything that uses exceptions from the framework - it is also auto detectable if exceptions are disabled for compilers (like with ```-fno-exceptions``` for GCC/Clang). What gets changed: diff --git a/doc/html_generated/faq.html b/doc/html_generated/faq.html index 5c316b71..bbc5d8ca 100644 --- a/doc/html_generated/faq.html +++ b/doc/html_generated/faq.html @@ -37,7 +37,6 @@ Aside from everything mentioned so far doctest has some [**features**](features. Missing stuff: -- xml reporter - matchers and generators - other small stuff @@ -142,7 +141,7 @@ Yes - but they can be disabled - see the [**```DOCTEST_CONFIG_NO_EXCEPTIONS```** ### Why do I get compiler errors in STL headers when including the doctest header? -Try using the [**```DOCTEST_CONFIG_USE_IOSFWD```**](configuration.html#doctest_config_use_iosfwd) configuration identifier. +Try using the [**```DOCTEST_CONFIG_USE_STD_HEADERS```**](configuration.html#doctest_config_use_std_headers) configuration identifier. ### Can different versions of the framework be used within the same binary (executable/dll)? diff --git a/doc/html_generated/features.html b/doc/html_generated/features.html index af76df74..f566b520 100644 --- a/doc/html_generated/features.html +++ b/doc/html_generated/features.html @@ -28,8 +28,8 @@ - Standards compliant **C++11** code - should work with any **C++11** capable compiler (use tag [**1.2.9**](https://github.com/onqtam/doctest/tree/1.2.9) for C++98 and older compilers) - tested with **GCC**: **4.8**, **4.9**, **5**, **6**, **7**, **8** -- tested with **Clang**: **3.5**, **3.6**, **3.7**, **3.8**, **3.9**, **4**, **5**, **6**, **7** (XCode 8+) -- tested with **MSVC**: **2015**, **2017** (also in 32 bit mode) +- tested with **Clang**: **3.5**, **3.6**, **3.7**, **3.8**, **3.9**, **4**, **5**, **6**, **7**, **8** (XCode 6+) +- tested with **MSVC**: **2015**, **2017**, **2019** (also in 32 bit mode) - per-commit tested on [**travis**](https://travis-ci.org/onqtam/doctest) and [**appveyor**](https://ci.appveyor.com/project/onqtam/doctest) CI services - warnings as errors even on the most aggressive warning levels - see [**here**](../../scripts/cmake/common.cmake#L84) - statically analyzed on the CI - [**Cppcheck**](http://cppcheck.sourceforge.net/) / [**Clang-Tidy**](http://oclint.org/) / [**Coverity Scan**](https://scan.coverity.com/) / [**OCLint**](https://scan.coverity.com/) / [**Visual Studio Analyzer**](https://docs.microsoft.com/en-us/visualstudio/code-quality/analyzing-c-cpp-code-quality-by-using-code-analysis) @@ -37,8 +37,8 @@ - all tests built and ran in **Debug**/**Release** modes - all tests ran through **valgrind** under **Linux** (sadly [not under OSX](https://github.com/onqtam/doctest/issues/11)) - all tests ran through **address**, **UB** and **thread** sanitizers under **Linux**/**OSX** - - tests are ran in more than **140** different configurations on UNIX (Linux + OSX) on **travis** CI - - tests are ran in a total of **10** different configurations on Windows on **appveyor** CI + - tests are ran in more than **150** different configurations on UNIX (Linux + OSX) on **travis** CI + - tests are ran in a total of **14** different configurations on Windows on **appveyor** CI ## Other features: diff --git a/doc/html_generated/parameterized-tests.html b/doc/html_generated/parameterized-tests.html index e3bf51a1..5b750df2 100644 --- a/doc/html_generated/parameterized-tests.html +++ b/doc/html_generated/parameterized-tests.html @@ -130,9 +130,9 @@ There are 2 ways to do it: CHECK(var == -1); } - TEST_CASE_TEMPLATE_INSTANTIATE(test_id, char, short, int, long long int); + TEST_CASE_TEMPLATE_INVOKE(test_id, char, short, int, long long int); - TEST_CASE_TEMPLATE_INSTANTIATE(test_id, float, double); + TEST_CASE_TEMPLATE_APPLY(test_id, std::tuple<float, double>); ``` If you are designing an interface or concept, you can define a suite of type-parameterized tests to verify properties that any valid implementation of the interface/concept should have. Then, the author of each implementation can just instantiate the test suite with his type to verify that it conforms to the requirements, without having to write similar tests repeatedly. diff --git a/doc/html_generated/reporters.html b/doc/html_generated/reporters.html index e1216cbe..99c6262c 100644 --- a/doc/html_generated/reporters.html +++ b/doc/html_generated/reporters.html @@ -5,59 +5,86 @@ ## Reporters -A very sloppy documentation of the partial reporters support. +Doctest has a modular reporter/listener system with which users can write their own reporters and register them. The reporter interface can also be used for "listening" to events. + +You can list all registered reporters with ```--list-reporters```. There are a few implemented reporters in the framework: +- ```console``` - streaming - writes normal lines of text with coloring if a capable terminal is detected +- ```xml``` - streaming - writes in xml format tailored to doctest + +Streaming means that results are delivered progressively and not at the end of the test run. + +The output is by default written to ```stdout``` but can be redirected with the use of the ```--out=<filename>``` [**command line option**](commandline.html). Example how to define your own reporter: ``` +#include <doctest/doctest.h> + +#include <mutex> + using namespace doctest; -struct XmlReporter : public IReporter +struct MyXmlReporter : public IReporter { - std::ostream& s; - std::vector<SubcaseSignature> subcasesStack; - - // caching pointers to objects of these types - safe to do - const ContextOptions* opt; + // caching pointers/references to objects of these types - safe to do + std::ostream& stdout_stream; + const ContextOptions& opt; const TestCaseData* tc; + std::mutex mutex; + + // constructor has to accept the ContextOptions by ref as a single argument + MyXmlReporter(const ContextOptions& in) + : stdout_stream(*in.cout) + , opt(in) {} - XmlReporter(std::ostream& in) - : s(in) {} + void report_query(const QueryData& /*in*/) override {} - void test_run_start(const ContextOptions& o) override { opt = &o; } + void test_run_start() override {} - void test_run_end(const TestRunStats& /*p*/) override {} + void test_run_end(const TestRunStats& /*in*/) override {} void test_case_start(const TestCaseData& in) override { tc = ∈ } - void test_case_end(const CurrentTestCaseStats& /*st*/) override {} + void test_case_end(const CurrentTestCaseStats& /*in*/) override {} - void subcase_start(const SubcaseSignature& subc) override { subcasesStack.push_back(subc); } + void test_case_exception(const TestCaseException& /*in*/) override {} - void subcase_end(const SubcaseSignature& /*subc*/) override { subcasesStack.pop_back(); } + void subcase_start(const SubcaseSignature& /*in*/) override {} - void log_assert(const AssertData& /*rb*/) override {} + void subcase_end() override {} - void log_message(const MessageData& /*mb*/) override {} + void log_assert(const AssertData& in) override { + // don't include successful asserts by default - this is done here + // instead of in the framework itself because doctest doesn't know + // if/when a reporter/listener cares about successful results + if(!in.m_failed && !opt.success) + return; - void test_case_skipped(const TestCaseData& /*in*/) override {} -}; + // make sure there are no races - this is done here instead of in the + // framework itself because doctest doesn't know if reporters/listeners + // care about successful asserts and thus doesn't lock a mutex unnecessarily + std::lock_guard<std::mutex> lock(mutex); -XmlReporter r(std::cout); -DOCTEST_REGISTER_REPORTER("xml", 1, r); -``` + // ... + } -Multiple reporters can be used at the same time - just specify them through the ```--reporters=...``` [**command line option**](commandline.html). The number ```1``` in this case is the priority - reporters will be called in the order defined by their priority when a few of them are selected to be used at the same time. + void log_message(const MessageData& /*in*/) override { + // messages too can be used in a multi-threaded context - like asserts + std::lock_guard<std::mutex> lock(mutex); -You can list all registered reporters with ```--list-reporters```. There is only 1 implemented reporter in the framework - a console reporter - an xml one is coming in the next version. the reporter interface can also be used for "listening" to events. + // ... + } -Reporters will be fully implemented and more thoroughly documented (with examples) for version 2.1 - [**roadmap**](roadmap.html). + void test_case_skipped(const TestCaseData& /*in*/) override {} +}; + +// "1" is the priority - used for ordering when multiple reporters/listeners are used +REGISTER_REPORTER("my_xml", 1, MyXmlReporter); +``` -TODO: think about: +Multiple reporters can be used at the same time - just specify them through the ```--reporters=...``` [**command line filtering option**](commandline.html) using commas to separate them like this: ```--reporters=myListener,xml``` and their order of execution will be based on their priority - that is the number "1" in the case of the example reporter above (lower means earlier - the default console/xml reporters from the framework have 0 as their priority and negative numbers are accepted as well). -- people having to handle locking on their own in the reporters - just like currently in the console reporter... - - or log_assert() can be called for reporters (so also for listeners) only when failed or when printing on success! - - or it could be a 'policy' for the reporter +When implementing a reporter users are advised to follow the comments from the example above and look at the few implemented reporters in the framework itself. Also check out the [**example**](../../examples/all_features/reporters_and_listeners.cpp). --------------- diff --git a/doc/html_generated/roadmap.html b/doc/html_generated/roadmap.html index d0af0227..76b502c1 100644 --- a/doc/html_generated/roadmap.html +++ b/doc/html_generated/roadmap.html @@ -12,39 +12,23 @@ This library is free, and will stay free but needs your support to sustain its d Planned features for future releases - order changes constantly... -### For 2.3: - -- finish the reporter system - - what is done: - - can be used for listening for events - - multiple reporters can be used - - custom reporters can be written - - register and choose reporters - - list all reporters - - what is left: - - output to file (or just not stdout) - - xml output - - xUnit reporter - - compact reporter - https://github.com/onqtam/doctest/issues/75 - - progress reporter - - [junit related stuff](https://github.com/ujiro99/doctest-junit-report)) - - options - - absolutely no output on success - - summary only - - make sure all relevant parts of the reporters can be accessed from the doctest namespace without the need for detail - - examples - - documentation -- matchers - should investigate what they are - look at google test/mock and Catch (also predicates and boost test) - ### For 2.4: +- https://github.com/onqtam/doctest/issues/208 +- reporters: + - xUnit/junit/TeamCity reporter - perhaps related: https://github.com/ujiro99/doctest-junit-report + - compact reporter + - options + - absolutely no output on success (AKA quiet mode) + - summary only (verbosity levels?) +- matchers - should investigate what they are - look at google test/mock and Catch (also predicates and boost test) - header with extensions - demangling with the use of the cxxabi header - stringification of types from std, also enums with the help of traits as discussed in #121 - esoteric reporters - convolution support for the assertion macros (with a predicate) - Value-Parameterized test cases -- generators? - look at Catch - and investigate what they are +- generators? - look at Catch - and investigate what they are (also SUBCASEs can be while() loops instead of if() statements! that might be useful...) - look at property based testing - [rapidcheck](https://github.com/emil-e/rapidcheck) - [autocheck](https://github.com/thejohnfreeman/autocheck) diff --git a/doc/html_generated/stringification.html b/doc/html_generated/stringification.html index 1cfe0b0e..894b5c8f 100644 --- a/doc/html_generated/stringification.html +++ b/doc/html_generated/stringification.html @@ -85,11 +85,13 @@ An alternative way to register an exception translator is to do the following in The order of registering exception translators can be controlled - simply call the explicit function in the required order or list the exception translators with the macro in a top-to-bottom fashion in a single translation unit - everything that auto-registers in doctest works in a top-to-bottom way for a single translation unit (source file). +You could also [override the translation mechanism](https://github.com/catchorg/Catch2/issues/539#issuecomment-454549904) for exceptions deriving from ```std::exception```. + ------ - Check out the [**example**](../../examples/all_features/stringification.cpp) which shows how to stringify ```std::vector<T>``` and other types/exceptions. - Note that the type ```String``` is used when specializing ```StringMaker<T>``` or overloading ```toString()``` - it is the string type **doctest** works with. ```std::string``` is not an option because doctest would have to include the ```<string>``` header. -- To support the ```operator<<(std::ostream&...``` stringification the library has to offer a forward declaration of ```std::ostream``` and that is what the library does - but it is forbidden by the standard. It currently works everywhere - on all tested compilers - but if the user wishes to be 100% standards compliant - then the [**```DOCTEST_CONFIG_USE_IOSFWD```**](configuration.html#doctest_config_use_iosfwd) identifier can be used to force the inclusion of ```<iosfwd>```. The reason the header is not included by default is that on MSVC (for example) it drags a whole bunch of stuff with it - and after the preprocessor is finished the translation unit has grown to 42k lines of C++ code - while Clang and the libc++ are so well implemented that including ```<iosfwd>``` there results in 400 lines of code. +- To support the ```operator<<(std::ostream&...``` stringification the library has to offer a forward declaration of ```std::ostream``` and that is what the library does - but it is forbidden by the standard. It currently works everywhere - on all tested compilers - but if the user wishes to be 100% standards compliant - then the [**```DOCTEST_CONFIG_USE_STD_HEADERS```**](configuration.html#doctest_config_use_std_headers) identifier can be used to force the inclusion of ```<iosfwd>```. The reason the header is not included by default is that on MSVC (for example) it drags a whole bunch of stuff with it - and after the preprocessor is finished the translation unit has grown to 42k lines of C++ code - while Clang and the libc++ are so well implemented that including ```<iosfwd>``` there results in 400 lines of code. --- diff --git a/doctest/doctest.h b/doctest/doctest.h index 71684139..b894efe8 100644 --- a/doctest/doctest.h +++ b/doctest/doctest.h @@ -47,9 +47,9 @@ // ================================================================================================= #define DOCTEST_VERSION_MAJOR 2 -#define DOCTEST_VERSION_MINOR 2 -#define DOCTEST_VERSION_PATCH 3 -#define DOCTEST_VERSION_STR "2.2.3" +#define DOCTEST_VERSION_MINOR 3 +#define DOCTEST_VERSION_PATCH 0 +#define DOCTEST_VERSION_STR "2.3.0" #define DOCTEST_VERSION \ (DOCTEST_VERSION_MAJOR * 10000 + DOCTEST_VERSION_MINOR * 100 + DOCTEST_VERSION_PATCH) diff --git a/doctest/parts/doctest_fwd.h b/doctest/parts/doctest_fwd.h index 066af547..f06e5f08 100644 --- a/doctest/parts/doctest_fwd.h +++ b/doctest/parts/doctest_fwd.h @@ -44,9 +44,9 @@ // ================================================================================================= #define DOCTEST_VERSION_MAJOR 2 -#define DOCTEST_VERSION_MINOR 2 -#define DOCTEST_VERSION_PATCH 3 -#define DOCTEST_VERSION_STR "2.2.3" +#define DOCTEST_VERSION_MINOR 3 +#define DOCTEST_VERSION_PATCH 0 +#define DOCTEST_VERSION_STR "2.3.0" #define DOCTEST_VERSION \ (DOCTEST_VERSION_MAJOR * 10000 + DOCTEST_VERSION_MINOR * 100 + DOCTEST_VERSION_PATCH) diff --git a/examples/all_features/test_output/version.txt b/examples/all_features/test_output/version.txt index fd81b480..75c9402d 100644 --- a/examples/all_features/test_output/version.txt +++ b/examples/all_features/test_output/version.txt @@ -1 +1 @@ -[doctest] doctest version is "2.2.3" +[doctest] doctest version is "2.3.0" diff --git a/examples/all_features/test_output/version_xml.txt b/examples/all_features/test_output/version_xml.txt index ad0544e6..d899ccff 100644 --- a/examples/all_features/test_output/version_xml.txt +++ b/examples/all_features/test_output/version_xml.txt @@ -1,4 +1,4 @@ <?xml version="1.0" encoding="UTF-8"?> -<doctest binary="all_features" version="2.2.3"> +<doctest binary="all_features" version="2.3.0"> <Options order_by="file" rand_seed="324" first="0" last="4294967295" abort_after="0" subcase_filter_levels="2147483647" case_sensitive="false" no_throw="false" no_skip="false"/> </doctest> diff --git a/scripts/version.txt b/scripts/version.txt index 6b4d1577..cc6612c3 100644 --- a/scripts/version.txt +++ b/scripts/version.txt @@ -1 +1 @@ -2.2.3
\ No newline at end of file +2.3.0
\ No newline at end of file |