Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/mono/corefx.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJose Perez Rodriguez <joperezr@microsoft.com>2016-11-02 19:34:06 +0300
committerGitHub <noreply@github.com>2016-11-02 19:34:06 +0300
commitbbb84f2e779fa1e8e588a5a1d6d43f43f6a0bfd0 (patch)
treecf78ea41d709fa5953db4c5287e77d17ad3fab9d /Documentation/project-docs
parentf8a950e5ab23f22a16aaad75156080a0db48e9f4 (diff)
Adding more documentation on how to run individual library tests from the command line (#12771)
* Adding more documentation on how to run individual library tests from the command line * PR Feedback * Adding info about TestTFM to the Project guidelines * Adding TargetGroup - TestTFMs supported table
Diffstat (limited to 'Documentation/project-docs')
-rw-r--r--Documentation/project-docs/developer-guide.md52
1 files changed, 51 insertions, 1 deletions
diff --git a/Documentation/project-docs/developer-guide.md b/Documentation/project-docs/developer-guide.md
index 8743f7dc02..54557cc9ac 100644
--- a/Documentation/project-docs/developer-guide.md
+++ b/Documentation/project-docs/developer-guide.md
@@ -154,9 +154,20 @@ If the tests are already built and you only want to run them, invoke `run-tests`
For more information about cross-platform testing, please take a look [here](https://github.com/dotnet/corefx/blob/master/Documentation/building/cross-platform-testing.md).
-You can also run the tests for an individual project by building it individually, e.g.:
+If you are interested in building and running the tests only for a specific library, then there are two different ways to do it:
+The easiest (and recommended) way to do it, is by simply building the test .builds file for that library.
+
+```cmd
+cd src\System.Collections.Immutable\tests
+msbuild System.Collections.Immutable.Tests.builds
```
+
+What that will do is to build the tests against all of the available configurations, and then apply some filtering to run only the project configurations that support the default TestTFM(more about this on the next section). This is the prefered way since even though we only run the tests on a specific TFM, we will cross compile the tests against all its available configurations, which is good in order to make sure that you are not breaking the test build. This is also how CI will run the tests, so if you are able to build the .builds file succesfully, chances are that CI will too.
+
+The second way to do it, is to build the .csproj and select to run the `BuildAndTest` target:
+
+```cmd
cd src\System.Collections.Immutable\tests
msbuild /t:BuildAndTest (or /t:Test to just run the tests if the binaries are already built)
```
@@ -171,6 +182,45 @@ There may be multiple projects in some directories so you may need to specify th
Tests participate in the incremental build. This means that if tests have already been run, and inputs to the incremental build have not changed, rerunning the tests target will not execute the test runner again. To force re-executing tests in this situation, use `msbuild /t:RebuildAndTest`.
+#### What is TestTFM and what possible values can it have
+
+`TestTFM` is the Framework that we will use to run your tests on. The same test assembly can be used to run tests on two different frameworks, for example, AssemblyWithMyTests.dll can be used to run your tests in TFM `A` and `B` as long as framworks `A` and `B` provide/support the NetStandard surface area needed for the test assembly to compile. For this reason, when you write your tests, you might want to run them against different frameworks, and the next section will point out how to do that.
+
+Some of the possible values for `TestTFM` are:
+
+_**`netcoreapp1.1` or `netcoreapp1.0`**_
+NetStandard implementations that run in CoreCLR.
+
+_**`netcore50` or `uap101aot`**_
+NetStandard implementations for UWP.
+
+-**`net46` or `net462` or `net463`**-
+NetStandard implementations for Desktop.
+
+#### Running tests in a different TFM
+
+Each test project corresponds to a test .builds file. There are some tests that might be OS specific, or might be testing API that is available only on some TFMs, which is what this tests.builds files are for. By default, we will build all of these different configurations always, but we will only execute the tests on one TFM. By default, our `TestTFM` is set to `netcoreapp1.1` [here](https://github.com/dotnet/corefx/blob/master/dir.props#L495) which means that we will run tests for the configurations that either:
+- Have `netcoreapp1.1` inside their `<TestTFMs>...</TestTFMs>` clause on the .builds files, or
+- Don't have a `<TestTFMs>...</TestTFMs>` metadata in the .builds file at all because we default it to netcoreapp1.1 [here](https://github.com/dotnet/corefx/blob/master/dir.traversal.targets#L146-L147)
+
+The rest of the configurations will still get built, but they won't be executed by default, or as part of the CI. In order to use a different TestTFM, pass in the `FilterToTestTFM` property like:
+
+```cmd
+cd src\System.Runtime\tests
+msbuild System.Runtime.Tests.builds /p:FilterToTestTFM=net462
+```
+
+The previous example will again build the System.Runtime csproj in all of it's different configurations, but will only execute the tests that have `net462` in their `<TestTFMs>...</TestTFMs>` metadata on System.Runtime.Tests.builds
+
+One more way to run tests on a specific TFM, is to do it by building the test csproj directly and set the value of `TestTFM` like:
+
+```cmd
+cd src\System.Runtime\tests
+msbuild System.Runtime.Tests.csproj /t:BuildAndTest /p:TestTFM=net462
+```
+
+#### Filtering tests using traits
+
The tests can also be filtered based on xunit trait attributes defined in [`xunit.netcore.extensions`](https://github.com/dotnet/buildtools/tree/master/src/xunit.netcore.extensions). These attributes are to be specified over the test method. The available attributes are:
_**`OuterLoop`:**_