1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
|
# Documentation for the superpmi.py tool
SuperPMI is a tool for developing and testing the JIT compiler.
General information on SuperPMI can be found [here](../ToolBox/superpmi/readme.md).
## Overview
superpmi.py is a tool to simplify the use of SuperPMI.
The tool has three primary modes: collect, replay, and asmdiffs.
Below you will find more specific information on each of the different modes.
superpmi.py lives in the dotnet/runtime GitHub repo, src\coreclr\scripts directory.
## General usage
From the usage message:
```
usage: superpmi.py [-h] {collect,replay,asmdiffs,upload,download,list-collections,merge-mch} ...
Script to run SuperPMI replay, ASM diffs, and collections. The script also manages the Azure
store of pre-created SuperPMI collection files. Help for each individual command can be shown
by asking for help on the individual command, for example `superpmi.py collect --help`.
positional arguments:
{collect,replay,asmdiffs,upload,download,list-collections,merge-mch}
Command to invoke
optional arguments:
-h, --help show this help message and exit
```
## Replay
The simplest usage is to replay using:
```
python f:\gh\runtime\src\coreclr\scripts\superpmi.py replay
```
In this case, everything needed is found using defaults:
- The processor architecture is assumed to be the current default (e.g., x64).
- The build type is assumed to be Checked.
- Core_Root is found by assuming superpmi.py is in the normal location in the
clone of the repo, and using the processor architecture, build type, and current
OS, to find it in the default `artifacts` directory location. Note that you must
have performed a product build for this platform / build type combination, and
created the appropriate Core_Root directory as well.
- The SuperPMI tool and JIT to use for replay is found in Core_Root.
- The SuperPMI collections to use for replay are found in the Azure store of
precomputed collections for this JIT-EE interface GUID, OS, and processor architecture.
If you want to use a specific MCH file collection, use the `-mch_files` argument to specify
one or more MCH files on your machine:
```
python f:\gh\runtime\src\coreclr\scripts\superpmi.py replay -mch_files f:\spmi\collections\tests.pmi.windows.x64.Release.mch
```
The `-mch_files` argument takes a list of one or more directories or files to use. For
each directory, all the MCH files in that directory are used.
If you want to use just a subset of the collections, either default collections or collections
specified by `-mch_files`, use the `-filter` argument to restrict the MCH files used, e.g.:
```
python f:\gh\runtime\src\coreclr\scripts\superpmi.py replay -filter tests
```
## ASM diffs
To generate ASM diffs, use the `asmdiffs` command. This requires a "diff" and "baseline"
JIT. By default, the "diff" JIT is determined automatically as for the "replay" command,
described above. Also by default, the baseline JIT is determined based on comparing the
state of your branch with the `main` branch, and downloading an appropriate baseline JIT from the JIT
rolling build system. Alternatively, you can specify the path to a baseline JIT
compiler using the `-base_jit_path` argument.
Example:
```
python f:\gh\runtime\src\coreclr\scripts\superpmi.py asmdiffs
```
ASM diffs requires the coredistools library. The script attempts to find
or download an appropriate version that can be used.
As for the "replay" case, the set of collections used defaults to the set available
in Azure, or can be specified using the `mch_files` argument. In either case, the
`-filter` argument can restrict the set used.
## Collections
SuperPMI requires a collection to enable replay. You can do a collection
yourself using the superpmi.py `collect` command, but it is more convenient
to use existing precomputed collections stored in Azure.
You can see which collections are available for your current settings using
the `list-collections` command. You can also see all the available collections
using the `list-collections --all` command. Finally, you can see which Azure stored
collections have been locally cached on your machine in the default cache location
by using `list-collections --local`.
Note that when collections are downloaded, they are cached locally. If there are
any cached collections, then no download attempt is made. To force re-download,
use the `--force_download` argument to the `replay`, `asmdiffs`, or `download` command.
### Creating a collection
Example commands to create a collection (on Linux, by running the tests):
```
# First, build the product, possibly the tests, and create a Core_Root directory.
/Users/jashoo/runtime/src/coreclr/scripts/superpmi.py collect bash "/Users/jashoo/runtime/src/tests/runtest.sh x64 checked"
```
The above command collects over all of the managed code called by the
child process. Note that this allows many different invocations of any
managed code.
You can also collect using PMI instead of running code. Do with with the `--pmi` and `-assemblies`
arguments. E.g.:
```
python f:\gh\runtime\src\coreclr\scripts\superpmi.py collect --pmi -assemblies f:\assembly_store -output_mch_path f:\collections\my_collection.mch
```
You can alternatively collect by running `crossgen` on a set of assemblies, e.g.:
```
python f:\gh\runtime\src\coreclr\scripts\superpmi.py collect --crossgen -assemblies f:\assembly_store -output_mch_path f:\collections\my_collection.mch
```
You can specify both `--pmi` and `--crossgen` to do both types of collections across the
same set of specified assemblies, with the results being accumulated into a single collection.
Note that the collection process generates gigabytes of data. Most of this data will
be removed when the collection is finished. It is recommended to set the TEMP variable
to a location with adequate space, and preferably on a fast SSD to improve performance,
before running `collect` to avoid running out of disk space.
### Azure Storage collections
As stated above, you can use the `list-collections` command to see which collections
are available in Azure.
There is also a `download` command to download one or more Azure stored collection
to the local cache, as well as an `upload` command to populate the Azure collection
set.
|
