| Age | Commit message (Collapse) | Author |
|---|
|
Also enabled cross compilation of x86 binaries on x64 host.
Ubuntu Dockerfiles:
<details>
<summary><b><ins>Baseline: x64 build on x64 host</ins></b></summary>
```dockerfile
FROM ubuntu
RUN cat /etc/*-release
RUN apt-get update
RUN apt-get install -y \
autoconf bash clang cmake gcc libtool curl \
libunwind-dev llvm make openssl lldb git uuid-dev
RUN git clone https://github.com/dotnet/corert -b master --single-branch
WORKDIR /corert
RUN ./build.sh # or ./build.sh x64
```
</details>
<details>
<summary><b><ins>PR: x86 build on x86 host</ins></b></summary>
```dockerfile
FROM i386/ubuntu
RUN cat /etc/*-release
RUN apt-get update
RUN apt-get install -y \
bash clang cmake gcc libtool curl \
libunwind-dev llvm make openssl lldb git uuid-dev
RUN git clone https://github.com/am11/corert -b linux-x86 --single-branch
WORKDIR /corert
RUN ./build.sh x86
```
</details>
<details>
<summary><b><ins>PR: x86 build on x64 host</ins></b></summary>
```dockerfile
FROM ubuntu
RUN cat /etc/*-release
RUN apt-get update
RUN apt-get install -y \
bash clang cmake gcc libtool curl \
libunwind-dev llvm make openssl lldb git uuid-dev \
g++-multilib
RUN git clone https://github.com/am11/corert -b linux-x86 --single-branch
WORKDIR /corert
RUN ./build.sh x86
```
</details>
|
|
|
|
`FastAllocateString` (the choke point through which all string allocations go through) wasn't as fast as it could be and we were 30% slower than CLR on allocating strings. We were leaving a lot of perf on the table.
Before this change, string allocation was using the same allocator as arrays. Since there's a subtle difference between the failure modes on overflow (string allocation throws OOM, array allocation throws OverflowException), `FastAllocateString` required a try/catch block to handle the corner case. This was inhibiting codegen optimizations around this code path - to fix that problem, we needed a separate allocator. And since we now had a separate allocator for strings, I also took the liberty of inlining some details around strings (component size and base size) into the helper. It turns out runtime already hardcodes the details around strings (the component size) in a couple places anyway, so this is not that big of a "separation of concerns" violation as it looks like.
[tfs-changeset: 1670224]
|
|
[tfs-changeset: 1664997]
|
|
MRT helpers
This change unifies StackFrameIterator behavior w.r.t. to special handling of various internal addresses within MRT helpers to the way I fixed this before for the UniversalTransitionThunk i.e. instead of exporting a public symbol in the middle of the method which is known to confuse the DIA stack unwinder, we export the pointer by means of an auxiliary data variable.
[tfs-changeset: 1661485]
|
|
Today our ProjectN images are not CFG-enabled, but CFG will kick in for applications that have CFG enabled, in particular, WWAHost (for JavaScript apps). Today with those apps and dynamic interop enabled we would trigger a CFG check as those thunks are never registered under CFG.
There are a couple of ways to achieve this. 1) You could take advantage of the CFG table which OS loader understands and would remap in MapViewOfFile. In order to make those thunks show up in CFG, you need to give them symbolic names. According to my testing, it only seem to work if we map the full mrt100_app.dll, which isn't ideal. 2) Use SetProcessValidCallTargets and supply the thunks. I went with SetProcessValidCallTargets, which works, but has its own set of challenges (if we discover that OS matrix we need to support with CFG becomes unsustainable with this approach we can revisit):
1. Our ARM test environment is on 8.1, while SetProcessValidCallTargets is Win10+.
Normally you would fix this by either 1) LoadLibrary/GetProcAddress or 2) delay loading. Unfortunately none of these work today because LoadLibrary is not part of WACK (and has a non-trivial cost to support) and 2) SetProcessValidCallTargets is on kernel32 and not delay loadable.
Fortunately there is a API set available load api-ms-win-core-memory-l1-1-3.dll which will has this API. It is effectively forwarded to kernelbase.dll but for the purpose of delay loading it works perfectly. When the API is not found, we would redirect the API to a predefined failure stub that simply returns true.
It is worth noting that if we opt to use LoadLibrary/GetProcAddress (which you can't do under UWP), calling SetProcessValidCallTarget would trigger CFG check as it is declared __declspec(guard(nocf)) for security reasons. Fortunately delay loading works fine.
2. Calling SetProcessValidCallTargets would fail on non-CFG-enabled processes. This can be fixed by calling GetProcessMitigationPolicy to determine whether the process is CFG-enabled.
3. SetProcessValidCallTargets require our thunks to be 0x10 aligned. x64/ARM is already 0x10 aligned but x86 is only aligned on 0x4. This means we'll need to waste 0xc for each thunk on x86. This a bit unfortunate but I doubt anyone would care. If this ends up wasting too much space we can revisit and optimize as needed.
4. SetProcessValidCallTargets is not allowed in WACK today. Fortunately it is already on pre-approved WACK list for RS2. It does mean that you won't be able to submit apps with mrt100_app.dll today but that's probably fine - our plan for the new toolchain is RS3+.
5. Our toolsets in razzle enlistment only has 8.1 the latest. For now I've work arounded by manually declaring the imports and copy the latest mincore.lib to rh/public/minwin folder. This has a unfortunate side effect that it also 'upgrades' RaiseFailFastException to a new API set that is not available down-level 8.1. I've made RaiseFailFastException also delay loading so that we wouldn't immediately fail to load under ARM + Win 8.1. I doubt any one cares about how N failfast under 8.1.
6. Similarly, ToF also has a copy of Win8 toolset in its own ExternalAPI (not the razzle one) and it doesn't support creating a CFG-aware binary. One possiblity is to use JavaScript app like in the bug, but you also need a reliable way to construct thunks and send it over to javaScript as v-table so that JavaScript can call it. This is basically dyanmic interop feature so the best way to cover this today without CFG-aware toolset is in dyanmic interop testing with full dyanmic interop mode on (everything dynamic). Once we have a CFG-aware toolset (I suspect this would only happen when we move out of ToF), we can have the main C++ app call reverse p/invoke thunks.
I've introduced a new API PalMarkThunksAsValidCallTargets to do the heavylifting - to allocate the correct data structure ro register the thunks and call SetProcessValidCallTargets. Rest of the stuff are less interesting.
After this change, I've verified the Yahoo app with dynamic interop turned on can proceed further (but still eventually timed out). I've also done a few more testing:
* ARM 8.1 private interop run,
* x86 private interop run,
* thunkpool tests
* x86chk precheckin
* verified ARM phone does have SetProcessValidCallTargets API.
* Build and ran all CoreRT tests
[tfs-changeset: 1655595]
|
|
Native
This change unifies the runtime delegate marshalling infrastructure between CoreRT and .NET Native by removing duplicate codes and moving shared helpers to PInvokeMarshal in CoreLib. I got rid of the InteropThunskHelpers from Interop.Native directory and instead used the ones already ported to CoreRT. With this unification the delegate marshalling memory leak issue(which was caused by not having a cache) should be gone.
[tfs-changeset: 1654801]
|
|
According to recent findings current Windows DIA stackwalker malfunctions at the address ReturnFromUniversalTransition_DebugStepTailCall because it misinterprets the label for the beginning of a new method and assumes that RSP / ESP points to the return address. As the ingestion of an updated DIA version is a longer-term process I'm proposing to temporarily fix this by exporting a temporary variable holding the return address instead.
[tfs-changeset: 1653716]
|
|
[tfs-changeset: 1653409]
|
|
Initital support for delegate marshalling.
- Enabled only in full runtime
- Doesn't work in non-windows platforms
- Only supports open static delegates
|
|
Merge nmirror to master
|
|
Handling of hardware exceptions had a hack to add +1 to the actual instruction IP. Windows x64 unwinder
is disassembling instructions at the IP passed in to detect method epilogs. If the bytes at IP + 1
happened to match the epilog pattern, the unwind is done as if we were in the middle of the epilog that
lead to spectacular crash.
This change is moving this adjustment to be done later for EH related things only, and not interfere
with stackwalking.
Fixes #2535
[tfs-changeset: 1645602]
|
|
|
|
CoreRT, and adding all the conversion helper stubs
|
|
RhpCastableObjectDispatchHelper is jumped-to from RhpInterfaceDispatchX functions. The RhpInterfaceDispatchX were not preserving r12 on ARM / eax on x86 that contain the address of the indirection cell, which lead to failures.
The fix is to let RhpCastableObjectDispatchHelper accept the cache entry pointer in r12 / eax instead.
I have also updated the CastableObject.cs based on Fadi Hanna's suggestion so that it can catch the issue this change fixes.
[tfs-changeset: 1641040]
|
|
[tfs-changeset: 1640866]
|
|
_RhpCastableObjectDispatchHelper is jumped-to from RhpInterfaceDispatchX functions. The RhpInterfaceDispatchX were not preserving r12 on ARM / eax on x86 that contain the address of the indirection cell.
[tfs-changeset: 1640787]
|
|
allocated dynamically in memory using VirtualAlloc. The thunk stubs pages are marked with RX permissions and the thunks data pages are marked with RW permissions.
Refactored some code in ThunkPool.cs to remove from it any knowledge about the thunk sections layout. Thunks layout and allocation become a black box to ThunkPool.cs. All the section layout information is moved to the native component, and section navigation is done by calls to some APIs.
The dynamically allocated thunks pages are enabled by a FEATURE_RX_THUNKS, and only in CoreRT's Full build for now. The portable build doesn't yet support thunks.
[tfs-changeset: 1638131]
|
|
Removing the hardcoded number, and making it into a function call.
[tfs-changeset: 1636433]
|
|
1) Removing the ThunkPool APIs from the System.Private.Corelib layer and moving it to mrt100_app.dll. Reasons:
a) A more correct layering
b) Thunks are also used by mrt100_app, and today it has to call into the S.P.Corelib layer to get thunks
c) The binder (which today takes care of laying out the thunks section) is going away with the ProjectX work, so the refactoring is needed
2) Addition of a new API: CreateThunksHeap. Thunks are allocated and deallocated from heaps, and each caller can create and managed their own heaps (TODO: heap deallocation feature)
Note: The use of a simple heap removes the need for a the hashtable that we previously had in the old implementation, and simplifies the implementation a bit.
4) Optimization (use of simple thunks linked list with better perf characteristics)
5) Added ThunkPool APIs into the internal RuntimeAugments assembly, to enable writing of tests that directly target the thunks APIs
6) Tests for the feature (including a multithreaded test).
Tested on all 3 architectures.
[tfs-changeset: 1634032]
|
|
RhpLoopHijack is called in a call chain of RhpGcStressProbe. This can happen during GC stress when GC calls GCToEEInterface::GcStartWork, which calls RestrictedCallouts::InvokeGcCallouts and that calls managed System::Runtime::InteropServices::RCWWalker.OnGCStarted, where there is a loop with with injected RhpLoopHijack down the call chain.
[tfs-changeset: 1633416]
|
|
opcode (or similar), which end up resolving interface methods using the RhpResolveInterfaceMethod API.
[tfs-changeset: 1632820]
|
|
saving callee saved registers in the RhpCallFilterFunclet asm helper and the NUTC doesn't save callee saved registers in the filter funclet. So when the FindFirstPassHandler called the filter, the filter have modified ESI, but the FindFirstPassHandler code was expecting to be preserved and so it failed to unwind properly.
It also adds saving callee saved floating point registers to windows amd64 and arm helpers
[tfs-changeset: 1631842]
|
|
resolution from a metadata token
- Instead of an interface/slot pair
- Cell information is now consolidated into a structure instead of being passed around as a tuple of interface/slot
- Encoding is more complex to avoid bloating the data format
- In addition to interface dispatch, the logic is also able to perform vtable dispatch
- Add support for the concept of dynamic modules
- A dynamic module provides a set of callbacks that are special around the behavior of interface dispatch
- ModuleList associates a DynamicModule with normal module. At some point we will consolidate the DynamicModule with the ModuleManager
Miscellaneous changes
- New variant of LockFreeReaderHashtable to be used with native pointers.
- Support for a cloned type to be cloned based on a direct pointer instead of an indirection
[tfs-changeset: 1630711]
|
|
Rewrote some of it in C++ or C#
[tfs-changeset: 1630452]
|
|
Wrighton, I have added support for x86 and ARM32 and investigated few general issues that surfaced during testing.
1. Logic to convert interface dispatch to use universal transition thunk instead of managedcalloutthunk. (This allows interface dispatch to safely throw exceptions if it needs to.) (This is implemented for all platforms)
2. Poor man’s interface type equivalence to allow the new ICastableObject interface to be called directly from the runtime instead of building a specialized calling routine. (We should probably consider implementing the general thing, but this shelveset is big enough as it is.)
3. Small feature adds around COFF emission of pdb records to allow the runtime to have an interface dispatch in it.
4. New CastableObject class
5. Tweak to reflection to allow both ICastable and CastableObject interface dispatch routines to work.
6. Test case for CastableObject
7. Assembly stubs and thunks to make castable object implementation work.
[tfs-changeset: 1630177]
|
|
- Clean up CORERT ifdefs around Object.GetRawData
- Replace some cases of m_pEEType pinning with Object.GetRawData. There is still a bunch left - once all of them are cleaned up, Object.m_pEEType can be made private.
- While I was on it, uncomment Array.AsReadOnly<T> implementation since it is going to be needed by netstandard 2.0 work
[tfs-changeset: 1627495]
|
|
to ProjectN. The nature of the changes in this changelist is very similar to the changes done in CoreCLR described in the linked PR:
1) This change synchronizes the ndp/FxCore/CoreRT/Native/gc and src/vm/gc directories of ProjectN and coreclr, respectively. There is one additional change present in this changelist that was not done to CoreCLR (and will need to be ported to CoreCLR), and that is the addition of IGCHeap::RegisterFrozenSegment and IGCHeap::UnregisterFrozenSegment to the interface. These functions are invoked from RedhawkGCInterface::RegisterFrozenSection and as such must be present on the interface. This is a very minor change that involved ensuring that Register/UnregisterFrozenSegment are defined for all builds (not just defined when FEATURE_BASICFREEZE is defined) and propagating the definition of those two functions to IGCHeap, instead of IGCHeapInternal where they resided on CoreCLR. Calling either of those two methods with FEATURE_BASICFREEZE not defined results in an assert.
2) Like the changes to the VM in CoreCLR, uses of GCHeap in CoreRT/Native/Runtime are redirected to the GCHeapUtilities class, which maintains the singular IGCHeap instance and provides a number of static methods used by the runtime to access and query the state of the heap. For most cases in Native/Runtime, this meant renaming uses of GCHeap to GCHeapUtilities. There is a notable exception to this in that GetMaxGeneration is turned into a virtual method call on IGCHeap on non-DAC code paths (see https://github.com/dotnet/coreclr/pull/6764#issuecomment-245462736 for details on why I chose to expose a virtual and non-virtual version of GetMaxGeneration)
3) This set of changes broke SOS GC heap dumping on CoreCLR when FEATURE_SVR_GC is defined, so this change contains a fix for that particular issue. As far as I know, FEATURE_SVR_GC is not defined on the ProjectN build, so this has no effect for now, but it will fix an issue that would be present if/when we enable Server GC. This particular fix is located in request.cpp.
There should be no surprises in this changelist except for portable.cpp, which declares a complete alloc_context struct out-of-line from the GC interface - this change updates the declared alloc_context struct to match the one exposed by the interface.
[tfs-changeset: 1627236]
|
|
[tfs-changeset: 1622102]
|
|
- Expand Debug.DebugBreak as IL intrinsic
- Implement RhDebugBreak JIT helper for IL break instruction
- Switch assembly code to use RhDebugBreak to avoid C++ name mangling goo
- Change PalDebugBreak to be inline to avoid extra frames under debugger
- Cleanup some left-over cruft in Unix PAL
|
|
Now that nutc generated code does not use them anymore, it is possible to cleanup RhExceptionHandling_ functions to not depend on GetReturnAddress intrinsic.
The binder generate wrappers around checked math helpers were calling RhExceptionHandling_ methods as well. I have changed these checked math helpers to be implemented in C# instead of using the wrappers. It makes them faster (e.g. checked multiplication is 20% faster on x86 now).
[tfs-changeset: 1613638]
|
|
[tfs-changeset: 1613407]
|
|
[tfs-changeset: 1612928]
|
|
- Use extern "C" names for methods called from asm helper to avoid dealing with name mangling for ports
- Move hooking up of the PInvoke tunnel to C++
- Change arm assembly code to follow same naming pattern as x86 and arm64
[tfs-changeset: 1609408]
|
|
[tfs-changeset: 1608916]
|
|
- Reduce amount of assembly code in PInvoke helpers by moving all slow paths to C++
- Share code between the slow paths of assembly and portable helpers
- Improve performance of the portable helpers by making the access to current thread and the trap thread statics inlineable (e.g. the portable RhpReversePInvoke2 helper has only two extra instructions compared to what the hand-optimized assembly helper would have)
[tfs-changeset: 1605153]
|
|
This eliminates need for BinderIntrinsics.GetReturnAddress() and CORERT ifdef
[tfs-changeset: 1603077]
|
|
This avoids hardcoded offsets that are incompatible with static linking and otherwise fragile.
[tfs-changeset: 1600424]
|
|
Remove shutdown finalization. It makes CoreRT/PN consistent with .NETCore / CoreCLR. The long story is in https://github.com/dotnet/corefx/issues/5205. The shutdown finalization was not used by UWP apps because they are never shutdown gracefully.
[tfs-changeset: 1597160]
|
|
- Replace asm wrappers for GC helpers with regular PInvoke for better portability. There is an extra GC mode switch now but it should not matter because of these helpers are either expensive or rarely used.
- Remove asm code for RhpBulkWriteBarrier by switching to C++ implementation unconditionally (remove CORERT ifdef)
- Move error handling for RhHandleAlloc* to System.Private.CoreLib (remove CORERT ifdef)
- Add missing error handling for RhReRegisterForFinalize
- A few other minor fixes and cleanup
[tfs-changeset: 1596653]
|
|
Recently while experimenting with interop performance micro-benchmarks, I observed that Reverse PInvoke code path in ProjectN is 8-10x slower
than Desktop CLR. Morgan also noticed the slow down and extra memory allocation on this code path while doing the Unity app investigation.
This change aims to improve this code path to achieve parity with Desktop CLR.
This change improve Reverse PInvoke performance by:
1. Remove the dictionary in McgModuleManager which maps between thunk address and delegate. Instead store a weak GCHandle of the delegate in the thunk data section.
2. Optimize open static delegate by storing the static function pointer directly in the thunk data. And later on reverse pinvoke code path directly do a CallI on the function pointer. Modified MCG to generate special function to handle open static delegates.
3. While storing the function pointer, store jump stub code target address. Added a runtime helper to get the jump stub target.
4. Reorder some instructions in RhpReversePInvoke and InteropNative_CommonStub functions so that the hot path get better instruction cache use.
Results:
X86
33% slower than Desktop CLR (75 vs 92 instruction)
5.5x faster than latest ProjectN
AMD64
9% faster than Desktop CLR (54 vs 67 instructions)
7.5x faster than latest ProjectN
[tfs-changeset: 1596255]
|
|
The inline TLS access in Windows assembly code depends on hardcoded TLS offset. The hardcoded offset does not work well for static linking used by CoreRT because of it can be anything depending on what other thread local variables got linked in. I have changed the TLS access in ExceptionHandling.asm where it is causing problem right now to call the RhpGetThread instead.
|
|
|
|
The loop hijack worker routine is not honoring the contract that it should be. Namely, the runtime is not allowed to trash any registers in our worker (except r12 on ARM). The two big oversights were scratch FP registers and the flags registers.
I have also added a per-module map from loop index to target address (thus requiring all the shash.h includes). This primarily helps gcstress throughput because the loop indirection cell address calculation ends up being surprisingly lengthy. I considered the other obvious approach of "back-patching" the loop indirection cell in the gcstress case (normal loop hijacking does this, but under gcstress, we do not). However, I ended up preferring this because it could help GC suspension latency in normal operation.
[tfs-changeset: 1573401]
|
|
[tfs-changeset: 1572900]
|
|
We have been keeping obsoleted code in the runtime for compatibility with older versions. The runtime version just got bumped up - it allows us to delete the obsolete code before the next compatibility band starts.
[tfs-changeset: 1572810]
|
|
|
|
These changes are targeted at completing/hardening the runtime's
support for the new UniversalTransition and CallDescr thunks and also
clarifying the invariants that hold throughout stack walker operation.
[tfs-changeset: 1568546]
|
|
The old format is binary compatible with the new one, which means programs compiled with the old tool chain should work with the new runtime. Since we had no free bits left in the first 4 bytes of the header (at least on x64), I repurposed the dynamicAlign bit to indicate there is extra data about the frame. At present that extra data consist of dynamic stack alignment and GS cookie offset (both of which are optional). I reserved three bits to allow future extensions of this extra data section. The gsCookieOffset field is inserted after the epilogCount field to improve memory layout.
CR: petersol, jkotas
[tfs-changeset: 1566536]
|
|
[tfs-changeset: 1562851]
|
