Enable devirtualization in more scenarios on crossgen2 (#53567)

Address deficiencies in current devirtualization infrastructure

- Remove the responsibility of creating a CORINFO_RESOLVED_TOKEN structure from the JIT and make it a responsibility of the VM side of the jit interface.
  - This enables the component (crossgen2) which has deeper understanding of the requirements here to correctly handle scenarios that would otherwise require expressing crossgen2 specific details across the jit interface.
- Add a new set of fixups (`READYTORUN_FIXUP_Check_VirtualFunctionOverride` and `READYTORUN_FIXUP_Verify_VirtualFunctionOverride`) these are used to validate that the behavior of the runtime and crossgen2 compiler is equivalent for a virtual resolution event
  - `READYTORUN_FIXUP_Check_VirtualFunctionOverride` will ensure that the virtual resolution decision is the same at crossgen2 time and runtime, and if the decision differs, any generated code affected by the decision will not be used.
  - `READYTORUN_FIXUP_Verify_VirtualFunctionOverride` will perform the same checks as `READYTORUN_FIXUP_Check_VirtualFunctionOverride`, but if it fails the check, the process will be terminated with a fail-fast. It is intended for use under the `--verify-type-and-field-layout` stress mode.
  - Currently only the `READYTORUN_FIXUP_Verify_VirtualFunctionOverride` is actually generated, and it is only generated when using the `--verify-type-and-field-layout` switch to crossgen2. Future work will identify if there are scenarios where we need to generate the `READYTORUN_FIXUP_Check_VirtualFunctionOverride` flag. One area of possible concern is around covariant returns, another is around handling of type equivalence.
- In order to express the fixup signature for the VirtualFunctionOverride fixups, a new flag has been added to `ReadyToRunMethodSigFlags`. `READYTORUN_METHOD_SIG_UpdateContext` will allow the method signature to internally specify the assembly which is associated with the method token, instead of relying on the ambient context.
- R2RDump and the ReadyToRun format documentation have been updated with the details of the new fixups/flags.
- Update the rules for handling unboxing stubs
  - See #51918 for details. This adds a new test, as well as proper handling for unboxing stubs to match the JIT behavior
  - Also revert #52605, which avoided the problem by simply disabling devirtualization in the presence of structs
- Adjust the rules for when it is legal to devirtualize and maintain version resiliency
  - The VersionsWithCode and VersionsWithType rules are unnecessarily restrictive.
  - Instead Validate that the metadata is safely checkable, and rely on the canInline logic to ensure that no IL that can't be handled is inlined.
  - This also involved adding a check that the chain of types from the implementation type to the declaration method table type is within the version bubble.
  - And changing the `VersionsWithType` check on the implementation type, to a new `VersionsWithTypeReference` check which can be used to validate that the type can be referred to, in combination with using `VersionsWithType` on the type definition.
  - By adjusting the way that the declMethod is referred to, it becomes possible to use the declMethod without checking the full method is `VersionsWithCode`, and it just needs a relationship to version matching code.
- In `resolveVirtualMethod` generate the `CORINFO_RESOLVED_TOKEN` structures for the jit
  - In particular we are now able to resolve to methods where the decl method is the resolution result but is not within the version bubble itself. This can happen if we can prove that the decl method is the only method which can possibly implement a virtual.
- Add support for devirtualization reasons to crossgen2
- Port all devirtualization abort conditions to crossgen2 from runtime that were not already present
- Fix devirtualization from a canonical virtual method when the actual implementation is more exact
- Fix variant interface override scenario where there is an interface that requires implementation of the variant interface as well as the variant interface itself.

3 files changed, 173 insertions, 7 deletions

diff --git a/src/coreclr/vm/jitinterface.cpp b/src/coreclr/vm/jitinterface.cpp
index 4be62db4c67..9250f436b4b 100644
--- a/src/coreclr/vm/jitinterface.cpp
+++ b/src/coreclr/vm/jitinterface.cpp
@@ -8904,6 +8904,8 @@ bool CEEInfo::resolveVirtualMethodHelper(CORINFO_DEVIRTUALIZATION_INFO * info)
     info->devirtualizedMethod = NULL;
     info->requiresInstMethodTableArg = false;
     info->exactContext = NULL;
+    memset(&info->resolvedTokenDevirtualizedMethod, 0, sizeof(info->resolvedTokenDevirtualizedMethod));
+    memset(&info->resolvedTokenDevirtualizedUnboxedMethod, 0, sizeof(info->resolvedTokenDevirtualizedUnboxedMethod));
 
     MethodDesc* pBaseMD = GetMethod(info->virtualMethod);
     MethodTable* pBaseMT = pBaseMD->GetMethodTable();
@@ -14257,6 +14259,130 @@ BOOL LoadDynamicInfoEntry(Module *currentModule,
         }
         break;
 
+    case ENCODE_VERIFY_VIRTUAL_FUNCTION_OVERRIDE:
+    case ENCODE_CHECK_VIRTUAL_FUNCTION_OVERRIDE:
+        {
+            PCCOR_SIGNATURE updatedSignature = pBlob;
+
+            ReadyToRunVirtualFunctionOverrideFlags flags = (ReadyToRunVirtualFunctionOverrideFlags)CorSigUncompressData(updatedSignature);
+
+            SigTypeContext typeContext;    // empty context is OK: encoding should not contain type variables.
+            ZapSig::Context zapSigContext(pInfoModule, (void *)currentModule, ZapSig::NormalTokens);
+            MethodDesc *pDeclMethod = ZapSig::DecodeMethod(pInfoModule, updatedSignature, &typeContext, &zapSigContext, NULL, NULL, NULL, &updatedSignature, TRUE);
+            TypeHandle thImpl = ZapSig::DecodeType(currentModule, pInfoModule, updatedSignature, CLASS_LOADED, &updatedSignature);
+
+            MethodDesc *pImplMethodCompiler = NULL;
+            
+            if ((flags & READYTORUN_VIRTUAL_OVERRIDE_VirtualFunctionOverriden) != 0)
+            {
+                pImplMethodCompiler = ZapSig::DecodeMethod(currentModule, pInfoModule, updatedSignature);
+            }
+
+            MethodDesc *pImplMethodRuntime;
+            if (pDeclMethod->IsInterface())
+            {
+                if (!thImpl.CanCastTo(pDeclMethod->GetMethodTable()))
+                {
+                    MethodTable *pInterfaceTypeCanonical = pDeclMethod->GetMethodTable()->GetCanonicalMethodTable();
+
+                    // Its possible for the decl method to need to be found on the only canonically compatible interface on the owning type
+                    MethodTable::InterfaceMapIterator it = thImpl.GetMethodTable()->IterateInterfaceMap();
+                    while (it.Next())
+                    {
+                        if (pInterfaceTypeCanonical == it.GetInterface()->GetCanonicalMethodTable())
+                        {
+                            pDeclMethod = MethodDesc::FindOrCreateAssociatedMethodDesc(pDeclMethod, it.GetInterface(), FALSE, pDeclMethod->GetMethodInstantiation(), FALSE, TRUE);
+                            break;
+                        }
+                    }
+                }
+                DispatchSlot slot = thImpl.GetMethodTable()->FindDispatchSlotForInterfaceMD(pDeclMethod, /*throwOnConflict*/ FALSE);
+                pImplMethodRuntime = slot.GetMethodDesc();
+            }
+            else
+            {
+                MethodTable *pCheckMT = thImpl.GetMethodTable();
+                MethodTable *pBaseMT = pDeclMethod->GetMethodTable();
+                WORD slot = pDeclMethod->GetSlot();
+
+                while (pCheckMT != nullptr)
+                {
+                    if (pCheckMT->HasSameTypeDefAs(pBaseMT))
+                    {
+                        break;
+                    }
+
+                    pCheckMT = pCheckMT->GetParentMethodTable();
+                }
+
+                if (pCheckMT == nullptr)
+                {
+                    pImplMethodRuntime = NULL;
+                }
+                else if (IsMdFinal(pDeclMethod->GetAttrs()))
+                {
+                    pImplMethodRuntime = pDeclMethod;
+                }
+                else
+                {
+                    _ASSERTE(slot < pBaseMT->GetNumVirtuals());
+                    pImplMethodRuntime = thImpl.GetMethodTable()->GetMethodDescForSlot(slot);
+                }
+            }
+
+            if (pImplMethodRuntime != pImplMethodCompiler)
+            {
+                if (kind == ENCODE_CHECK_VIRTUAL_FUNCTION_OVERRIDE)
+                {
+                    return FALSE;
+                }
+                else
+                {
+                    // Verification failures are failfast events
+                    DefineFullyQualifiedNameForClassW();
+                    SString methodNameDecl;
+                    SString methodNameImplRuntime(W("(NULL)"));
+                    SString methodNameImplCompiler(W("(NULL)"));
+
+                    pDeclMethod->GetFullMethodInfo(methodNameDecl);
+
+                    if (pImplMethodRuntime != NULL)
+                    {
+                        methodNameImplRuntime.Clear();
+                        pImplMethodRuntime->GetFullMethodInfo(methodNameImplRuntime);
+                    }
+
+                    if (pImplMethodCompiler != NULL)
+                    {
+                        methodNameImplCompiler.Clear();
+                        pImplMethodCompiler->GetFullMethodInfo(methodNameImplCompiler);
+                    }
+
+                    SString fatalErrorString;
+                    fatalErrorString.Printf(W("Verify_VirtualFunctionOverride Decl Method '%s' on type '%s' is '%s'(actual) instead of expected '%s'(from compiler)"),
+                        methodNameDecl.GetUnicode(),
+                        GetFullyQualifiedNameForClassW(thImpl.GetMethodTable()),
+                        methodNameImplRuntime.GetUnicode(),
+                        methodNameImplCompiler.GetUnicode());
+
+#ifdef _DEBUG
+                    {
+                        StackScratchBuffer buf;
+                        _ASSERTE_MSG(false, fatalErrorString.GetUTF8(buf));
+                    }
+#endif
+                    _ASSERTE(!IsDebuggerPresent() && "Stop on assert here instead of fatal error for ease of live debugging");
+
+                    EEPOLICY_HANDLE_FATAL_ERROR_WITH_MESSAGE(-1, fatalErrorString.GetUnicode());
+                    return FALSE;
+
+                }
+            }
+
+            result = 1;
+        }
+        break;
+
 
     case ENCODE_CHECK_INSTRUCTION_SET_SUPPORT:
         {
diff --git a/src/coreclr/vm/zapsig.cpp b/src/coreclr/vm/zapsig.cpp
index 87fad284f02..8ef797ea572 100644
--- a/src/coreclr/vm/zapsig.cpp
+++ b/src/coreclr/vm/zapsig.cpp
@@ -741,7 +741,8 @@ Module *ZapSig::DecodeModuleFromIndexIfLoaded(Module *fromModule,
 TypeHandle ZapSig::DecodeType(Module *pEncodeModuleContext,
                               Module *pInfoModule,
                               PCCOR_SIGNATURE pBuffer,
-                              ClassLoadLevel level)
+                              ClassLoadLevel level,
+                              PCCOR_SIGNATURE *ppAfterSig /*=NULL*/)
 {
     CONTRACTL
     {
@@ -766,6 +767,12 @@ TypeHandle ZapSig::DecodeType(Module *pEncodeModuleContext,
                                             NULL,
                                             pZapSigContext);
 
+    if (ppAfterSig != NULL)
+    {
+        IfFailThrow(p.SkipExactlyOne());
+        *ppAfterSig = p.GetPtr();
+    }
+
     return th;
 }
 
@@ -778,7 +785,7 @@ MethodDesc *ZapSig::DecodeMethod(Module *pReferencingModule,
 
     SigTypeContext typeContext;    // empty context is OK: encoding should not contain type variables.
     ZapSig::Context zapSigContext(pInfoModule, (void *)pReferencingModule, ZapSig::NormalTokens);
-    return DecodeMethod(pInfoModule, pBuffer, &typeContext, &zapSigContext, ppTH);
+    return DecodeMethod(pInfoModule, pBuffer, &typeContext, &zapSigContext, ppTH, NULL, NULL);
 }
 
 MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
@@ -787,7 +794,9 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
                                  ZapSig::Context *pZapSigContext,
                                  TypeHandle *ppTH, /*=NULL*/
                                  PCCOR_SIGNATURE *ppOwnerTypeSpecWithVars, /*=NULL*/
-                                 PCCOR_SIGNATURE *ppMethodSpecWithVars /*=NULL*/)
+                                 PCCOR_SIGNATURE *ppMethodSpecWithVars, /*=NULL*/
+                                 PCCOR_SIGNATURE *ppAfterSig /*=NULL*/,
+                                 BOOL actualOwnerRequired /*=FALSE*/)
 {
     STANDARD_VM_CONTRACT;
 
@@ -801,6 +810,22 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
 
     TypeHandle thOwner = NULL;
 
+    if ( methodFlags & ENCODE_METHOD_SIG_UpdateContext)
+    {
+        uint32_t updatedModuleIndex;
+        IfFailThrow(sig.GetData(&updatedModuleIndex));
+#ifdef FEATURE_MULTICOREJIT
+        if (pZapSigContext->externalTokens == ZapSig::MulticoreJitTokens)
+        {
+            pInfoModule = MulticoreJitManager::DecodeModuleFromIndex(pZapSigContext->pModuleContext, updatedModuleIndex);
+        }
+        else
+#endif
+        {
+            pInfoModule = pZapSigContext->GetZapSigModule()->GetModuleFromIndex(updatedModuleIndex);
+        }
+    }
+
     if ( methodFlags & ENCODE_METHOD_SIG_OwnerType )
     {
         if (ppOwnerTypeSpecWithVars != NULL)
@@ -843,7 +868,7 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
                 MethodDesc * pMD = NULL;
                 FieldDesc * pFD = NULL;
 
-                MemberLoader::GetDescFromMemberRef(pInfoModule, TokenFromRid(rid, mdtMemberRef), &pMD, &pFD, NULL, FALSE, &th);
+                MemberLoader::GetDescFromMemberRef(pInfoModule, TokenFromRid(rid, mdtMemberRef), &pMD, &pFD, NULL, actualOwnerRequired, &th);
                 _ASSERTE(pMD != NULL);
 
                 thOwner = th;
@@ -863,7 +888,12 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
     if (thOwner.IsNull())
     {
         if (pZapSigContext->externalTokens == ZapSig::MulticoreJitTokens)
+        {
+            if (ppAfterSig != NULL)
+                *ppAfterSig = sig.GetPtr();
+
             return NULL;
+        }
 
         thOwner = pMethod->GetMethodTable();
     }
@@ -926,7 +956,8 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
     pMethod = MethodDesc::FindOrCreateAssociatedMethodDesc(pMethod, thOwner.GetMethodTable(),
                                                             isUnboxingStub,
                                                             inst,
-                                                            !(isInstantiatingStub || isUnboxingStub));
+                                                            !(isInstantiatingStub || isUnboxingStub) && !actualOwnerRequired,
+                                                            actualOwnerRequired);
 
     g_IBCLogger.LogMethodDescAccess(pMethod);
 
@@ -940,6 +971,9 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
                                                 NULL,
                                                 pZapSigContext);
 
+        if (ppAfterSig != NULL)
+            IfFailThrow(sig.SkipExactlyOne());
+
         MethodDesc * directMethod = constrainedType.GetMethodTable()->TryResolveConstraintMethodApprox(thOwner.GetMethodTable(), pMethod);
         if (directMethod == NULL)
         {
@@ -960,6 +994,9 @@ MethodDesc *ZapSig::DecodeMethod(Module *pInfoModule,
         }
     }
 
+    if (ppAfterSig != NULL)
+        *ppAfterSig = sig.GetPtr();
+
     return pMethod;
 }
 
diff --git a/src/coreclr/vm/zapsig.h b/src/coreclr/vm/zapsig.h
index f8c5d873817..bff06fd1585 100644
--- a/src/coreclr/vm/zapsig.h
+++ b/src/coreclr/vm/zapsig.h
@@ -163,7 +163,8 @@ public:
         Module              *referencingModule,
         Module              *fromModule,
         PCCOR_SIGNATURE     pBuffer,
-        ClassLoadLevel      level = CLASS_LOADED);
+        ClassLoadLevel      level = CLASS_LOADED,
+        PCCOR_SIGNATURE     *ppAfterSig = NULL);
 
     static MethodDesc *DecodeMethod(
         Module              *referencingModule,
@@ -178,7 +179,9 @@ public:
         ZapSig::Context     *pZapSigContext,
         TypeHandle          *ppTH = NULL,
         PCCOR_SIGNATURE     *ppOwnerTypeSpecWithVars = NULL,
-        PCCOR_SIGNATURE     *ppMethodSpecWithVars = NULL);
+        PCCOR_SIGNATURE     *ppMethodSpecWithVars = NULL,
+        PCCOR_SIGNATURE     *ppAfterSig = NULL,
+        BOOL                actualOwnerRequired = FALSE);
 
     static FieldDesc *DecodeField(
         Module              *referencingModule,