Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/dotnet/runtime.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorJeremy Koritzinsky <jekoritz@microsoft.com>2022-03-23 19:06:32 +0300
committerGitHub <noreply@github.com>2022-03-23 19:06:32 +0300
commita02b49a0486e92ace50803b977adf8c533ab118f (patch)
tree738b0eff3dbbb212c4b26b6b0a82bf7c333bb981 /docs
parent3c126e6e42b0f9c879cae93d94a1deb7e7680f66 (diff)
Add the CustomTypeMarshallerAttribute type to make it easier to identify marshaller types (#65591)
Diffstat (limited to 'docs')
-rw-r--r--docs/design/libraries/LibraryImportGenerator/SpanMarshallers.md187
-rw-r--r--docs/design/libraries/LibraryImportGenerator/StructMarshalling.md170
-rw-r--r--docs/project/list-of-diagnostics.md27
3 files changed, 244 insertions, 140 deletions
diff --git a/docs/design/libraries/LibraryImportGenerator/SpanMarshallers.md b/docs/design/libraries/LibraryImportGenerator/SpanMarshallers.md
index eccefc62e87..b787c172e6c 100644
--- a/docs/design/libraries/LibraryImportGenerator/SpanMarshallers.md
+++ b/docs/design/libraries/LibraryImportGenerator/SpanMarshallers.md
@@ -24,11 +24,11 @@ We have decided to match the managed semantics of `(ReadOnly)Span<T>` to provide
As part of this design, we would also want to include some in-box marshallers that follow the design laid out in the [Struct Marshalling design doc](./StructMarshalling.md) to support some additional scenarios:
-- A marshaler that marshals an empty span as a non-null pointer.
+- A marshaller that marshals an empty span as a non-null pointer.
- This marshaller would only support empty spans as it cannot correctly represent non-empty spans of non-blittable types.
-- A marshaler that marshals out a pointer to the native memory as a Span instead of copying the data into a managed array.
+- A marshaller that marshals out a pointer to the native memory as a Span instead of copying the data into a managed array.
- This marshaller would only support blittable spans by design.
- - This marshaler will require the user to manually release the memory. Since this will be an opt-in marshaler, this scenario is already advanced and that additional requirement should be understandable to users who use this marshaler.
+ - This marshaller will require the user to manually release the memory. Since this will be an opt-in marshaller, this scenario is already advanced and that additional requirement should be understandable to users who use this marshaller.
- Since there is no mechansim to provide a collection length, the question of how to provide the span's length in this case is still unresolved. One option would be to always provide a length 1 span and require the user to create a new span with the correct size, but that feels like a bad design.
### Pros/Cons of Design 1
@@ -40,7 +40,7 @@ Pros:
Cons:
-- Defining custom marshalers for non-empty spans of non-blittable types generically is impossible since the marshalling rules of the element's type cannot be known.
+- Defining custom marshallers for non-empty spans of non-blittable types generically is impossible since the marshalling rules of the element's type cannot be known.
- Custom non-default marshalling of the span element types is impossible for non-built-in types.
- Inlining the span marshalling fully into the stub increases on-disk IL size.
- This design does not enable developers to easily define custom marshalling support for their own collection types, which may be desireable.
@@ -55,43 +55,86 @@ Span marshalling would still be implemented with similar semantics as mentioned
### Proposed extension to the custom type marshalling design
-Introduce a new attribute named `GenericContiguousCollectionMarshallerAttribute`. This attribute would have the following shape:
+Introduce a marshaller kind named `LinearCollection`.
-```csharp
+```diff
namespace System.Runtime.InteropServices
-{
- [AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class)]
- public sealed class GenericContiguousCollectionMarshallerAttribute : Attribute
- {
- public GenericContiguousCollectionMarshallerAttribute();
- }
+{
+ [AttributeUsage(AttributeTargets.Struct)]
+ public sealed class CustomTypeMarshallerAttribute : Attribute
+ {
++ /// <summary>
++ /// This type is used as a placeholder for the first generic parameter when generic parameters cannot be used
++ /// to identify the managed type (i.e. when the marshaller type is generic over T and the managed type is T[])
++ /// </summary>
++ public struct GenericPlaceholder
++ {
++ }
+ }
+
+ public enum CustomTypeMarshallerKind
+ {
+ Value,
++ LinearCollection
+ }
}
```
The attribute would be used with a collection type like `Span<T>` as follows:
```csharp
-[NativeTypeMarshalling(typeof(DefaultSpanMarshaler<>))]
+[NativeTypeMarshalling(typeof(DefaultSpanMarshaller<>))]
public ref struct Span<T>
{
...
}
-[GenericContiguousCollectionMarshaller]
-public ref struct DefaultSpanMarshaler<T>
+[CustomTypeMarshaller(typeof(Span<>), CustomTypeMarshallerKind.LinearCollection)]
+public ref struct DefaultSpanMarshaller<T>
{
...
}
```
-The `GenericContiguousCollectionMarshallerAttribute` attribute is applied to a generic marshaler type with the "collection marshaller" shape described below. Since generic parameters cannot be used in attributes, open generic types will be permitted in the `NativeTypeMarshallingAttribute` constructor as long as they have the same arity as the type the attribute is applied to and generic parameters provided to the applied-to type can also be used to construct the type passed as a parameter.
+The `CustomTypeMarshallerKind.LinearCollection` kind is applied to a generic marshaller type with the "LinearCollection marshaller shape" described below.
+
+#### Supporting generics
+
+Since generic parameters cannot be used in attributes, open generic types will be permitted in the `NativeTypeMarshallingAttribute` and the `CustomTypeMarshallerAttribute` as long as they have the same arity as the type with the attribute and generic parameters provided to the type with the attribute can also be used to construct the type passed as a parameter.
+
+If a `CustomTypeMarshaller`-attributed type is a marshaller for a type for a pointer, an array, or a combination of pointers and arrays, the `CustomTypeMarshallerAttribute.GenericPlaceholder` type can be used in the place of the first generic parameter of the marshaller type.
+
+For example:
+
+```csharp
+[CustomTypeMarshaller(typeof(CustomTypeMarshallerAttribute.GenericPlaceholder), Direction = CustomTypeMarshallerDirection.In)]
+struct Marshaller<T>
+{
+ public Marshaller(T managed);
+}
+[CustomTypeMarshaller(typeof(CustomTypeMarshallerAttribute.GenericPlaceholder[]), Direction = CustomTypeMarshallerDirection.In)]
+struct Marshaller<T>
+{
+ public Marshaller(T[] managed);
+}
+[CustomTypeMarshaller(typeof(CustomTypeMarshallerAttribute.GenericPlaceholder*), Direction = CustomTypeMarshallerDirection.In)]
+struct Marshaller<T> where T : unmanaged
+{
+ public Marshaller(T* managed);
+}
+[CustomTypeMarshaller(typeof(CustomTypeMarshallerAttribute.GenericPlaceholder*[]), Direction = CustomTypeMarshallerDirection.In)]
+struct Marshaller<T> where T : unmanaged
+{
+ public Marshaller(T*[] managed);
+}
+```
-#### Generic collection marshaller shape
+#### LinearCollection marshaller shape
-A generic collection marshaller would be required to have the following shape, in addition to the requirements for marshaler types used with the `NativeTypeMarshallingAttribute`, excluding the constructors.
+A generic collection marshaller would be required to have the following shape, in addition to the requirements for marshaller types used with the `CustomTypeMarshallerKind.Value` shape, excluding the constructors.
```csharp
-[GenericContiguousCollectionMarshaller]
+[CustomTypeMarshaller(typeof(GenericCollection<, , ,...>), CustomTypeMarshallerKind.LinearCollection)]
public struct GenericContiguousCollectionMarshallerImpl<T, U, V,...>
{
// this constructor is required if marshalling from native to managed is supported.
@@ -100,29 +143,33 @@ public struct GenericContiguousCollectionMarshallerImpl<T, U, V,...>
public GenericContiguousCollectionMarshallerImpl(GenericCollection<T, U, V, ...> collection, int nativeSizeOfElement);
public GenericContiguousCollectionMarshallerImpl(GenericCollection<T, U, V, ...> collection, Span<byte> stackSpace, int nativeSizeOfElement); // optional
- public const int StackBufferSize = /* */; // required if the span-based constructor is supplied.
-
/// <summary>
- /// A span that points to the memory where the managed values of the collection are stored (in the marshalling case) or should be stored (in the unmarshalling case).
+ /// A span that points to the memory where the managed values of the collection are stored.
/// </summary>
- public Span<TCollectionElement> ManagedValues { get; }
-
+ public ReadOnlySpan<TCollectionElement> GetManagedValuesSource();
/// <summary>
- /// Set the expected length of the managed collection based on the parameter/return value/field marshalling information.
- /// Required only when unmarshalling is supported.
+ /// A span that points to the memory where the unmarshalled managed values of the collection should be stored.
/// </summary>
- public void SetUnmarshalledCollectionLength(int length);
-
- public IntPtr Value { get; set; }
+ public Span<TCollectionElement> GetManagedValuesDestination(int length);
+ /// <summary>
+ /// A span that points to the memory where the native values of the collection are stored after the native call.
+ /// </summary>
+ public ReadOnlySpan<TCollectionElement> GetNativeValuesSource(int length);
+ /// <summary>
+ /// A span that points to the memory where the native values of the collection should be stored.
+ /// </summary>
+ public Span<TCollectionElement> GetNativeValuesDestination();
/// <summary>
/// A span that points to the memory where the native values of the collection should be stored.
/// </summary>
public unsafe Span<byte> NativeValueStorage { get; }
- // The requirements on the Value property are the same as when used with `NativeTypeMarshallingAttribute`.
+ // The requirements on the TNative type are the same as when used with `NativeTypeMarshallingAttribute`.
// The property is required with the generic collection marshalling.
- public TNative Value { get; set; }
+ public TNative ToNativeValue();
+
+ public void FromNativeValue(TNative value);
}
```
@@ -162,7 +209,7 @@ To support supplying information about collection element counts, a parameterles
The `ElementIndirectionLevel` property is added to support supplying marshalling info for element types in a collection. For example, if the user is passing a `List<List<Foo>>` from managed to native code, they could provide the following attributes to specify marshalling rules for the outer and inner lists and `Foo` separately:
```csharp
-private static partial void Bar([MarshalUsing(typeof(ListAsArrayMarshaller<List<Foo>>), CountElementName = nameof(count)), MarshalUsing(ConstantElementCount = 10, ElementIndirectionLevel = 1), MarshalUsing(typeof(FooMarshaler), ElementIndirectionLevel = 2)] List<List<Foo>> foos, int count);
+private static partial void Bar([MarshalUsing(typeof(ListAsArrayMarshaller<List<Foo>>), CountElementName = nameof(count)), MarshalUsing(ConstantElementCount = 10, ElementIndirectionLevel = 1), MarshalUsing(typeof(FooMarshaller), ElementIndirectionLevel = 2)] List<List<Foo>> foos, int count);
```
Multiple `MarshalUsing` attributes can only be supplied on the same parameter or return value if the `ElementIndirectionLevel` property is set to distinct values. One `MarshalUsing` attribute per parameter or return value can leave the `ElementIndirectionLevel` property unset. This attribute controls the marshalling of the collection object passed in as the parameter. The sequence of managed types for `ElementIndirectionLevel` is based on the elements of the `ManagedValues` span on the collection marshaller of the previous indirection level. For example, for the marshalling info for `ElementIndirectionLevel = 1` above, the managed type is the type of the following C# expression: `ListAsArrayMarshaller<List<Foo>>.ManagedValues[0]`.
@@ -174,14 +221,16 @@ Alternatively, the `MarshalUsingAttribute` could provide a `Type ElementNativeTy
This design could be used to provide a default marshaller for spans and arrays. Below is an example simple marshaller for `Span<T>`. This design does not include all possible optimizations, such as stack allocation, for simpilicity of the example.
```csharp
-[GenericContiguousCollectionMarshaller]
-public ref struct SpanMarshaler<T>
+[CustomTypeMarshaller(typeof(Span<>), CustomTypeMarshallerKind.LinearCollection, Features = CustomTypeMarshallerFeatures.UnmanagedResources | CustomTypeMarshallerFeatures.TwoStageMarshalling)]
+public ref struct SpanMarshaller<T>
{
private Span<T> managedCollection;
private int nativeElementSize;
+
+ private IntPtr Value { get; set; }
- public SpanMarshaler(Span<T> collection, int nativeSizeOfElement)
+ public SpanMarshaller(Span<T> collection, int nativeSizeOfElement)
{
managedCollection = collection;
Value = Marshal.AllocCoTaskMem(collection.Length * nativeSizeOfElement);
@@ -189,19 +238,20 @@ public ref struct SpanMarshaler<T>
nativeElementSize = nativeSizeOfElement;
}
- public Span<T> ManagedValues => managedCollection;
-
- public void SetUnmarshalledCollectionLength(int length)
- {
- managedCollection = new T[value];
- }
+ public ReadOnlySpan<T> GetManagedValuesSource() => managedCollection;
+
+ public Span<T> GetManagedValuesDestination(int length) => managedCollection = new T[length];
- public IntPtr Value { get; set; }
+ public unsafe Span<byte> GetNativeValuesDestination() => MemoryMarshal.CreateSpan(ref *(byte*)(Value), managedCollection.Length);
- public unsafe Span<byte> NativeValueStorage => MemoryMarshal.CreateSpan(ref *(byte*)(Value), Length);
+ public unsafe Span<byte> GetNativeValuesSource(int length) => MemoryMarshal.CreateSpan(ref *(byte*)(Value), length);
public Span<T> ToManaged() => managedCollection;
+ public IntPtr ToNativeValue() => Value;
+
+ public void FromNativeValue(IntPtr value) => Value = value;
+
public void FreeNative()
{
if (Value != IntPtr.Zero)
@@ -235,19 +285,20 @@ public static partial Span<int> DuplicateValues([MarshalUsing(typeof(WrappedInt)
public static partial unsafe Span<int> DuplicateValues(Span<int> values, int length)
{
SpanMarshaller<int> __values_marshaller = new SpanMarshaller<int>(values, sizeof(WrappedInt));
- for (int i = 0; i < __values_marshaller.ManagedValues.Length; ++i)
{
- WrappedInt native = new WrappedInt(__values_marshaller.ManagedValues[i]);
- MemoryMarshal.Write(__values_marshaller.NativeValueStorage.Slice(sizeof(WrappedInt) * i), ref native);
+ ReadOnlySpan<int> __values_managedSpan = __values_marshaller.GetManagedValuesSource();
+ Span<byte> __values_nativeSpan = __values_marshaller.GetNativeValuesDestination();
+ for (int i = 0; i < __values_managedSpan.Length; ++i)
+ {
+ WrappedInt native = new WrappedInt(__values_managedSpan[i]);
+ MemoryMarshal.Write(__values_nativeSpan.Slice(sizeof(WrappedInt) * i), ref native);
+ }
}
- IntPtr __retVal_native = __PInvoke__(__values_marshaller.Value, length);
- SpanMarshaller<int> __retVal_marshaller = new
- {
- Value = __retVal_native
- };
- __retVal_marshaller.SetUnmarshalledCollectionLength(length);
- MemoryMarshal.Cast<byte, int>(__retVal_marshaller.NativeValueStorage).CopyTo(__retVal_marshaller.ManagedValues);
+ IntPtr __retVal_native = __PInvoke__(__values_marshaller.ToNativeValue(), length);
+ SpanMarshaller<int> __retVal_marshaller = new();
+ __retVal_marshaller.FromNativeValue(__retVal_native);
+ MemoryMarshal.Cast<byte, int>(__retVal_marshaller.GetNativeValuesSource(length)).CopyTo(__retVal_marshaller.GetManagedValuesDestination(length));
return __retVal_marshaller.ToManaged();
[DllImport("Native", EntryPoint="DuplicateValues")]
@@ -261,30 +312,24 @@ This design could also be applied to support the built-in array marshalling if i
If a managed or native representation of a collection has a non-contiguous element layout, then developers currently will need to convert to or from array/span types at the interop boundary. This section proposes an API that would enable developers to convert directly between a managed and native non-contiguous collection layout as part of marshalling.
-A new attribute named `GenericCollectionMarshaller` attribute could be added that would specify that the collection is noncontiguous in either managed or native representations. Then additional methods should be added to the generic collection model, and some methods would be removed:
+A new marshaller kind named `GenericCollection` could be added that would specify that the collection is noncontiguous in either managed or native representations. Then additional methods should be added to the generic collection model, and some methods would be removed:
```diff
-- [GenericContiguousCollectionMarshaller]
-+ [GenericCollectionMarshaller]
+- [CustomTypeMarshaller(typeof(Span<>), CustomTypeMarshallerKind.LinearCollection)]
++ [CustomTypeMarshaller(typeof(Span<>), CustomTypeMarshallerKind.GenericCollection)]
public struct GenericContiguousCollectionMarshallerImpl<T, U, V,...>
{
// these constructors are required if marshalling from managed to native is supported.
public GenericContiguousCollectionMarshallerImpl(GenericCollection<T, U, V, ...> collection, int nativeSizeOfElements);
public GenericContiguousCollectionMarshallerImpl(GenericCollection<T, U, V, ...> collection, Span<byte> stackSpace, int nativeSizeOfElements); // optional
- public const int StackBufferSize = /* */; // required if the span-based constructor is supplied.
+ public TNative ToNativeValue();
+ public void FromNativeValue(TNative value);
-- public Span<TCollectionElement> ManagedValues { get; }
-
-- public void SetUnmarshalledCollectionLength(int length);
-
- public IntPtr Value { get; set; }
-
-- public unsafe Span<byte> NativeValueStorage { get; }
-
- // The requirements on the Value property are the same as when used with `NativeTypeMarshallingAttribute`.
- // The property is required with the generic collection marshalling.
- public TNative Value { get; set; }
+- public ReadOnlySpan<TCollectionElement> GetManagedValuesSource();
+- public Span<TCollectionElement> GetManagedValuesDestination(int length);
+- public ReadOnlySpan<TCollectionElement> GetNativeValuesSource(int length);
+- public Span<TCollectionElement> GetNativeValuesDestination();
+ public ref byte GetOffsetForNativeValueAtIndex(int index);
+ public TCollectionElement GetManagedValueAtIndex(int index);
@@ -293,9 +338,9 @@ public struct GenericContiguousCollectionMarshallerImpl<T, U, V,...>
}
```
-The `GetManagedValueAtIndex` method and `Count` getter are used in the process of marshalling from managed to native. The generated code will iterate through `Count` elements (retrieved through `GetManagedValueAtIndex`) and assign their marshalled result to the address represented by `GetOffsetForNativeValueAtIndex` called with the same index. Then either the `Value` property getter will be called or the marshaller's `GetPinnableReference` method will be called, depending on if pinning is supported in the current scenario.
+The `GetManagedValueAtIndex` method and `Count` getter are used in the process of marshalling from managed to native. The generated code will iterate through `Count` elements (retrieved through `GetManagedValueAtIndex`) and assign their marshalled result to the address represented by `GetOffsetForNativeValueAtIndex` called with the same index. Then the `ToNativeValue` method will be called to get the value to pass to native code.
-The `SetManagedValueAtIndex` method and the `Count` setter are used in the process of marshalling from native to managed. The `Count` property will be set to the number of elements that the native collection contains, and the `Value` property will be assigned the result value from native code. Then the stub will iterate through the native collection `Count` times, calling `GetOffsetForNativeValueAtIndex` to get the offset of the native value and calling `SetManagedValueAtIndex` to set the unmarshalled managed value at that index.
+The `SetManagedValueAtIndex` method and the `Count` setter are used in the process of marshalling from native to managed. The `Count` property will be set to the number of elements that the native collection contains, and the `FromNativeValue` property will be called with the result value from native code. Then the stub will iterate through the native collection `Count` times, calling `GetOffsetForNativeValueAtIndex` to get the offset of the native value and calling `SetManagedValueAtIndex` to set the unmarshalled managed value at that index.
### Pros/Cons of Design 2
@@ -312,6 +357,6 @@ Cons:
- Introduces more attribute types into the BCL.
- Introduces more complexity in the marshalling type model.
- It may be worth describing the required members (other than constructors) in interfaces just to simplify the mental load of which members are required for which scenarios.
- - A set of interfaces (one for managed-to-native members, one for native-to-managed members, and one for the sequential-specific members) could replace the `GenericContiguousCollectionMarshaller` attribute.
+ - A set of interfaces (one for managed-to-native members, one for native-to-managed members, and one for the sequential-specific members) could replace the new marshaller kind.
- The base proposal only supports contiguous collections.
- The feeling at time of writing is that we are okay asking developers to convert to/from arrays or spans at the interop boundary.
diff --git a/docs/design/libraries/LibraryImportGenerator/StructMarshalling.md b/docs/design/libraries/LibraryImportGenerator/StructMarshalling.md
index 63781d6d77d..7028227fedd 100644
--- a/docs/design/libraries/LibraryImportGenerator/StructMarshalling.md
+++ b/docs/design/libraries/LibraryImportGenerator/StructMarshalling.md
@@ -25,89 +25,119 @@ All design options would use these attributes:
```csharp
[AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class)]
-public class GeneratedMarshallingAttribute : Attribute {}
+public sealed class GeneratedMarshallingAttribute : Attribute {}
[AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class)]
-public class NativeMarshallingAttribute : Attribute
+public sealed class NativeMarshallingAttribute : Attribute
{
public NativeMarshallingAttribute(Type nativeType) {}
}
[AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue | AttributeTargets.Field)]
-public class MarshalUsingAttribute : Attribute
+public sealed class MarshalUsingAttribute : Attribute
{
public MarshalUsingAttribute(Type nativeType) {}
}
-```
-
-The `NativeMarshallingAttribute` and `MarshalUsingAttribute` attributes would require that the provided native type `TNative` is a `struct` that does not require any marshalling and has a subset of three methods with the following names and shapes (with the managed type named TManaged):
-```csharp
-partial struct TNative
+[AttributeUsage(AttributeTargets.Struct)]
+public sealed class CustomTypeMarshallerAttribute : Attribute
{
- public TNative(TManaged managed) {}
- public TManaged ToManaged() {}
+ public CustomTypeMarshallerAttribute(Type managedType, CustomTypeMarshallerKind marshallerKind = CustomTypeMarshallerKind.Value)
+ {
+ ManagedType = managedType;
+ MarshallerKind = marshallerKind;
+ }
- public void FreeNative() {}
+ public Type ManagedType { get; }
+ public CustomTypeMarshallerKind MarshallerKind { get; }
+ public int BufferSize { get; set; }
+ public CustomTypeMarshallerDirection Direction { get; set; } = CustomTypeMarshallerDirection.Ref;
+ public CustomTypeMarshallerFeatures Features { get; set; }
}
-```
-
-The analyzer will report an error if neither the constructor nor the `ToManaged` method is defined. When one of those two methods is missing, the direction of marshalling (managed to native/native to managed) that relies on the missing method is considered unsupported for the corresponding managed type. The `FreeNative` method is only required when there are resources that need to be released.
-
-
-> :question: Does this API surface and shape work for all marshalling scenarios we plan on supporting? It may have issues with the current "layout class" by-value `[Out]` parameter marshalling where the runtime updates a `class` typed object in place. We already recommend against using classes for interop for performance reasons and a struct value passed via `ref` or `out` with the same members would cover this scenario.
-
-If the native type `TNative` also has a public `Value` property, then the value of the `Value` property will be passed to native code instead of the `TNative` value itself. As a result, the type `TNative` will be allowed to require marshalling and the type of the `Value` property will be required be passable to native code without any additional marshalling. If the `Value` property is settable, then when marshalling in the native-to-managed direction, a default value of `TNative` will have its `Value` property set to the native value. If `Value` does not have a setter, then marshalling from native to managed is not supported.
-If a `Value` property is provided, the developer may also provide a ref-returning or readonly-ref-returning `GetPinnableReference` method. The `GetPinnableReference` method will be called before the `Value` property getter is called. The ref returned by `GetPinnableReference` will be pinned with a `fixed` statement, but the pinned value will not be used (it acts exclusively as a side-effect).
-
-A `ref` or `ref readonly` typed `Value` property is unsupported. If a ref-return is required, the type author can supply a `GetPinnableReference` method on the native type to pin the desired `ref` to return and then use `System.Runtime.CompilerServices.Unsafe.AsPointer` to get a pointer from the `ref` that will have already been pinned by the time the `Value` getter is called.
+public enum CustomTypeMarshallerKind
+{
+ Value
+}
-```csharp
-[NativeMarshalling(typeof(TMarshaler))]
-public struct TManaged
+[Flags]
+public enum CustomTypeMarshallerFeatures
{
- // ...
+ None = 0,
+ /// <summary>
+ /// The marshaller owns unmanaged resources that must be freed
+ /// </summary>
+ UnmanagedResources = 0x1,
+ /// <summary>
+ /// The marshaller can use a caller-allocated buffer instead of allocating in some scenarios
+ /// </summary>
+ CallerAllocatedBuffer = 0x2,
+ /// <summary>
+ /// The marshaller uses the two-stage marshalling design for its <see cref="CustomTypeMarshallerKind"/> instead of the one-stage design.
+ /// </summary>
+ TwoStageMarshalling = 0x4
}
+[Flags]
+public enum CustomTypeMarshallerDirection
+{
+ /// <summary>
+ /// No marshalling direction
+ /// </summary>
+ [EditorBrowsable(EditorBrowsableState.Never)]
+ None = 0,
+ /// <summary>
+ /// Marshalling from a managed environment to an unmanaged environment
+ /// </summary>
+ In = 0x1,
+ /// <summary>
+ /// Marshalling from an unmanaged environment to a managed environment
+ /// </summary>
+ Out = 0x2,
+ /// <summary>
+ /// Marshalling to and from managed and unmanaged environments
+ /// </summary>
+ Ref = In | Out,
+}
+```
+
+The `NativeMarshallingAttribute` and `MarshalUsingAttribute` attributes would require that the provided native type `TNative` is a `struct` that does not require any marshalling and has the `CustomTypeMarshallerAttribute` with the first parameter being a `typeof()` of the managed type (with the managed type named TManaged in this example), an optional `CustomTypeMarshallerKind`, `CustomTypeMarshallerDirection`, and optional `CustomTypeMarshallerFeatures`:
-public struct TMarshaler
+```csharp
+[CustomTypeMarshaller(typeof(TManaged), CustomTypeMarshallerKind.Value, Direction = CustomTypeMarshallerDirection.Ref, Features = CustomTypeMarshallerFeatures.None)]
+partial struct TNative
{
- public TMarshaler(TManaged managed) {}
+ public TNative(TManaged managed) {}
public TManaged ToManaged() {}
+}
+```
- public void FreeNative() {}
-
- public ref TNative GetPinnableReference() {}
+If the attribute specifies the `Direction` is either `In` or `Ref`, then the example constructor above must be provided. If the attribute specifies the `Direction` is `Out` or `Ref`, then the `ToManaged` method must be provided. If the `Direction` property is unspecified, then it will be treated as if the user provided `CustomTypeMarshallerDirection.Ref`. The analyzer will report an error if the attribute provides `CustomTypeMarshallerDirection.None`, as a marshaller that supports no direction is unusable.
- public TNative* Value { get; set; }
-}
+If the attribute provides the `CustomTypeMarshallerFeatures.UnmanagedResources` flag to the `Features` property then a `void`-returning parameterless instance method name `FreeNative` must be provided. This method can be used to release any non-managed resources used during marshalling.
-```
+> :question: Does this API surface and shape work for all marshalling scenarios we plan on supporting? It may have issues with the current "layout class" by-value `[Out]` parameter marshalling where the runtime updates a `class` typed object in place. We already recommend against using classes for interop for performance reasons and a struct value passed via `ref` or `out` with the same members would cover this scenario.
### Performance features
#### Pinning
-Since C# 7.3 added a feature to enable custom pinning logic for user types, we should also add support for custom pinning logic. If the user provides a `GetPinnableReference` method on the managed type that matches the requirements to be used in a `fixed` statement and the pointed-to type would not require any additional marshalling, then we will support using pinning to marshal the managed value when possible. The analyzer should issue a warning when the pointed-to type would not match the final native type, accounting for the `Value` property on the native type. Since `MarshalUsingAttribute` is applied at usage time instead of at type authoring time, we will not enable the pinning feature since the implementation of `GetPinnableReference` is likely designed to match the default marshalling rules provided by the type author, not the rules provided by the marshaller provided by the `MarshalUsingAttribute`.
+Since C# 7.3 added a feature to enable custom pinning logic for user types, we should also add support for custom pinning logic. If the user provides a `GetPinnableReference` method on the managed type that matches the requirements to be used in a `fixed` statement and the pointed-to type would not require any additional marshalling, then we will support using pinning to marshal the managed value when possible. The analyzer should issue a warning when the pointed-to type would not match the final native type, accounting for any optional features used by the custom marshaller type. Since `MarshalUsingAttribute` is applied at usage time instead of at type authoring time, we will not enable the pinning feature since the implementation of `GetPinnableReference` is likely designed to match the default marshalling rules provided by the type author, not the rules provided by the marshaller provided by the `MarshalUsingAttribute`.
#### Caller-allocated memory
-Custom marshalers of collection-like types or custom string encodings (such as UTF-32) may want to use stack space for extra storage for additional performance when possible. If the `TNative` type provides additional members with the following signatures, then it will opt in to using a caller-allocated buffer:
+Custom marshalers of collection-like types or custom string encodings (such as UTF-32) may want to use stack space for extra storage for additional performance when possible. If the `[CustomTypeMarshaller]` attribute sets the `Features` property to value with the `CustomTypeMarshallerFeatures.CallerAllocatedBuffer` flag, then `TNative` type must provide additional constructor with the following signature and set the `BufferSize` field on the `CustomTypeMarshallerAttribute`. It will then be opted-in to using a caller-allocated buffer:
```csharp
+[CustomTypeMarshaller(typeof(TManaged), BufferSize = /* */, Features = CustomTypeMarshallerFeatures.CallerAllocatedBuffer)]
partial struct TNative
{
public TNative(TManaged managed, Span<byte> buffer) {}
-
- public const int BufferSize = /* */;
-
- public const bool RequiresStackBuffer = /* */;
}
```
-When these members are present, the source generator will call the two-parameter constructor with a possibly stack-allocated buffer of `BufferSize` bytes when a stack-allocated buffer is usable. If a stack-allocated buffer is a requirement, the `RequiresStackBuffer` field should be set to `true` and the `buffer` will be guaranteed to be allocated on the stack. Setting the `RequiresStackBuffer` field to `false` is the same as omitting the field definition. Since a dynamically allocated buffer is not usable in all scenarios, for example Reverse P/Invoke and struct marshalling, a one-parameter constructor must also be provided for usage in those scenarios. This may also be provided by providing a two-parameter constructor with a default value for the second parameter.
+When these `CallerAllocatedBuffer` feature flag is present, the source generator will call the two-parameter constructor with a possibly stack-allocated buffer of `BufferSize` bytes when a stack-allocated buffer is usable. Since a dynamically allocated buffer is not usable in all scenarios, for example Reverse P/Invoke and struct marshalling, a one-parameter constructor must also be provided for usage in those scenarios.
-Type authors can pass down the `buffer` pointer to native code by defining a `Value` property that returns a pointer to the first element, generally through code using `MemoryMarshal.GetReference()` and `Unsafe.AsPointer`. If `RequiresStackBuffer` is not provided or set to `false`, the `buffer` span must be pinned to be used safely. The `buffer` span can be pinned by defining a `GetPinnableReference()` method on the native type that returns a reference to the first element of the span.
+Type authors can pass down the `buffer` pointer to native code by using the `TwoStageMarshalling` feature to provide a `ToNativeValue` method that returns a pointer to the first element, generally through code using `MemoryMarshal.GetReference()` and `Unsafe.AsPointer`. The `buffer` span must be pinned to be used safely. The `buffer` span can be pinned by defining a `GetPinnableReference()` method on the native type that returns a reference to the first element of the span.
### Determining if a type is doesn't need marshalling
@@ -238,7 +268,7 @@ All generated stubs will be marked with [`SkipLocalsInitAttribute`](https://docs
### Special case: Transparent Structures
-There has been discussion about Transparent Structures, structure types that are treated as their underlying types when passed to native code. The support for a `Value` property on a generated marshalling type supports the transparent struct support. For example, we could support strongly typed `HRESULT` returns with this model as shown below:
+There has been discussion about Transparent Structures, structure types that are treated as their underlying types when passed to native code. The source-generated model supports this design through the `TwoStageMarshalling` feature flag on the `CustomTypeMarshaller` attribute.
```csharp
[NativeMarshalling(typeof(HRESULT))]
@@ -251,46 +281,78 @@ struct HResult
public readonly int Result;
}
+[CustomTypeMarshaller(typeof(HResult), Features = CustomTypeMarshallerFeatures.TwoStageMarshalling)]
struct HRESULT
{
+ private HResult managed;
public HRESULT(HResult hr)
{
- Value = hr;
+ managed = hr;
}
- public HResult ToManaged() => new HResult(Value);
- public int Value { get; set; }
+ public HResult ToManaged() => managed;
+ public int ToNativeValue() => managed.Result;
}
```
+For the more detailed specification, we will use the example below:
+
+```csharp
+[NativeMarshalling(typeof(TMarshaller))]
+public struct TManaged
+{
+ // ...
+}
+
+[CustomTypeMarshaller(typeof(TManaged))]
+public struct TMarshaller
+{
+ public TMarshaller(TManaged managed) {}
+ public TManaged ToManaged() {}
+
+ public ref T GetPinnableReference() {}
+
+ public unsafe TNative* ToNativeValue();
+ public unsafe void FromNativeValue(TNative*);
+}
+
+```
+
In this case, the underlying native type would actually be an `int`, but the user could use the strongly-typed `HResult` type as the public surface area.
-> :question: Should we support transparent structures on manually annotated types that wouldn't need marshalling otherwise? If we do, we should do so in an opt-in manner to make it possible to have a `Value` property on the type without assuming that it is for interop in all cases.
+If a type `TMarshaller` with the `CustomTypeMarshaller` attribute specifies the `TwoStageMarshalling` feature, then it must provide the `ToNativeValue` feature if it supports the `In` direction, and the `FromNativeValue` method if it supports the `Out` direction. The return value of the `ToNativeValue` method will be passed to native code instead of the `TMarshaller` value itself. As a result, the type `TMarshaller` will be allowed to require marshalling and the return type of the `ToNativeValue` method, will be required be passable to native code without any additional marshalling. When marshalling in the native-to-managed direction, a default value of `TMarshaller` will have the `FromNativeValu` method called with the native value. If we are marshalling a scenario where a single frame covers the whole native call and we are marshalling in and out, then the same instance of the marshller will be reused.
+
+If the `TwoStageMarshalling` feature is specified, the developer may also provide a ref-returning or readonly-ref-returning `GetPinnableReference` method. The `GetPinnableReference` method will be called before the `ToNativeValue` method is called. The ref returned by `GetPinnableReference` will be pinned with a `fixed` statement, but the pinned value will not be used (it acts exclusively as a side-effect). As a result, `GetPinnableReference` can return a `ref` to any `T` that can be used in a fixed statement (a C# `unmanaged` type).
+
+A `ref` or `ref readonly` typed `ToNativeValue` method is unsupported. If a ref-return is required, the type author can supply a `GetPinnableReference` method on the native type to pin the desired `ref` to return and then use `System.Runtime.CompilerServices.Unsafe.AsPointer` to get a pointer from the `ref` that will have already been pinned by the time the `ToNativeValue` method is called.
+
+> :question: Should we support transparent structures on manually annotated types that wouldn't need marshalling otherwise? If we do, we should do so in an opt-in manner to make it possible to have a `ToNativeValue` method on the type without assuming that it is for interop in all cases.
#### Example: ComWrappers marshalling with Transparent Structures
Building on this Transparent Structures support, we can also support ComWrappers marshalling with this proposal via the manually-decorated types approach:
```csharp
-[NativeMarshalling(typeof(ComWrappersMarshaler<Foo, FooComWrappers>))]
+[NativeMarshalling(typeof(ComWrappersMarshaler<Foo, FooComWrappers>), Features = CustomTypeMarshallerFeatures.UnmanagedResources | CustomTypeMarshallerFeatures.TwoStageMarshalling)]
class Foo
{}
-struct ComWrappersMarshaler<TClass, TComWrappers>
- where TComWrappers : ComWrappers, new()
+struct FooComWrappersMarshaler
{
- private static readonly TComWrappers ComWrappers = new TComWrappers();
+ private static readonly FooComWrappers ComWrappers = new FooComWrappers();
private IntPtr nativeObj;
- public ComWrappersMarshaler(TClass obj)
+ public ComWrappersMarshaler(Foo obj)
{
nativeObj = ComWrappers.GetOrCreateComInterfaceForObject(obj, CreateComInterfaceFlags.None);
}
- public IntPtr Value { get => nativeObj; set => nativeObj = value; }
+ public IntPtr ToNativeValue() => nativeObj;
+
+ public void FromNativeValue(IntPtr value) => nativeObj = value;
- public TClass ToManaged() => (TClass)ComWrappers.GetOrCreateObjectForComInstance(nativeObj, CreateObjectFlags.None);
+ public Foo ToManaged() => (Foo)ComWrappers.GetOrCreateObjectForComInstance(nativeObj, CreateObjectFlags.None);
public unsafe void FreeNative()
{
diff --git a/docs/project/list-of-diagnostics.md b/docs/project/list-of-diagnostics.md
index 4e2b859114d..edd42ecefb5 100644
--- a/docs/project/list-of-diagnostics.md
+++ b/docs/project/list-of-diagnostics.md
@@ -158,18 +158,15 @@ The diagnostic id values reserved for .NET Libraries analyzer warnings are `SYSL
| __`SYSLIB1052`__ | Specified configuration is not supported by source-generated P/Invokes |
| __`SYSLIB1053`__ | Current target framework is not supported by source-generated P/Invokes |
| __`SYSLIB1054`__ | Specified LibraryImportAttribute arguments cannot be forwarded to DllImportAttribute |
-| __`SYSLIB1055`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1056`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1057`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1058`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1059`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1060`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1061`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1062`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1063`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1064`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1065`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1066`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1067`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1068`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
-| __`SYSLIB1069`__ | *_`SYSLIB1055`-`SYSLIB1069` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1055`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1056`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1057`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1058`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1059`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1060`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1061`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1062`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1063`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1064`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1065`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |
+| __`SYSLIB1066`__ | *_`SYSLIB1055`-`SYSLIB1066` reserved for Microsoft.Interop.LibraryImportGenerator._* |