Passing Managed Structures With Arrays To Unmanaged Code Part 1

• http://limbioliong.wordpress.com/2011/06/29/passing-managed-structures-with-arrays-to-unmanaged-code-part-1/
  
  
  ABOUT

//

you're reading...

INTEROP MARSHALING

Passing Managed Structures With Arrays To Unmanaged Code Part 1

POSTED BY LIM BIO LIONG  ⋅ JUNE 29, 2011 ⋅ 2 COMMENTS

FILED UNDER   .NET,  C-STYLE ARRAY,  INTEROP,  INTEROP MARSHALER,  MANAGED,  MANAGED ARRAY,  SAFEARRAY,  SAFEARRAYALLOCDATA, SAFEARRAYALLOCDESCRIPTOREX,  SAFEARRAYDESTROY,  SIZECONST,  STRUCTURE,  UNMANAGED,  UNMANAGEDTYPE.BYVALARRAY

1. Introduction

1.1 Managed structures that contain arrays are a common requirement in .NET development. Such structures, however, cannot be conveniently passed to unmanaged code. This is because arrays are referenced types and are not blittable. And because arrays are not blittable, the containing structure is also non-blittable.

1.2 This series of blogs will explore how managed structures (which contain arrays) are transferred to unmanaged code. I shall also endeavour to show low-level activities including the location of the actual structures as they are passed to unmanaged APIs.

2. Example Structure.

2.1 Let’s define an example structure in C# :

public struct TestStruct01
{
  public int m_int;
  public int[] m_int_array;
};

2.2 Note that a managed structure, when passed to unmanaged code, must be constructed based on the memory layout expected by the unmanaged code. For example, where the unmanaged code is C/C++, the memory layout of the structure must be expressed C/C++ style.

2.3 In order to ensure the required memory layout, the CLR uses the services of the interop marshaler. The interop marshaler is the entity that performs the actual marshaling and calling of unmanaged code. Hence such a structure, if it is to be successfully passed to unmanaged code, must be further specified for the benefit of the interop marshaler. These specifications come in the form of the StructLayoutAttribute and the MarshalAsAttribute.

2.4 The StructLayoutAttribute indicates how the structure members are to be laid out in memory. Two arguments for this attribute are important :

• The LayoutKind argument.
• The Pack argument.

The LayoutKind argument indicates how the members of the structure are to be arranaged. The “LayoutKind.Sequential” value is the most commonly used. It indicates that the members are laid out back to back as they appear in the declaration. For example, “m_int” is the first member in memory, followed by “m_int_array”, etc.

The Pack argument indicates how the members of the structure will be spaced out. This argument corresponds directly to the C/C++ concept of struct member alignment. This affects the byte offset of each member in the structure. It can be helpful for optimization purposes. For simplicity, we shall use the value of 1 (i.e. 1-byte alignment).

Note, however, that the corresponding unmanaged version of the structure must also use the same struct member alignment. Things can go awry if the alignments do not match.

2.5 Hence at minimum, the structure should be declared as follows with the StructLayoutAttribute :

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct TestStruct01
{
...
};

2.6 Now, a blittable structure (i.e. one that contains only members of blittable type), is directly accessible from unmanaged code. As an optimization technique, the interop marshaler may, in certain circumstances, simply pass a pointer to such structure (in managed memory) to unmanaged code. A non-blittable structure (i.e. one that contains at least one member of non-blittable type), on the other hand, when passed to unmanaged code, must be represented by an unmanaged version that resembles the memory layout of an equivalent structure expected by the unmanaged code.

2.7 If the unmanaged code is developed in C/C++, the unmanaged structure is the equivalent of the managed structure but with each non-blittable member flattened and serialized. It must eventually resemble a C/C++ structure.

2.8 For more information on unmanaged representation of managed structs, please refer to :

“Passing Structures between Managed and Unmanaged Code”

http://limbioliong.wordpress.com/2011/06/03/passing-structures-between-managed-and-unmanaged-code/

2.9 In order to be serialized properly into the unmanaged representation of the structure, each struct member must provide information on how this is to be done. This is where the MarshalAsAttribute comes into play. If no MarshalAsAttribute is specified for a member, default marshaling procedures will be used.

2.10 Now the “m_int” member is a blittable type and so default marshaling will be used. This member need not be further specified using MarshalAsAttributes. The “m_int_array” member, however, is a referenced type. It is not blittable. It must necessarily be specified further using MarshalAsAttributes. The purpose of the MarshalAsAttribute is to indicate to the interop marshaler how the “m_int_array” member is to be represented in unmanaged code.

2.11 There are 2 ways that the “m_int_array” member may be represented in an unmanaged C/C++ structure :

• As a fixed-length array.
• As a SAFEARRAY (non-fixed-length array).

2.12 In section 3, we will examine how to express “m_int_array” as a fixed-length array in an unmanaged C/C++ structure and in section 5, we will expore how “m_int_array” may be expressed as a SAFEARRAY.

3. Representing a Managed Array Struct Member as a Fixed-Length Array.

3.1 To achieve this, the MarshalAsAttribute must be expressed as follows :

[MarshalAs(UnmanagedType.ByValArray, SizeConst = <length of array>)]

Note the following points about the “UnmanagedType.ByValArray” argument :

• The “UnmanagedType.ByValArray” argument indicates that the array is a  “ByVal” array.
• This means that the array is passed  “by value” (as opposed to ”by reference”).
• This also means that all elements of the array will be laid out in memory back-to-back in the containing unmanaged version of the structure.

Note the following point about the “SizeConst” argument :

• The value set for the “SizeConst” argument indicates the number of elements in the array.

3.2 Hence a full declaration for the TestStruct01 structure is listed below :

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct TestStruct01
{
  public int m_int;
  [MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
  public int[] m_int_array;
};

The value 10 for the SizeConst argument is just an arbitrary value that I picked for example purposes.

3.3 The equivalent of such a structure in C/C++ would be :

#pragma pack(1)
struct TestStruct01
{
  int m_int;
  int m_int_array[10];
};

3.4 Let’s say we want to pass this structure to an unmanaged C++ API with the following code :

void __stdcall DisplayStruct01(TestStruct01 test_struct_01)
{
  printf ("test_struct_01.m_int : <%d>.\r\n", test_struct_01.m_int);

  for (int i = 0; i < 10; i++)
  {
    printf ("test_struct_01.m_int_array[%d] : <%d>.\r\n", i, test_struct_01.m_int_array[i]);
  }

  return;
}

Note that the “test_struct_01″ struct parameter is passed by value.

3.5 The following would be the way the API is declared in managed C# :

[DllImport(@"TestDLL.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void DisplayStruct01(TestStruct01 test_struct_01);

3.6 The following is an example C# function that calls this API :

static void DisplayStruct01InUnmanagedCode()
{
  TestStruct01 test_struct_01 = new TestStruct01();

  test_struct_01.m_int = 0;
  test_struct_01.m_int_array = new int[10];

  for (int i = 0; i < 10; i++)
  {
    test_struct_01.m_int_array[i] = i;
  }

  DisplayStruct01(test_struct_01);
}

This function creates a TestStruct01 structure, fills it with values, and then calls the DisplayStruct01() API.

3.7 In the next section, we shall examine how the managed structure ends up being represented by an unmanaged version of it and where this unmanaged structure resides in memory so as to be accessible by the API.

4.Memory Location of the Structure.

4.1 Now, as mentioned in point 2.7, when a non-blittable structure is passed to unmanaged code, an unmanaged representation of this structure must be created and then passed to the unmanaged code. In the example that we have built up, where does this unmanaged representation reside ?

4.2 As things turn out, this unmanaged representation resides in the call stack of the called API. This is because the parameter is passed by value :

void __stdcall DisplayStruct01(TestStruct01 test_struct_01); 

[DllImport(@"TestDLL.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void DisplayStruct01(TestStruct01 test_struct_01);

Hence a copy of the TestStruct01 structure is to be pushed onto the stack and will then be accessed by value by the DisplayStruct01() API. Now this is where the interop marshaler takes the opportunity to perform its task : it transforms the managed StructWithArray structure into its unmanaged equivalent using the call stack as the target memory.

Using the call stack is most natural not only because the structure is passed by value, but also because the memory is immediately recovered when the API returns.

5. Representing a Managed Array Struct Member as a SAFEARRAY.

5.1 To achieve this, the the MarshalAsAttribute must be expressed as follows :

[MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VarEnum.VT_I4)]

Note the following point about the UnmanagedType.SafeArray argument :

• It indicates that the array is to be marshaled across to unmanaged code as a COM SAFEARRAY
• Now, being a SAFEARRAY, the number of elements in the array need not be fixed. It can be set dynamically. This is one advantage of using a SAFEARRAY over an inline array.
• This time, the corresponding array member in the C/C++ structure can no longer be declared as an inline array (as indicated in point 3.3).
• It needs to be declared as a pointer to a SAFEARRAY (more on this below).

Note the following point about the SafeArraySubType argument :

• It indicates the VARIANT Type which is to be contained inside the SAFEARRAY to be created.
• In the case of our example, this is set to VT_I4 which fits the description of a 4-byte integer (i.e. a 32-bit integer).

As mentioned above, the corresponding array member of the unmanaged struct must be declared as a pointer to a SAFEARRAY instead of an inline array :

struct TestStruct02
{
  int m_int;
  SAFEARRAY* pSafeArrayOfInt;
};

Note that I have renamed the struct to TestStruct02 so as to differentiate it from the previously declared TestStruct01.

5.2 A full declaration for the managed TestStruct02 structure is listed below :

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct TestStruct02
{
  public int m_int;
  [MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VarEnum.VT_I4)]
  public int[] m_int_array;
};

Notice that besides the different arguments used for the MarshalAsAttribute, this struct is practically the same as the previous TestStruct01 structure.

5.3 This time, however, the unmanaged C++ code for displaying the array would have to use SAFEARRAY APIs :

void __stdcall DisplayStruct02(TestStruct02 test_struct_02)
{
  printf ("test_struct_02.m_int : [%d].\r\n", test_struct_02.m_int);

  long lLbound = 0;
  long lUbound = 0;

  // Obtain bounds information of the SAFEARRAY.
  SafeArrayGetLBound(test_struct_02.pSafeArrayOfInt, 1, &lLbound);
  SafeArrayGetUBound(test_struct_02.pSafeArrayOfInt, 1, &lUbound);
  long lDimSize = lUbound - lLbound + 1;

  for (int i = 0; i < lDimSize; i++)
  {
    long rgIndices[1];
    int value;

    rgIndices[0] = i;
    SafeArrayGetElement
    (
      test_struct_02.pSafeArrayOfInt,
      rgIndices,
      (void FAR*)&value
    );
    printf ("[%d]\r\n", value);
  }
}

5.4 And the C# code for calling this API is listed below together with the declaration for the DisplayStruct02() API :

[DllImport(@"TestDLL.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void DisplayStruct02(TestStruct02 test_struct_02);

static void DisplayStruct02InUnmanagedCode()
{
  TestStruct02 test_struct_02 = new TestStruct02();

  test_struct_02.m_int = 0;
  test_struct_02.m_int_array = new int[9];

  for (int i = 0; i < 9; i++)
  {
    test_struct_02.m_int_array[i] = i;
  }

  DisplayStruct02(test_struct_02);
}

5.5.Notice that there is practically no difference between DisplayStruct02InUnmanagedCode() and the earlier DisplayStruct01InUnmanagedCode() (point 3.6) except that different but similar structs were used and different APIs were called.

5.6 The only significant difference is that the “m_int_array” member array is now variable in size and need not be restricted to a fixed number of elements. In the above example code, the upper bound of 9 is just an arbitrary number used for example purposes.

6.Memory Location of the Structure and the SAFEARRAY.

6.1 Now, similar to the previous example where TestStruct01 was passed to the DisplayStruct01() API, the TestStruct02 struct was also passed to the DisplayStruct02() API by value.

6.2 This means that the entire unmanaged representation of the “test_struct_02″ struct is copied to the call stack for the DisplayStruct02() API. 

6.3 Now, what about the SAFEARRAY member “pSafeArrayOfInt” ? This is where the interop marshaler plays the significant role of SAFEARRAY creation, data assignment and final destruction.

6.4 As the unmanaged representation of the “test_struct_02″ struct is being prepared for the DisplayStruct02() API, the interop marshaler creates a SAFEARRAY using the following APIs :

• SafeArrayAllocDescriptorEx()
• SafeArrayAllocData()

After SafeArrayAllocData() is called, the interop marshaler proceeds to directly copy the data of the managed array to the buffer of the SAFEARRAY using the “pvData” member of the created SAFEARRAY.

6.5 When all data has been copied, a pointer to this SAFEARRAY is then set to the “pSafeArrayOfInt” member of the TestStruct02 struct which is finally passed to the DisplayStruct02() API.

6.6 Then, when the API returns, the SAFEARRAY is destroyed using the SafeArrayDestroy() API.

7. In Conclusion.

7.1 In part 1, we have studied how an array containing structure may be passed to unmanaged code as a “by value” parameter.

7.2 We have also examined the various options available for expressing a managed array struct member in unmanaged code. That is, as an inline array or as a SAFEARRAY.

7.3 In part 2, we shall explore how to pass an array containing structure as a return (i.e. “out”) parameter.