Down to the MIL

I’ve never really spent a lot of time looking at WPF, but the recent discussions about HTML5 and whether Silverlight still has a place in the application world have encouraged me to have a closer look at it. Declarative GUI has got to be the future, so I thought it would be interesting to see how much of WPF is implemented in managed code and to get an idea of where the boundary is between the managed classes and the unmanaged code which drive DirectX.

In a previous job, where I worked on a Common Lisp programming environment, there was a portable GUI library named CAPI  which allowed the programmer to write cross-platform GUIs. The trick was to implement most of layout and widgets in the managed language (Common Lisp) and then have various backends that interacted with the unmanaged world in the form of the various graphics system. This worked well for Motif, Windows, Cocoa and GTK+.

I was therefore interested to see if WPF has the same pattern – implement the front end of the library in managed code, and then pass data off to the unmanaged world to actually do the rendering. I took my copy of Reflector Pro, decompiled the various assemblies and started stepping through a simple code example.

The world of WPF is slightly different. WPF doesn’t run on top of another windowing system, and therefore doesn’t need to translate the  application windows into a series of widgets that run on top of particular GUI library. Instead it does all of the rendering itself, allowing a series of much better effects such as real transparent backgrounds, hence allowing irregular shaped items. The front end code in C# takes care of managing the layout and items on the screen, and passes off the rendering and composition to an unmanaged component called the Media Integration Layer, shortened to MIL in the code base. The MIL does still know about a series of widget type objects, and in WPF, front end and back end link together via the Visual class. If we take the example of a Grid that contains a button and a TextBox,

<Window x:Class=”WpfApplication6.MainWindow”
        Title=”MainWindow” Height=”350″ Width=”525″>
        <Button Content=”Button” Height=”22″ HorizontalAlignment=”Left” Margin=”180,176,0,0″
                Name=”button1″ VerticalAlignment=”Top” Width=”68″ Click=”button1_Click” />
        <TextBox Height=”23″ HorizontalAlignment=”Left” Margin=”84,52,0,0″
                Name=”textBox1″ VerticalAlignment=”Top” Width=”120″ />

Using Reflector Pro to trace the execution, we can see that a proxy for the Button is generated when the CreateOrAddRefOnChannel method is used in the System.Windows.Media.Composition.VisualProxy class. This method takes three parameters. The first is the instance which is the Button object, the third is the resourceType which has value DUCE.resourceType.TYPE_VISUAL, and the communication is being passed along a DUCE.Channel which is the remaining argument. The resource is represented using an instance of the DUCE.ResourceHandle struct.

This is all driven by the RenderRecursive method in System.Windows.Media.Visual, which is triggered by a need to resize the hosting window.


The HandleMessage function in the above is processing windows messages on the main window, and then dispatching on the message. In the above case we are handling the WM_SIZE message.

On subsequent runs, the various properties of the unmanaged object are updated by calls in the RenderRecursive method which contains the code:

          this.UpdateCacheMode(channel, @null, none, isOnChannel);
          this.UpdateTransform(channel, @null, none, isOnChannel);
          this.UpdateClip(channel, @null, none, isOnChannel);
          this.UpdateOffset(channel, @null, none, isOnChannel);
          this.UpdateEffect(channel, @null, none, isOnChannel);
          this.UpdateGuidelines(channel, @null, none, isOnChannel);
          this.UpdateContent(ctx, none, isOnChannel);
          this.UpdateOpacity(channel, @null, none, isOnChannel);
          this.UpdateOpacityMask(channel, @null, none, isOnChannel);
          this.UpdateRenderOptions(channel, @null, none, isOnChannel);
          this.UpdateChildren(ctx, @null);
          this.UpdateScrollableAreaClip(channel, @null, none, isOnChannel);
          this.SetFlags(channel, false, VisualProxyFlags.None |

There’s one other complication in the WPF world. WPF allows the user to override the rendering that the system does in order to do user drawn objects around the standard drawing. We can see the MIL in operation at this level too.

public class MyTextBox : TextBox
  protected override void OnRender(DrawingContext drawingContext)
         new Pen(Brushes.Bisque, 2.0), new Point(0, 0), 3.0, 5.0);

and modify the TextBox in the earlier example to

<WpfApplication6:MyTextBox Height=”23″ HorizontalAlignment=”Left”
        Margin=”84,52,0,0″ x:Name=”textBox1″ VerticalAlignment=”Top”
        Width=”120″ />

If we now add a button handler, and add the following code into it.

private void button1_Click(object sender, RoutedEventArgs e)
  DrawingGroup group = VisualTreeHelper.GetDrawing(textBox1);
  string output = System.Windows.Markup.XamlWriter.Save(group);

we see the textual representation of the rendering we did within the drawing context.

<DrawingGroup xmlns=>
    <GeometryDrawing Brush=”#FFFFE4C4″>
        <Pen Brush=”#FFFFE4C4″ Thickness=”2″ />
        <EllipseGeometry RadiusX=”3″ RadiusY=”5″ Center=”0,0″ />

What the VisualTreeHelper does in this case is quite interesting. The system goes to the GetDrawing method defined in System.Windows.UIElement, which calls into DrawingServices.DrawingGroupFromRenderData. This uses an object to type DrawingContextDrawingContextWalker which walks a data structure that encodes the extra drawing commands we did on top of the button.

How does the RenderData arrive in place? The DrawingContext passed into our OnRender method is of type RenderDataDrawingContext, and each of the drawing methods that can be called are responsible for allocating a data structure that records (in the managed world) the details of the drawing command, and moreover record this in a form that can be passed as a stream of bytes to the unmanaged layer.

public override unsafe void DrawEllipse(
     Brush brush, Pen pen, Point center, double radiusX, double radiusY)
  if ((brush != null) || (pen != null))
    MILCMD_DRAW_ELLIPSE milcmd_draw_ellipse =
        center, radiusX, radiusY);
        MILCMD.MilDrawEllipse, (byte*) &milcmd_draw_ellipse, 40);

The RenderData produces a structure that records the information about the rendering, and buffers it in a byte array in the RenderData.WriteDataRecord method. This is pushed out to the rendering system in the call to this.UpdateContent which we saw above, which calls RenderData.MarshalToDUCE. This is quite tidy – the data can be passed to the unmanaged world as extra drawing commands, but stays around in the managed world and can be walked using a visitor pattern to reconstruct the drawing as we did in the code above.

In fact, RenderContext is used by more than just the user code. If you take the first version of the code which simply used a WPF Button,  you’ll find that a TextBlock is used to hold the textual part of the button. When the TextBlock has it’s UpdateContext method called in the RenderRecursive (which we saw above), the


calls through to the method

private void UpdateContent(
     RenderContext ctx, VisualProxyFlags flags, bool isOnChannel)
   if ((flags & VisualProxyFlags.IsContentDirty) != VisualProxyFlags.None)
     this.RenderContent(ctx, isOnChannel);
     this.SetFlags(ctx.Channel, false, VisualProxyFlags.IsContentDirty);

which uses the RenderContent on UIElement. This again uses a stored DrawingContext

internal override void RenderContent(RenderContext ctx, bool isOnChannel)
  DUCE.Channel channel = ctx.Channel;
  if (this._drawingContent != null)
    DUCE.IResource resource = this._drawingContent;
                  resource.GetHandle(channel), channel);
    base.SetFlags(channel, true, VisualProxyFlags.IsContentConnected);
  else if (isOnChannel)
                          DUCE.ResourceHandle.Null, channel);

which is placed there when OnRender method of System.Windows.Controls.TextBlock is called. This method uses the various drawing primitives to render the text block.

In contrast, Silverlight has a different set of objects that work across the boundary. MS.Internal.ManagedPeerTable maintains a link between IntPtr from the unmanaged world and the managed objects that they represent. In the Silverlight case the unmanaged equivalent of the Button is generated in the constructor of the Button.

[Managed to Native Transition]   
[Managed to Native Transition]   

The initial cause of the instantiation is the incoming event at the bottom of the stack.

    internal static uint FireEvent(IntPtr unmanagedObj, IntPtr unmanagedObjArgs, int argsTypeIndex, int actualArgsTypeIndex, [MarshalAs(UnmanagedType.LPWStr)] string eventName)

The event comes into the system as a string “M@5” – this is parsed to get the value 5 which is converted into an appropriate delegate. In this case we find something

    [System.Windows.StartupEventHandler]: {System.Windows.StartupEventHandler}
    _methodBase: null
    _methodPtr: 87408664
    _methodPtrAux: 0
    _target: {SilverlightApplication7.App}

The dispatching to the handler is done in MS.Internal.CoreHandler.InvokeEventHandler which uses the typeIndex to decide on an action. In this case we just call into the Application_Startup of our application.

In the Silverlight world, more of the rendering has been moved into the unmanaged code. In WPF, there are lots of places where RenderData is generated to describe the content, allowing user and system rendering to happen for a given element. This kind of customisation isn’t available in the Silverlight world. This lack of flexibility means that the amount of managed code is a lot smaller.

In the days of the Common Lisp system that I worked on, we actively tried to move as much code as possible into the managed world. Common Lisp is in a fairly unique position in that the system could be patched dynamically, and the more code that was managed, the more code we could change by issuing patches, rather than having to redeploy the managed and unmanaged parts.

In some ways it is a shame that all of the composition code is unmanaged. I assume this was done to allow the MIL to be shared with other unmanaged components like the window manager. Some of the Channel9 videos hint that there was an intention to make the MIL (unmanaged) interface accessible by user code, though I don’t believe that has happened yet. It is quite impressive to see the buffering of the RenderData inside the managed world inside byte vectors that can be efficiently pushed into the unmanaged world, but Silverlight needs to be small and fast, so this flexibility has been removed from the platform.

This entry was posted in Computers and Internet. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s