Tuesday, June 23, 2009

Managed Extensibility Framework - Part1

I wanted to make this post since last two weeks. For some reason it didn’t happen. Today I was asked by one of my colleagues so I couldn’t take another exception.

First let me introduce you what is officially told about this framework

“The Managed Extensibility Framework (MEF) is a new library in .NET that enables greater reuse of applications and components. Using MEF, .NET applications can make the shift from being statically compiled to dynamically composed. If you are building extensible applications, extensible frameworks and application extensions, then MEF is for you.”

So take that middle sentence “statically compiled to dynamically composed”. It is all about that.

Building reusable code into libraries and using them in applications is normal. In legacy Win32 applications if a library is linked to an application, linker used to add reference to that library in PE header. When runtime tries to load this application loader used to perform following operations

  1. resolve all dependencies
  2. load them into memory
  3. map them to respective pointers
  4. then start main routine

But with CLR, and JIT compilation in the middle, things are slightly different. Now loader tries to load those libraries which are needed to execute just the Main method. As other methods are called from Main, IL for those methods are JIT compiled on demand and loaded into memory. These things can be simplified with an example. [By the way, if you are thinking that I am going away from main subject of MEF; you are thinking correctly. But I am doing this internationally. Because I feel understanding about the problem that MEF is trying to solve makes you an informed user of MEF, so that you will be able clearly distinguish when and where MEF can be used. So I will keep this part of the series just for introduction.]

Now coming back to example. Let us start with a simple console application. Name it “WOMEF”. (without MEF.)

Replace the code in Program.cs with following snippet.

using System;
using WOMEFLib;

namespace WOMEF
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Enter 0 to quit, any other number to continue");
while (Console.ReadLine() != "0")
{
new Program().SayHelloToUser();
}
}

public void SayHelloToUser()
{
Console.WriteLine("In SayHelloToUser");
IWOMEFLib womeflib = new CWOMEFLib();
womeflib.SayHello(Environment.UserDomainName + Environment.UserName);
Console.ReadLine();
}
}
}
Let us define IWOMEFLib and CWOMEFLib in a library. So add a class library project to this solution by name “WOMEFLib”.

Rename Class1.cs to CWOMEFLib.cs. Then replace code with following snippet.


using System;
namespace WOMEFLib
{
public interface IWOMEFLib
{
int SayHello(string username);
}
public class CWOMEFLib : IWOMEFLib
{

#region IWOMEFLib Members

public int SayHello(string username)
{
Console.WriteLine("\"" + "Say Hello to " + username + "\"");
return 0;
}

#endregion
}
}
Now in WOMEF project add a project reference to WOMEFLib project. Change configuration to Release, build and test application.

So nothing great about this. But we are going to witness some interesting facts with these two assemblies.

  1. When loader loads CWOMEFLib.dll

  2. What happens if CWOMEFLib.dll is missing when application is starting

  3. When SayHelloToUser methods is JIT compiled

  4. What happens if CWOMEFLib.dll is copied to application directory while it is waiting for user input

In order to check these scenarios we are going to view method table and descriptors of application WOMEFLib.exe in runtime.

We are going to use Windbg here. If you don’t have it installed download it at link.

When loader loads CWOMEFLib.dll

Here we are going to check when loader loads CWOMEFLib.dll.

  1. Start Windbg.

  2. In Windbg, click on File –> Open executable

  3. Navigate to Release folder of solution and select WOMEF.exe

  4. Windbg stops with interrupt 3. Press F5 to continue.
  5. ntdll!DbgBreakPoint:
    7c90120e cc int 3


  6. Now application starts and waits for user entry with message “Enter 0 to quit, any other number to continue”

  7. In Windbg go to Debug menu and click on break. This is the ideal time to break into application as it is up and running.

  8. Now load sos.dll in windbg by entering following command in Windbg “.loadby sos.dll mscorwks”
  9. ntdll!DbgBreakPoint:
    7c90120e cc int 3
    Missing image name, possible paged-out or corrupt data.
    0:003> .loadby sos.dll mscorwks


  10. Now dumpdomains by entering command “!dumpdomain”

  11. 0:003> !dumpdomain
    *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\Microsoft.NET\... -
    PDB symbol for mscorwks.dll not loaded
    --------------------------------------
    System Domain: 7a3bd058
    LowFrequencyHeap: 7a3bd07c
    HighFrequencyHeap: 7a3bd0c8
    StubHeap: 7a3bd114
    Stage: OPEN
    Name: None
    --------------------------------------
    Shared Domain: 7a3bc9a8
    LowFrequencyHeap: 7a3bc9cc
    HighFrequencyHeap: 7a3bca18
    StubHeap: 7a3bca64
    Stage: OPEN
    Name: None
    Assembly: 001b0628
    --------------------------------------
    Domain 1: 0016d298
    LowFrequencyHeap: 0016d2bc
    HighFrequencyHeap: 0016d308
    StubHeap: 0016d354
    Stage: OPEN
    SecurityDescriptor: 0016e5c0
    Name: WOMEF.exe
    Assembly: 001b0628 [C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll]
    ClassLoader: 001b06a8
    SecurityDescriptor: 001ae190
    Module Name
    033e1000 C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll


    Assembly: 001b9800 [D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe]
    ClassLoader: 001b9e68
    SecurityDescriptor: 001b96c8
    Module Name
    00a72c5c D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe


    Assembly: 001bcb30 [D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEFLib.dll]
    ClassLoader: 001bbc98
    SecurityDescriptor: 001bcd28
    Module Name
    00a730c8 D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEFLib.dll

  12. As expected there are three app domains. In application specific appdomain three assemblies are loaded. First one (mscorlib.dll) is pretty standard one. Second one (WOMEF.exe) is application itself. And third one (WOMEFLib.dll) is referenced assembly.

  13. So loader loaded referenced assembly though we have not yet used any type from it so far. We are just at first line in Main method. Also we don’t use any type from WOMEF.dll in Main method. Still loader loaded this assembly because it is there in manifest. Now let us see what happens if this assembly is not available when we start WOMEF.exe. Kill application by pressing Control+C. If needed close Windbg and reopen.

What happens if CWOMEFLib.dll is missing when application is starting

Here we are going to check what happens if CWOMEFLib.dll is missing when application is starting. Move CWOMEFLib.dll from Release folder to some other folder outside solution directory (say to Desktop. Don’t copy cut it.)

  1. Start Windbg.

  2. In Windbg, click on File –> Open executable

  3. Navigate to Release folder of solution and select WOMEF.exe

  4. Windbg stops with interrupt 3. Press F5 to continue.
  5. ntdll!DbgBreakPoint:
    7c90120e cc int 3

  6. Now application starts and waits for user entry with message “Enter 0 to quit, any other number to continue”.

  7. Though one of the dependent assembly is not available application runs normally till this point. This makes an interesting point. Why loader didn’t complain that CWOMEFLib.dll is not available? Answer is SayHelloToUser is not yet JIT compiled. And we will verify this fact in next section. Kill application by pressing Control+C. If needed close Windbg and reopen.

When SayHelloToUser methods is JIT compiled

Here we are going to check when SayHelloToUser methods is actually JIT compiled. Move back CWOMEFLib.dll to Release folder.

  1. Start Windbg.

  2. In Windbg, click on File –> Open executable

  3. Navigate to Release folder of solution and select WOMEF.exe

  4. Windbg stops with interrupt 3. Press F5 to continue.
  5. ntdll!DbgBreakPoint:
    7c90120e cc int 3
  6. Now application starts and waits for user entry with message “Enter 0 to quit, any other number to continue”.

  7. In Windbg go to Debug menu and click on break. This is the ideal time to break into application as it is up and running.

  8. Now load sos.dll in windbg by entering following command in Windbg “.loadby sos.dll mscorwks”
  9. ntdll!DbgBreakPoint:
    7c90120e cc int 3
    Missing image name, possible paged-out or corrupt data.
    0:003> .loadby sos.dll mscorwks


  10. Now dumpdomains by entering command “!dumpdomain”

  11. 0:003> !dumpdomain
    *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\Microsoft.NET\... -
    PDB symbol for mscorwks.dll not loaded
    --------------------------------------
    System Domain: 7a3bd058
    LowFrequencyHeap: 7a3bd07c
    HighFrequencyHeap: 7a3bd0c8
    StubHeap: 7a3bd114
    Stage: OPEN
    Name: None
    --------------------------------------
    Shared Domain: 7a3bc9a8
    LowFrequencyHeap: 7a3bc9cc
    HighFrequencyHeap: 7a3bca18
    StubHeap: 7a3bca64
    Stage: OPEN
    Name: None
    Assembly: 001b0628
    --------------------------------------
    Domain 1: 0016d298
    LowFrequencyHeap: 0016d2bc
    HighFrequencyHeap: 0016d308
    StubHeap: 0016d354
    Stage: OPEN
    SecurityDescriptor: 0016e5c0
    Name: WOMEF.exe
    Assembly: 001b0628 [C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll]
    ClassLoader: 001b06a8
    SecurityDescriptor: 001ae190
    Module Name
    033e1000 C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll


    Assembly: 001b9800 [D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe]
    ClassLoader: 001b9e68
    SecurityDescriptor: 001b96c8
    Module Name
    00a72c5c D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe


    Assembly: 001bcb30 [D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEFLib.dll]
    ClassLoader: 001bbc98
    SecurityDescriptor: 001bcd28
    Module Name
    00a730c8 D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEFLib.dll

  12. Now we are just at first line of Main method. Let us get address if method table of assembly WOMEF.exe by using command !dumpmodule –mt <module address 00a72c5c>. See module address in red color.
  13. 0:003> !dumpmodule -mt 00a72c5c
    Name: D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe
    Attributes: PEFile
    Assembly: 001b9780
    LoaderHeap: 00000000
    TypeDefToMethodTableMap: 00a700c0
    TypeRefToMethodTableMap: 00a700cc
    MethodDefToDescMap: 00a70128
    FieldDefToDescMap: 00a70138
    MemberRefToDescMap: 00a7013c
    FileReferencesMap: 00a701a4
    AssemblyReferencesMap: 00a701a8
    MetaData start address: 004020c0 (1864 bytes)

    Types defined in this module

    MT TypeDef Name
    ------------------------------------------------------------------------------
    00a73010 0x02000002 WOMEF.Program

    Types referenced in this module

    MT TypeRef Name
    ------------------------------------------------------------------------------
    03650508 0x01000001 System.Object
    03654258 0x01000012 System.Console
    036508ec 0x01000013 System.String
    00a73484 0x01000016 WOMEFLib.IWOMEFLib


  14. Using method table address let us get module descriptions by using command !dumpmt –md <method table 00a73010>. See mt address in red color.
  15. 0:003> !dumpmt -md 00a73010
    EEClass: 00a712f4
    Module: 00a72c5c
    Name: WOMEF.Program
    mdToken: 02000002 (D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe)
    BaseSize: 0xc
    ComponentSize: 0x0
    Number of IFaces in IFaceMap: 0
    Slots in VTable: 7
    --------------------------------------
    MethodDesc Table
    Entry MethodDesc JIT Name
    035a6a70 03424934 PreJIT System.Object.ToString()
    035a6a90 0342493c PreJIT System.Object.Equals(System.Object)
    035a6b00 0342496c PreJIT System.Object.GetHashCode()
    036172f0 03424990 PreJIT System.Object.Finalize()
    00a7c019 00a73008 NONE WOMEF.Program..ctor()
    00de0070 00a72ff0 JIT WOMEF.Program.Main(System.String[])
    00a7c015 00a72ffc NONE WOMEF.Program.SayHelloToUser()

  16. Here observe the JIT status of SayHelloToUser method as NONE. Means that SayHelloToUser is not yet JIT compiled. Let us run the application by pressing F5 in Windbg and entering a non zero value in console. As soon as you enter a non-zero value in console CLR compiles SayHelloUser, loads it from with reference to SayHello method in WOMEFLib.dll. Now console waits for input because of ReadLine.

  17. Now let us break into debugger again by clicking on Break menu option of Debug menu in Windbg.

  18. Let us check status of method description again by using same command !dumpmt –md <method table 00a73010>. You can even use up arrow twice in windbg to get there.
  19. 0:003> !dumpmt -md 00a73010
    EEClass: 00a712f4
    Module: 00a72c5c
    Name: WOMEF.Program
    mdToken: 02000002 (D:\My Documents\Visual Studio 2008\Projects\MEF\WOMEF\WOMEF\bin\Release\WOMEF.exe)
    BaseSize: 0xc
    ComponentSize: 0x0
    Number of IFaces in IFaceMap: 0
    Slots in VTable: 7
    --------------------------------------
    MethodDesc Table
    Entry MethodDesc JIT Name
    035a6a70 03424934 PreJIT System.Object.ToString()
    035a6a90 0342493c PreJIT System.Object.Equals(System.Object)
    035a6b00 0342496c PreJIT System.Object.GetHashCode()
    036172f0 03424990 PreJIT System.Object.Finalize()
    00a7c019 00a73008 NONE WOMEF.Program..ctor()
    00de0070 00a72ff0 JIT WOMEF.Program.Main(System.String[])
    00de00d0 00a72ffc JIT WOMEF.Program.SayHelloToUser()


  20. Now observe status of SayHelloToUser method as JIT. This makes it clear that SayHelloToUser method would have never been JIT compiled if a zero is entered in console window as the first option. So when control goes inside while loop CLR started JIT compiling called method.

  21. Let us see a tricky case now. Assume that we wont make WOMEFLib.dll available to WOMEF.exe when WOMEF.exe is starting but we will copy this assembly while application is waiting for user input with message “Enter 0 to quit, any other number to continue” for the first time. Kill application by pressing Control+C. For this last case we don’t need windbg.

What happens if CWOMEFLib.dll is copied to application directory while it is waiting for user input

Here we are going to check what happens if CWOMEFLib.dll is copied to application directory while it is waiting for user input for the first time. Move CWOMEFLib.dll from Release folder to some other folder outside solution directory (say to Desktop. Don’t copy cut it.)
  1. Open command prompt

  2. Navigate to Release folder of solution and start WOMEF.exe

  3. Application starts and waits for user entry with message “Enter 0 to quit, any other number to continue”. At this point move back CWOMEFLib.dll to Release folder.

  4. Enter a non-zero value and press enter

  5. Now when application tries to compile SayHelloToUser, it tries to refer type CWOMEF in WOMEFLib.dll and fails to resolve assembly. It comes out with an exception

  6. Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'WOMEFLib, ..


  7. That means, even though the assembly is made available when it is actually being referred, loader ignores to reload.

  8. It is this particular aspect that MEF tries to solve.

Existing possibilities

OK with that background I hope you are clear on what MEF is trying to solve. But before looking at MEF let us see how we can solve this particular problem with existing technology. With CLR 2.0 we have an option of loading assembly in runtime. After loading assembly in runtime we can create objects of types defined in it and we can even invoke them using reflection techniques. This is quite straight forward. I will just provide a working sample. Do following changes to our solution
  1. In WOMEF project remove reference to WOMEFLib.dll.

  2. Then replace code in Program.cs with following snippet

  3. using System;
    using System.Reflection;
    namespace WOMEF
    {

    class Program
    {
    static void Main(string[] args)
    {
    Console.WriteLine("Enter 0 to quit, any other number to continue");
    while (Console.ReadLine() != "0")
    {
    new Program().SayHelloToUser();
    }
    }

    public void SayHelloToUser()
    {
    Console.WriteLine("In SayHelloToUser");
    InvokeMethod("WOMEFLib", "CWOMEFLib", "SayHello", new object[] { Environment.UserDomainName+Environment.UserName });
    Console.ReadLine();
    }

    public object InvokeMethod(string assemblyName, string className, string methodName, object[] parameters)
    {
    System.Type[] parametersTypes;
    int parametersCount = 0;
    object returnObject = null;
    try
    {
    Type type = System.Reflection.Assembly.LoadFrom(assemblyName + ".dll").GetType(
    assemblyName + "." + className);

    parametersTypes = new System.Type[parameters.GetUpperBound(0) + 1];

    foreach (object parameter in parameters)
    {
    if (parameter != null)
    parametersTypes[parametersCount] = parameter.GetType();
    parametersCount++;
    }

    MethodInfo mi = type.GetMethod(methodName, parametersTypes);
    ConstructorInfo ci = type.GetConstructor(Type.EmptyTypes);
    object objectinstance = ci.Invoke(null);
    //Invoke
    returnObject = mi.Invoke(objectinstance, parameters);
    }
    catch (TargetException tex)
    {
    throw tex;
    }

    return returnObject;
    }
    }
    }

  4. Here Invoke method loads assembly (WOMEFLib) in runtime, constructs class object (CWOMEFLib) and invokes a method (SayHello) in it.

In next post we will see how MEF framework not only simplifies this process but also provides many more useful options for tihis kind of scenarios.

2 comments:

Venkatesu Punugupati said...

Excelent article.
Expecting More like this.

Anonymous said...

Nice brief and this enter helped me alot in my college assignement. Say thank you you seeking your information.