Archive for October, 2014

Some WinDbg tips

October 30, 2014


You are reading the old blog! This post has been moved to


I’ve gathered some WinDbg tips over time (mostly for managed dump analysis) and this seems like as good a place as any to share them, so here you go.

Preparation (one time)

  • Install the latest debugging tools from the Dev Center
    • Let’s assume you install them to c:debuggers
  • Download sosex.dll and place it in c:debuggers
  • Create an environment variable named _NT_SYMBOL_PATH with the value C:Symbols;srv*C:symbols*
    • Most Microsoft symbols should be found in the symbol server and will be cached in C:Symbols
    • You can copy any other symbols (PDBs) you have to the C:Symbols folder as well
  • Create an environment variable named _NT_SOURCE_PATH with the value srv* 
    • This will enable source serving (when source indexing was included in the build) – simply double-click the relevant line in the Call Stack (calls) window and you should jump straight to the relevant line in the source code.
    • You might wonder how that’s possible when no server was specified. The answer is a bit surprising – strictly speaking, there is no such thing as a source server! The name is a bit misleading. What actually happens is that the source-indexed PDB contains the proper command(s) to retrieve the relevant source files. By default this “command” would be something like C:srcfoo.cs, and it would only work if the file in the correct version is actually there. However, if source-indexing was enabled in TFS build, it would look something like tf get MyClass.cs /version:C8 (i.e. the file will be retrieved directly from source, with the correct version).

Preparation (per debugging session)

  • Open the dump file in WinDbg
    • be sure to match the architecture to the analyzed process – use WinDbg x86 for 32-bit processes and WinDbg x64 for 64-bit processes
  • Enable Debugger Markup Language (DML) by issuing .prefer_dml 1
    • This will make the output of some commands contain convenient hyperlinks
  • Load the SOSEX extension by issuing the command .load sosex.dll
    • Don’t load SOS.dll manually – the first command below (analyze -v) will load it with the correct version automatically


  • Automatic exception analysis: !analyze -v
    • In many cases this will suffice to find the root cause!
  • Threads
    • !Threads – lists all the managed threads in the process
  • Stack commands
    • !clrstack – provides a true stack trace for managed code only
    • !dumpstack – provides a verbose stack trace
    • !eestac – runs !dumpStack on all threads in the process
    • !dso – displays any managed objects within the bounds of the current stack
    • ! – produces and displays a merged stack trace of managed and unmanaged frames. Note that in addition to the native offset, a managed (IL) offset is specified – this is extremely useful for debugging NullReferenceException’s in that the exact offending IL instruction is indicated (actually it will be one instruction before the offending one in the specific case of NullReferenceException)
    • !sosex.mdso – dumps object references on the stack and in CPU registers in the current context
    • !sosex.mdv – displays argument and local variable information for the current frame
    • !sosex.mframe – displays or sets the current managed frame for the !mdt and !mdv commands
  • Heap commands
    • !eeheap – enumerates process memory consumed by internal CLR data structures
    • !DumpHeap – traverses the garbage collected heap
  • Object commands
    • !do – allows you to examine the fields of an object, as well as learn important properties of the object
    • !dumparray – examines elements of an array object
    • !dumpvc – examines the fields of a value class
    • !sosex.mdt – displays the fields of the specified object or type
  • Method commands
    • !dumpmt – examines a MethodTable
    • !DumpIL – prints the IL code associated with a managed method
    • !U – presents an annotated disassembly of a managed method
    • !sosex.muf – disassembles the method specified by the given MD or code address with interleaved source, IL, and assembly code
  • Exception commands
    • !pe exceptionAddress – formats fields of any object derived from System.Exception
  • GC commands
    • !GCRoot – looks for references (or roots) to an object
  • SOS Help
    • !
    • ! FAQ
  • SOSEX Help
    • !sosexhelp or !

Mismatched SOS.dll versions

!analyze -v  should get the correct SOS.dll version for you. Even if for some reason it doesn’t, SOS.dll warnings can be ignored most of the time, so long as the mscordacwks.dll version is correct (see the next section if that’s not the case). However there may be cases where the correct SOS.dll version is needed (or perhaps you just want to get rid of the warning). Here are some places to look for it (once you find it, copy it to your debugger folder and issue .load sos.dll):

  • Your best bet is the machine on which the dump was taken. Provided that it’s accessible and wasn’t patched since the time the dump was taken, the correct version should be found at C:WindowsMicrosoft.NETFramework64v4.0.30319SOS.dll. Of course it should be there on your machine as well, and your local version may happen to match (or be close enough).
  • You may find the version you’re looking for here (you should be able to extract the file from the update package itself using 7-zip).
  • You may also want to try Psscor4.dll instead of SOS.dll (Psscor is a superset of SOS) – its version need not match the dump (aside of being .NET 4.0). Note that it is less maintained than SOS.
  • For more information see

Mismatched mscordacwks.dll versions

WinDBG should find the correct mscordacwks.dll automatically and download it from the symbol server. If that doesn’t happen, try and do it explicitly by running .cordll -ve -u -l (you’d might want to first run !sym noisy and/or !symfix in order to troubleshoot better – see the next section for details). Failing that, try and get the correct version from the following places, and run .cordll -u -ve -lp PathToFolderContainingMscorDAC once you have it.

  • Again, your best bet is the machine on which the dump was taken (under the same caveats as above). It should be found at C:WindowsMicrosoft.NETFramework64v4.0.30319mscordacwks.dll. As before, it will be there on your machine so you could luck out.
  • The following post lists many versions of CLR 4.0, you may be able to extract the correct version with the same method as above (use 7-zip to open the cab files inside the archive).
  • For more information see

Troubleshooting missing symbols and sources

  • !sym noisy – increases symbol verbosity (always enable this when troubleshooting symbol issues)
  • .srcnoisy 3 – increases source resolution vebosity (use it when double-clicking lines in the callstack window doesn’t bring up the correct sources)
  • lm – displays the specified loaded modules
    • lme displays only modules that have a symbol problem (very useful)
  • .reload /f – forces the reloading of all symbols
    • .reload /f SomeAssembly.dll – forces the reloading of a specified DLL

For further reading try to hit F1 inside WinDbg – the documentation is very good !

Getting started with ETW using .NET’s EventSource

October 13, 2014


You are reading the old blog! This post has been moved to


.NET 4.5 introduced the EventSource class, allowing convenient access to Event Tracing for Windows (ETW) from managed code. This is a boon for enterprise developers, and I encourage you to go read up on it at MSDN. Also, be sure sure to check out Vance Morrison’s EventSource blog entries. Another useful blog by MS MVP Kathleen Dollard is Leaning into Windows, and Muhammad Shujaat Siddiqi’s blog is worth checking out as well.

Once you’ve got the hang of EventSource, take it to the next level with the Enterprise Library Semantic Logging Application Block (SLAB). You can do some pretty cool stuff with it, including verification of your EventSource class validity, and automatic routing of ETW events to different storage platforms (such as databases, Azure tables, and text files). Start with the developer’s guide.

Finally, if you take a close look at the documentation of the EventSource class, you’ll notice the following note:

There is a NuGet version of the EventSource class that provides more features. For more information, see Microsoft EventSource Library 1.0.16.

I couldn’t find any documentation for this mysterious NuGet package, but I did find its samples package. Once you install it you can take a look at the extensively documented code as well as debug and step into the parts you’re interested in. Be sure to read the guides that accompany the samples: _EventRegisterUsersGuide.docx and _EventSourceUsersGuide.docx – they pertain to the vanilla library that comes with .NET as well (not just the NuGet package).

One thing I couldn’t find is a list of differences between the vanilla .NET EventSource and the NuGet package. Going by the signatures, I only saw a couple more WriteEvent overloads. If you’ve read the documentation, you should know by now that that’s a good thing, however digging a little into the code I found another important difference – event type support.

The .NET version currently supports the following types (taken from ManifestBuilder.GetTypeName):

  1. Enum
  2. Boolean
  3. SByte
  4. Byte
  5. Int16
  6. UInt16
  7. Int32
  8. UInt32
  9. Int64
  10. UInt64
  11. Single
  12. Double
  13. DateTime
  14. String
  15. Guid

The NuGet package adds support for the following:

  1. Char
  2. IntPtr
  3. Byte*
  4. Byte[]

Finally, the beta pre-relase includes a very interesting overload: public void Write<T>(string eventName, T data). The data parameter is documented as follows:

The object containing the event payload data. The type T must be an anonymous type or a type with an [EventData] attribute. The public instance properties of data will be written recursively to create the fields of the event.

In other words, we should be able to use annotated custom types (in a similar fashion to WCF DataContract-annotated objects). But even better than that, we should be able to say:

public void Bar(int i, string s, DateTime d)
    Write(null, new {I = i, S = s, D = d});

Notice I said should, because I could make neither work (beta after all). But why do we even need this when we already have the WriteEvent(int eventId, params Object[]) overload? Well, let’s look at the documentation of the latter:

By default, the compiler calls this overload if the parameters for the call do not match one of the other method overloads. This overload is much slower than the other overloads, because it does the following:

  1. It allocates an array to hold the variable argument.
  2. It casts each parameter to an object (which causes allocations for primitive types).
  3. It assigns these objects to the array.
  4. It calls the function, which then determines the type of each argument so it can be serialized for ETW.

Using an anonymous types, we save the boxing (bullet 2) and unboxing (post bullet 4). I’m not sure how big of a benefit that is, so we may still be forced to use the WriteEventCore method, but I suppose every little bit helps!