Becoming a Console Programmer : Extending The Watch Window (debugger addin)

So previously we established how to use autoexp.dat for simple format changes and how to use a DLL to achieve those changes. There are major limitations to the DLL approach however in that we cannot access global symbols and thus any system that relies on global state for debug (as many do) cannot be debugged.


Here I will discuss how to create a VS2010 plugin using C# (plugins also work for VS2008 however not using C#).

The Development Tools Environment framework was setup around a decade ago to allow Visual Studio extensibility. I would recommend that any programmers read through what is possible using DTE however i'm not going to invest much time into that here; I will simply use one aspect of it to achieve our goal: global symbol access.

VS2010 has a great Wizard to help us here, Select File->New->Project then on the left hand side go to Other Project Types->Extensibility->Visual Studio Addin, click next and then choose option (default) Create an Add-in using Visual C#, leave application host as default, then give your addin a recognizable name. Set your addin to load when host application starts. The rest is default.

This provides us with a very simple addin with a LOT of C# goodness and very little code. I should note that when i wrote this project it was my first foray into C# and thus my knowledge in this area is shady at best, i had lots of help from experts local to me in setting this up.

If you read through the code you will find some relatively basic functionality. Your new addin has interfaces to intercept callbacks for various VS2010 events, OnConnection is the only default interface i used during which i installed new event handlers.

So lets extend the default a little:

1. Add a member to the class of type "DebuggerEvents"
2. instantiate it to "_applicationObject.Events.DebuggerEvents"
3. setup handles for:
1. OnEnterBreakMode
2. OnEnterRunMode
3. OnEnterDesignMode
4. VS2010 should auto-complete and auto generate these for you but the basics should look like :

   _debugger_events.OnEnterBreakMode += new _dispDebuggerEvents_OnEnterBreakModeEventHandler(DebuggerEvents_OnEnterBreakMode);
   _debugger_events.OnEnterRunMode += new _dispDebuggerEvents_OnEnterRunModeEventHandler(_debugger_events_OnEnterRunMode);
   _debugger_events.OnEnterDesignMode += new _dispDebuggerEvents_OnEnterDesignModeEventHandler(_debugger_events_OnEnterDesignMode);

    
5. For now ignore all but OnEnterBreakMode, this will be called whenever we enter any type of breakmode (stepping, breakpoints, exceptions etc). At this point we can do something quite nifty... any expression you can type into a watch window, can be evaluated here for example

   Debugger debugger = _application_object.Debugger;
   Expression exp = debugger.GetExpression("&g_my_global_foo,d");
   
   if (exp.IsValidValue)
   {
       Int64 value = Convert.ToInt64(exp.Value.ToString(), 10);
       System.Diagnostics.Debug.WriteLine(exp.Value.ToString());
   
       // do other fancy things here
   }

6. At this point its really up to you to choose how you communicate this information to any external process. Personally my first simple solution was old school environment variable. This works perfectly well for a limited project, simply push your symbol address to the environment variable and let any other process (EEAddin for instance) read that environment variable and use it. Should you choose to publish the root address of your global systems for instance one might conceivably achieve full global memory addressing on your target.
7. The OnEnterRunMode hook should be used to "create" any resources you need on events "Attach" and "Launch"
8. The OnEnterDesignMode hook should be used to "destroy" those resources for events "Detach, EndProgram & Stop Debugging"

Debugging the debugger addin isn't difficult but is not obvious. Given the setup (Client Project) using (Debugger Addin) which launches Watch Window DLLS (EEAddin) the debugging method i used is:

1. Open the Debugger Addin project and build/run. This should open up an instance of VS2010 with your Addin loaded (check under Tools->Addin Manager
2. Within this second instance of VS2010 open up your Client Project, Build and run.
3. At this point breakpoints within the Debugger Addin are possible and you should be able to trap the incoming events.

Notes:

• using environment variables will work for very simple projects however for production code i would recommend using process specific memory mapped files or another less hacky method
• VS2008 can be supported however this has to be done as a C++/ATL project and requires a LOT of boilerplate code. Jason Weiler @ Airtight discusses this here and played a large part in enlightening me to this process.
• As previous with EEAddin, this framework is not very forgiving and will require you to write defensively in all aspects. C# does handle a lot of this for you however the C++/ATL version will not. If you are writing an addin framework for a large group of programmers i would suggest significant boilerplate on all code.
• the DTE framework is VERY powerful, anyone using it should read through the docs thoroughly to appreciate everything it can do and share what you do if you can :D

Becoming a Console Programmer : Extending The Watch Window (EEAddin)

So we established that its possible to re-format what a Watch Window shows you, make things look better and therefore easier to consume. What we cannot do with that method tho is any real processing... but we can with a little more work.

consider the example

// blogtest.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <stdio.h>
#include <intrin.h>

enum e_union_test_case
{
    k_unset,
    k_type_string,
    k_type_int,
    k_type_float,
};

struct u_test_level1
{
    int m_type;
    union u_data
    {
        const char *m_string;
        int m_count;
        float m_value;
    }m_data;

    void set(const char *string)
    {
        m_type= k_type_string;
        m_data.m_string= string;
    }

    void set(int int_value)
    {
        m_type= k_type_int;
        m_data.m_count= int_value;
    }

    void set(float float_value)
    {
        m_type= k_type_float;
        m_data.m_value= float_value;
    }

    void print(void)
    {
        if (m_type == k_unset)
        {
            printf("unset\n");
        }

        if (m_type == k_type_string)
        {
            printf("%s \n",m_data.m_string);
        }
        if (m_type == k_type_int)
        {
            printf("%d \n",m_data.m_count);
        }
        if (m_type == k_type_float)
        {
            printf("%f \n",m_data.m_value);
        }
    }
};

struct s_test_level2
{
    static const int k_num_elements= 32;

    u_test_level1 m_elements[k_num_elements];
};


int _tmain(int argc, _TCHAR* argv[])
{
    s_test_level2 test;

    memset(&test,0,sizeof(test));

    test.m_elements[0].set("my foo");
    test.m_elements[1].set(7);
    test.m_elements[2].set(3.14159265358f);    // everyone loves Pi

    return 0;
}

similar to before setup a basic console project and drop in this code. Compile and run to the breakpoint and drop "test" into your watch window opening up the m_elements member... you should see something like.

not very readable at all... the print function shows what we "might" do to display these elements by using "m_type" to change the formatting options...

EEAddin is an option here, (Expression Evaluation Addin). This allows the autoexp.dat [AutoExpand] section to call into a dll for the preview display string. The dll can output any string it desires into the provided char buffer (obeying the limits of course).

you can find an example EEAddin within the VS2010 install, assuming you have the samples extracted (most installs will have a zip file) you will find the solution at

C:\Program Files (x86)\Microsoft Visual Studio 10.0\Samples\1033\VC2010Samples\C++\Debugging\EEaddin

NOTE: the example does not work out of the box, you must edit "ADDIN_API" and make it

#define ADDIN_API __declspec(dllexport)  __stdcall

then move the HRESULT return to the opposite side of the ADDIN_API on use. Optionally you could achieve the same results by altering the project compiler options.

now paste the code

enum e_union_test_case
{
    k_unset,
    k_type_string,
    k_type_int,
    k_type_float,
};

struct u_test_level1
{
    int m_type;
    union u_data
    {
        const char *m_string;
        int m_count;
        float m_value;
    }m_data;

    void print(char *result_buffer, int result_size)
    {
        if (m_type == k_unset)
        {
            sprintf_s(result_buffer, result_size,"unset\n");
        }
        if (m_type == k_type_string)
        {
            sprintf_s(result_buffer, result_size,"str:\n");
        }
        if (m_type == k_type_int)
        {
            sprintf_s(result_buffer, result_size,"int:%d \n",m_data.m_count);
        }
        if (m_type == k_type_float)
        {
            sprintf_s(result_buffer, result_size,"flt:%f \n",m_data.m_value);
        }

    }
};

HRESULT ADDIN_API AddIn_blogtest( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )
{
    DWORD nGot;

    u_test_level1 example;

    // read file time from debuggee memory space
    if (pHelper->ReadDebuggeeMemoryEx(pHelper, pHelper->GetRealAddress(pHelper), sizeof(u_test_level1 ), &example, &nGot) != S_OK)
        return E_FAIL;
    if (nGot != sizeof(u_test_level1))
        return E_FAIL;

    

    example.print(pResult, max);

    return S_OK;
}

Into the existing "timeaddin.cpp" file and add the export to both the header (timeaddin.h) and the def file (eeaddin.def).

Compile this and it will generate EEaddin.dll, copy this dll to your VS2010 IDE folder (C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE).

Add the line

u_test_level1=$ADDIN(eeaddin.dll,AddIn_blogtest)

to your autoexp.dat right below the previous example, below the [AutoExpand] tag.

debug your application... your new display should be

each element now displays using the m_type correctly. We have an issue however, the string is not immediately available. This is due to the EEAddin working in a different memory space than the executable it is accessing. You'll note that the EEaddin DLL reads memory from the application manually using the supplied address AND by dint of the autoexp.dat line we know the size of the type we're representing. We must therefore pull any pointed at string data over manually in order to display it.

the new code is

enum e_union_test_case
{
    k_unset,
    k_type_string,
    k_type_int,
    k_type_float,
};

struct u_test_level1
{
    int m_type;
    union u_data
    {
        const char *m_string;
        int m_count;
        float m_value;
    }m_data;

    void print(char *result_buffer, int result_size)
    {
        if (m_type == k_unset)
        {
            sprintf_s(result_buffer, result_size,"unset\n");
        }
        if (m_type == k_type_string)
        {
            sprintf_s(result_buffer, result_size,"str:%s\n",m_data.m_string);
        }
        if (m_type == k_type_int)
        {
            sprintf_s(result_buffer, result_size,"int:%d \n",m_data.m_count);
        }
        if (m_type == k_type_float)
        {
            sprintf_s(result_buffer, result_size,"flt:%f \n",m_data.m_value);
        }

    }
};

HRESULT ADDIN_API AddIn_blogtest( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )
{
    DWORD nGot;

    u_test_level1 example;

    // read file time from debuggee memory space
    if (pHelper->ReadDebuggeeMemoryEx(pHelper, pHelper->GetRealAddress(pHelper), sizeof(u_test_level1 ), &example, &nGot) != S_OK)
        return E_FAIL;
    if (nGot != sizeof(u_test_level1))
        return E_FAIL;

    static const int local_buffer_size= 1024;
    char local_buffer[local_buffer_size];

    if (example.m_type == k_type_string && example.m_data.m_string != 0)
    {
        // pull over the string to our local buffer
        pHelper->ReadDebuggeeMemoryEx(pHelper, (DWORDLONG)example.m_data.m_string, local_buffer_size, local_buffer, &nGot);

        example.m_data.m_string= local_buffer;
    }

    example.print(pResult, max);

    return S_OK;
}

this code uses the pointer within the original "example" and pulls over an arbitrary length data stream (1kb) relying upon the string being null terminated. It then points our local copy of example.m_data.m_string to that local buffer before calling our new print(...) function which will now output the string as before. Our new display is:

which is exactly what we need.

Notes:

• EEAddin is not forgiving
• if you do something heinous within an EEAddin entrypoint it WILL take out the devenv that loaded it. Within Addins you MUST program defensively at all points, assume you will be sent bad data, assume your strings will not be null terminated, assume that the pointers you need to follow are bad and take you into no mans land. Verify everything. You'll note that i did not do this in my example but that is merely for the purposes of keeping the code samples short and sweet.
  
• Debugging the EEAddin
• load up the EEAddin project, Attach to the instance of devenv (debug it) that your Main project runs in, put breakpoints into your Addin entrypoint then hit debug on your application. Each time your Watch Window updates your EEAddin dll is loaded/called/unloaded so expect a lot of calls in the array case.
• Global Symbols are not accessible
• Accessing client memory requires the physical address of said client memory, if this address isn't accessible from the element you're debugging then you have no way to get to that address. An example of this limitation would be a handle into a global manager. The client code can call the manager to get the object, the debugger can only display the value of the handle, EEAddin can only access the internals OF the handle. It does not have knowledge of the manager and thus it cannot access the object itself.
• Not all Endianess was made equal
• if your target is not the same endianess as PC (little) such as xbox 360 then you must switch the endianess of data before reading it.
• Alignment & Pointer size
• you will have to manually handle both alignment differences (target <=> client) and pointer size differences. In my support thus far i have chosen 2 pathways. Some structures i have made cross platform entirely by supplying a construct that is 64bit and aware of its pointer size. For other types where this isn't possible i have used an element of the next topic to disclose to the DLL the size of our pointer.

Becoming a Console Programmer : Extending The Watch Window (autoexp.dat)

Watch windows are important, they show us our data in various forms and generally enable us to debug effectively, sometimes however they need help.

Setup a simple windows console project and use the following code:

struct s_test_level1
{
    int m_count;
    char* m_string;
    float m_value;

    void print(void)
    {
        if (m_string)
        {
            printf("%s %d %f\n",m_string, m_count, m_value);
        }
    }
};

static const int k_num_elements= 32;
struct s_test_level2
{
    s_test_level1 m_elements[k_num_elements];
};

int _tmain(int argc, _TCHAR* argv[])
{
    s_test_level2 test;

    memset(&test,0,sizeof(test));

    test.m_elements[0].m_string= "my foo";
    test.m_elements[0].m_count= strlen(test.m_elements[0].m_string);
    test.m_elements[0].m_value=3.14159265358f; // everyone loves Pi

    test.m_elements[0].print();

    return 0; // <<= breakpoint here
}

now run the program putting a breakpoint on indicated line. Open up a watch window and drop test into it opening up the m_elements array, you should see something like

This is a really simple but even now we see some details we don't always need... lets reformat it.

open up the file

VS2010 PC: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Packages\Debugger\autoexp.dat
VS2008 PC: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\autoexp.dat
Xbox360: C:\Program Files (x86)\Microsoft Xbox 360 SDK\bin\win32\autoexp.dat for 360

search for "[AutoExpand]" and add the line

s_test_level1=<m_count,d>, <m_string,s>, <m_value,f>f

save that file and re run the test program.... you should now see

a much easier to read version of the same data.

All the original information is still available should you need it but this new format is much easier to quickly consume.

This form of watch window help is somewhat limited however, you can only interpret existing data, remove superfluous detail, do some VERY rudimentary math and generally make things cleaner and easier to consume; see the next post for MORE.

All of the help required to use autoexp.dat is within the file itself. Note that the "[Visualizer]" section is currently not available for xbox 360 targets but is VERY powerful for other targets.

enjoy and feel free to ask questions.

Incoming Topics (what should I blog about next?)

So i've not had much time recently for various reasons and i apologize to those who were following avidly. I should however have some time very soon and wanted to give people the change to decide which topic i talk about... so go for it... comment away (these topics are for x86, x64 and xbox360 tho i cannot directly discuss xbox360 almost all code applies to all 3 targets)

• Extending "Watch Window" Vizualization
  
• Visual Studio debugger plugins
• How to get YOUR object to construct first
• Creating a "Premain" function that runs before global constructors
• Suggestions?