Friday, July 22, 2011

What is an "Engineer Architect"

A question that i get asked rather a lot is "What is an Engineer Architect"

en·gi·neer
noun /ˌenjəˈni(ə)r/ 
A person who designs, builds, or maintains engines, machines, or public works

ar·chi·tect

noun /ˈärkiˌtekt/ 
((computer science) the structure and organization of a computer's hardware or system software) "the architecture of a computer's system software"

This basically translates to the same thing all engineers do:

  • Analyze the problem
  • Formulate a possible set of solutions
  • Analyze the solutions
  • Decide what to implement
  • Implement
  • Test/bug fix
  • Profile/sanitize
  • Repeat until satisfied

however with one important difference. My role involves the entire program, how its parts interact and how they are designed, what their dependencies are and how they affect the final output and the performance of that output. Combine this with data parallel infrastructure, task parallel objects, multiple platforms, myriad hardware limitations and a large programming team ... under many circumstances one might imagine something akin to

and you often wouldn't be too far from the truth on a daily basis. Long term however we (the team) have a plan and a set of goals. Over time we re-assess based on those we hit and those we don't, new requirements and ultimately how the game is progressing and where it needs to go. The role I play in this is simply to ensure that under the stress and strain of day to day development, we're still aiming in the right direction as a whole. That decisions take into account as much of the big picture as possible and weigh that against the immediate requirements of the current goal. There are several people in similar roles at Bungie and our interactions provide a simple but effective method of applying "Checks & Balances" to the progress we make; we each bring our own flavor to the table.

Day to day this involves tasks such as

  • Advising/Teaching on how to handle concurrency
  • Long term interface design
  • Short term prototyping/bug fix hacks
  • Discussing/Advising on future platforms
  • Optimizing programmer iteration, debugging & general workflow
  • Managing external teams
  • Auditioning Middleware
  • designing/writing/debugging infrastructure systems

Technically i'm a member of Bungies "Infrastructure" team. This means that if its something "unexciting", "behind the scenes" or seemingly doesn't affect the final game at all... we handle it. This involves systems like

  • Memory Allocation
  • File System
  • Network Transport
  • Crash Handling, Minidumps
  • Debugger Plugins
  • Multi-threading Infrastructure & Architecture
  • Asset import/baking
  • Math Library
  • Schematization/Reflection
  • Audio Engine
  • Container Classes
  • Profiler Infrastructure
  • Compiler Configuration
  • Low & High level Optimization
  • Build systems
  • Flux Capacitor Maintenance
  • - you still reading?

as i said... the stuff most most programmers find tiresome and boring. Our team love this stuff and we're good at it.

So there you have it, a much better idea of what i do @ Bungie and hopefully a guide to those in school who might want to progress towards a similar role (or avoid it).

Thursday, July 7, 2011

Writing a "Pre-Main" function - Forcing Global Initialization order within VC++

One rule of C/C++ programming that bites everyone eventually is that initialization order of global variables across compilation units is not guaranteed. I've seen programs with global variable dependencies run fine for years then suddenly develop "issues" resulting in a hard lock before main(..) is hit. I've seen other programs have dependency issues but not actually crash resulting in subtle bugs that never seem to negatively affect run-time... until they do, usually on the day before gold master when the lead programmer is in Peru.

So here's the situation, you want to create objects "a" through "g" in various files who register themselves with a manager into a list on construction... a relatively common setup and relatively simple within a single compilation module; for VC++ define them from bottom to top manager first and it will work. However across compilation modules things are not so simple and more often than not the manager will construct itself at a very inopportune time; likely after some objects have registered themselves and before all objects have.

Without simply knowing that the manager will initialize first there really isn't a "Good" solution tho there are several lazy initialization techniques.

So here's where things get "fun" and by "fun" i mean, people look at the code and get confused looks on their face.

Put the following code into a cpp file on its own and add it to your project of choice.

#pragma init_seg( ".CRT$XCB" )

class c_blog_first_class_construction
{
public:
    c_blog_first_class_construction() 
    {
        printf("first class construction\n");
    }

    ~c_blog_first_class_construction() {};
};

static c_blog_first_class_construction blog_first_class_construction;

the constructor for c_blog_first_class_construction will be called before any other constructor in your code (assuming you have nothing else like this in there).

NOTE: Expect C4075: initializers put in unrecognized initialization area, disable it if you feel the need.

the key to the code is
#pragma init_seg(...)

for more info on what this does see kb104248

The crux of it however is we are naming this compilation unit with a specific identifier, when the linker reads various .CRT groups, it combines them in one section and orders them alphabetically. This means that the user-defined global initializers (which the Visual C++ compiler puts in .CRT$XCU) will always come after CRT$XCA and before .CRT$XCZ.

So we are depending linker to do the right thing and insert our section into the CRT sorted sections (verified in VS2010 as of 7/6/2011 for x86,x64,x360). It works and its very useful.

Now if you want to get really fancy... make the constructor for c_blog_first_class_construction call a function that initializes your "Pre construction" requirements engine wide (usually memory management, assert systems).

For other platforms (GCC/SNC are the only others i use) there are other options such as "init_priority" which is a per instance attribute. These can be used to achieve the same result but on a more granular level.

It should be noted that this is not a method that one should use all over a codebase, personally i only ever use it on one compilation module and it is always VERY well commented.

enjoy.

Sunday, May 22, 2011

Becoming a Console Programmer : Extending The Watch Window (debugger addin)

So previously we established how to use autoexp.dat for simple format changes and how to use a DLL to achieve those changes. There are major limitations to the DLL approach however in that we cannot access global symbols and thus any system that relies on global state for debug (as many do) cannot be debugged.


Here I will discuss how to create a VS2010 plugin using C# (plugins also work for VS2008 however not using C#).

The Development Tools Environment framework was setup around a decade ago to allow Visual Studio extensibility. I would recommend that any programmers read through what is possible using DTE however i'm not going to invest much time into that here; I will simply use one aspect of it to achieve our goal: global symbol access.

VS2010 has a great Wizard to help us here, Select File->New->Project then on the left hand side go to Other Project Types->Extensibility->Visual Studio Addin, click next and then choose option (default) Create an Add-in using Visual C#, leave application host as default, then give your addin a recognizable name. Set your addin to load when host application starts. The rest is default.

This provides us with a very simple addin with a LOT of C# goodness and very little code. I should note that when i wrote this project it was my first foray into C# and thus my knowledge in this area is shady at best, i had lots of help from experts local to me in setting this up.

If you read through the code you will find some relatively basic functionality. Your new addin has interfaces to intercept callbacks for various VS2010 events, OnConnection is the only default interface i used during which i installed new event handlers.

So lets extend the default a little:

  1. Add a member to the class of type "DebuggerEvents"
  2. instantiate it to "_applicationObject.Events.DebuggerEvents"
  3. setup handles for:
    1. OnEnterBreakMode
    2. OnEnterRunMode
    3. OnEnterDesignMode
  4. VS2010 should auto-complete and auto generate these for you but the basics should look like :
    _debugger_events.OnEnterBreakMode += new _dispDebuggerEvents_OnEnterBreakModeEventHandler(DebuggerEvents_OnEnterBreakMode);
    _debugger_events.OnEnterRunMode += new _dispDebuggerEvents_OnEnterRunModeEventHandler(_debugger_events_OnEnterRunMode);
    _debugger_events.OnEnterDesignMode += new _dispDebuggerEvents_OnEnterDesignModeEventHandler(_debugger_events_OnEnterDesignMode);
     
  5. For now ignore all but OnEnterBreakMode, this will be called whenever we enter any type of breakmode (stepping, breakpoints, exceptions etc). At this point we can do something quite nifty... any expression you can type into a watch window, can be evaluated here for example
    Debugger debugger = _application_object.Debugger;
    Expression exp = debugger.GetExpression("&g_my_global_foo,d");
    
    if (exp.IsValidValue)
    {
        Int64 value = Convert.ToInt64(exp.Value.ToString(), 10);
        System.Diagnostics.Debug.WriteLine(exp.Value.ToString());
    
        // do other fancy things here
    }
  6. At this point its really up to you to choose how you communicate this information to any external process. Personally my first simple solution was old school environment variable. This works perfectly well for a limited project, simply push your symbol address to the environment variable and let any other process (EEAddin for instance) read that environment variable and use it. Should you choose to publish the root address of your global systems for instance one might conceivably achieve full global memory addressing on your target.
  7. The OnEnterRunMode hook should be used to "create" any resources you need on events "Attach" and "Launch"
  8. The OnEnterDesignMode hook should be used to "destroy" those resources for events "Detach, EndProgram & Stop Debugging"
Debugging the debugger addin isn't difficult but is not obvious. Given the setup (Client Project) using (Debugger Addin) which launches Watch Window DLLS (EEAddin) the debugging method i used is:

  1. Open the Debugger Addin project and build/run. This should open up an instance of VS2010 with your Addin loaded (check under Tools->Addin Manager
  2. Within this second instance of VS2010 open up your Client Project, Build and run.
  3. At this point breakpoints within the Debugger Addin are possible and you should be able to trap the incoming events.


Notes:
  • using environment variables will work for very simple projects however for production code i would recommend using process specific memory mapped files or another less hacky method
  • VS2008 can be supported however this has to be done as a C++/ATL project and requires a LOT of boilerplate code. Jason Weiler @ Airtight discusses this here and played a large part in enlightening me to this process.
  • As previous with EEAddin, this framework is not very forgiving and will require you to write defensively in all aspects. C# does handle a lot of this for you however the C++/ATL version will not. If you are writing an addin framework for a large group of programmers i would suggest significant boilerplate on all code.
  • the DTE framework is VERY powerful, anyone using it should read through the docs thoroughly to appreciate everything it can do and share what you do if you can :D

Saturday, May 21, 2011

Becoming a Console Programmer : Extending The Watch Window (EEAddin)

So we established that its possible to re-format what a Watch Window shows you, make things look better and therefore easier to consume. What we cannot do with that method tho is any real processing... but we can with a little more work.

consider the example

// blogtest.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <stdio.h>
#include <intrin.h>

enum e_union_test_case
{
    k_unset,
    k_type_string,
    k_type_int,
    k_type_float,
};

struct u_test_level1
{
    int m_type;
    union u_data
    {
        const char *m_string;
        int m_count;
        float m_value;
    }m_data;

    void set(const char *string)
    {
        m_type= k_type_string;
        m_data.m_string= string;
    }

    void set(int int_value)
    {
        m_type= k_type_int;
        m_data.m_count= int_value;
    }

    void set(float float_value)
    {
        m_type= k_type_float;
        m_data.m_value= float_value;
    }

    void print(void)
    {
        if (m_type == k_unset)
        {
            printf("unset\n");
        }

        if (m_type == k_type_string)
        {
            printf("%s \n",m_data.m_string);
        }
        if (m_type == k_type_int)
        {
            printf("%d \n",m_data.m_count);
        }
        if (m_type == k_type_float)
        {
            printf("%f \n",m_data.m_value);
        }
    }
};

struct s_test_level2
{
    static const int k_num_elements= 32;

    u_test_level1 m_elements[k_num_elements];
};


int _tmain(int argc, _TCHAR* argv[])
{
    s_test_level2 test;

    memset(&test,0,sizeof(test));

    test.m_elements[0].set("my foo");
    test.m_elements[1].set(7);
    test.m_elements[2].set(3.14159265358f);    // everyone loves Pi

    return 0;
}


similar to before setup a basic console project and drop in this code. Compile and run to the breakpoint and drop "test" into your watch window opening up the m_elements member... you should see something like.

not very readable at all... the print function shows what we "might" do to display these elements by using "m_type" to change the formatting options...

EEAddin is an option here, (Expression Evaluation Addin). This allows the autoexp.dat [AutoExpand] section to call into a dll for the preview display string. The dll can output any string it desires into the provided char buffer (obeying the limits of course).

you can find an example EEAddin within the VS2010 install, assuming you have the samples extracted (most installs will have a zip file) you will find the solution at

C:\Program Files (x86)\Microsoft Visual Studio 10.0\Samples\1033\VC2010Samples\C++\Debugging\EEaddin

NOTE: the example does not work out of the box, you must edit "ADDIN_API" and make it
#define ADDIN_API __declspec(dllexport)  __stdcall

then move the HRESULT return to the opposite side of the ADDIN_API on use. Optionally you could achieve the same results by altering the project compiler options.
now paste the code


enum e_union_test_case
{
    k_unset,
    k_type_string,
    k_type_int,
    k_type_float,
};

struct u_test_level1
{
    int m_type;
    union u_data
    {
        const char *m_string;
        int m_count;
        float m_value;
    }m_data;

    void print(char *result_buffer, int result_size)
    {
        if (m_type == k_unset)
        {
            sprintf_s(result_buffer, result_size,"unset\n");
        }
        if (m_type == k_type_string)
        {
            sprintf_s(result_buffer, result_size,"str:\n");
        }
        if (m_type == k_type_int)
        {
            sprintf_s(result_buffer, result_size,"int:%d \n",m_data.m_count);
        }
        if (m_type == k_type_float)
        {
            sprintf_s(result_buffer, result_size,"flt:%f \n",m_data.m_value);
        }

    }
};

HRESULT ADDIN_API AddIn_blogtest( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )
{
    DWORD nGot;

    u_test_level1 example;

    // read file time from debuggee memory space
    if (pHelper->ReadDebuggeeMemoryEx(pHelper, pHelper->GetRealAddress(pHelper), sizeof(u_test_level1 ), &example, &nGot) != S_OK)
        return E_FAIL;
    if (nGot != sizeof(u_test_level1))
        return E_FAIL;

    

    example.print(pResult, max);

    return S_OK;
}



Into the existing "timeaddin.cpp" file and add the export to both the header (timeaddin.h) and the def file (eeaddin.def).

Compile this and it will generate EEaddin.dll, copy this dll to your VS2010 IDE folder (C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE).

Add the line

u_test_level1=$ADDIN(eeaddin.dll,AddIn_blogtest)

to your autoexp.dat right below the previous example, below the [AutoExpand] tag.

debug your application... your new display should be

each element now displays using the m_type correctly. We have an issue however, the string is not immediately available. This is due to the EEAddin working in a different memory space than the executable it is accessing. You'll note that the EEaddin DLL reads memory from the application manually using the supplied address AND by dint of the autoexp.dat line we know the size of the type we're representing. We must therefore pull any pointed at string data over manually in order to display it.

the new code is


enum e_union_test_case
{
    k_unset,
    k_type_string,
    k_type_int,
    k_type_float,
};

struct u_test_level1
{
    int m_type;
    union u_data
    {
        const char *m_string;
        int m_count;
        float m_value;
    }m_data;

    void print(char *result_buffer, int result_size)
    {
        if (m_type == k_unset)
        {
            sprintf_s(result_buffer, result_size,"unset\n");
        }
        if (m_type == k_type_string)
        {
            sprintf_s(result_buffer, result_size,"str:%s\n",m_data.m_string);
        }
        if (m_type == k_type_int)
        {
            sprintf_s(result_buffer, result_size,"int:%d \n",m_data.m_count);
        }
        if (m_type == k_type_float)
        {
            sprintf_s(result_buffer, result_size,"flt:%f \n",m_data.m_value);
        }

    }
};

HRESULT ADDIN_API AddIn_blogtest( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )
{
    DWORD nGot;

    u_test_level1 example;

    // read file time from debuggee memory space
    if (pHelper->ReadDebuggeeMemoryEx(pHelper, pHelper->GetRealAddress(pHelper), sizeof(u_test_level1 ), &example, &nGot) != S_OK)
        return E_FAIL;
    if (nGot != sizeof(u_test_level1))
        return E_FAIL;

    static const int local_buffer_size= 1024;
    char local_buffer[local_buffer_size];

    if (example.m_type == k_type_string && example.m_data.m_string != 0)
    {
        // pull over the string to our local buffer
        pHelper->ReadDebuggeeMemoryEx(pHelper, (DWORDLONG)example.m_data.m_string, local_buffer_size, local_buffer, &nGot);

        example.m_data.m_string= local_buffer;
    }

    example.print(pResult, max);

    return S_OK;
}


this code uses the pointer within the original "example" and pulls over an arbitrary length data stream (1kb) relying upon the string being null terminated. It then points our local copy of example.m_data.m_string to that local buffer before calling our new print(...) function which will now output the string as before. Our new display is:

which is exactly what we need.

Notes:
  • EEAddin is not forgiving
    • if you do something heinous within an EEAddin entrypoint it WILL take out the devenv that loaded it. Within Addins you MUST program defensively at all points, assume you will be sent bad data, assume your strings will not be null terminated, assume that the pointers you need to follow are bad and take you into no mans land. Verify everything. You'll note that i did not do this in my example but that is merely for the purposes of keeping the code samples short and sweet.
  • Debugging the EEAddin
    • load up the EEAddin project, Attach to the instance of devenv (debug it) that your Main project runs in, put breakpoints into your Addin entrypoint then hit debug on your application. Each time your Watch Window updates your EEAddin dll is loaded/called/unloaded so expect a lot of calls in the array case.
  • Global Symbols are not accessible
    • Accessing client memory requires the physical address of said client memory, if this address isn't accessible from the element you're debugging then you have no way to get to that address. An example of this limitation would be a handle into a global manager. The client code can call the manager to get the object, the debugger can only display the value of the handle, EEAddin can only access the internals OF the handle. It does not have knowledge of the manager and thus it cannot access the object itself.
  • Not all Endianess was made equal
    • if your target is not the same endianess as PC (little) such as xbox 360 then you must switch the endianess of data before reading it.
  • Alignment & Pointer size
    • you will have to manually handle both alignment differences (target <=> client) and pointer size differences. In my support thus far i have chosen 2 pathways. Some structures i have made cross platform entirely by supplying a construct that is 64bit and aware of its pointer size. For other types where this isn't possible i have used an element of the next topic to disclose to the DLL the size of our pointer.

Becoming a Console Programmer : Extending The Watch Window (autoexp.dat)

Watch windows are important, they show us our data in various forms and generally enable us to debug effectively, sometimes however they need help.

Setup a simple windows console project and use the following code:
struct s_test_level1
{
    int m_count;
    char* m_string;
    float m_value;

    void print(void)
    {
        if (m_string)
        {
            printf("%s %d %f\n",m_string, m_count, m_value);
        }
    }
};

static const int k_num_elements= 32;
struct s_test_level2
{
    s_test_level1 m_elements[k_num_elements];
};

int _tmain(int argc, _TCHAR* argv[])
{
    s_test_level2 test;

    memset(&amp;test,0,sizeof(test));

    test.m_elements[0].m_string= "my foo";
    test.m_elements[0].m_count= strlen(test.m_elements[0].m_string);
    test.m_elements[0].m_value=3.14159265358f; // everyone loves Pi

    test.m_elements[0].print();

    return 0; // &lt;&lt;= breakpoint here
}

now run the program putting a breakpoint on indicated line. Open up a watch window and drop test into it opening up the m_elements array, you should see something like


This is a really simple but even now we see some details we don't always need... lets reformat it.

open up the file

VS2010 PC: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Packages\Debugger\autoexp.dat
VS2008 PC: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\autoexp.dat
Xbox360: C:\Program Files (x86)\Microsoft Xbox 360 SDK\bin\win32\autoexp.dat for 360

search for "[AutoExpand]" and add the line

s_test_level1=<m_count,d>, <m_string,s>, <m_value,f>f

save that file and re run the test program.... you should now see



a much easier to read version of the same data.

All the original information is still available should you need it but this new format is much easier to quickly consume.

This form of watch window help is somewhat limited however, you can only interpret existing data, remove superfluous detail, do some VERY rudimentary math and generally make things cleaner and easier to consume; see the next post for MORE.

All of the help required to use autoexp.dat is within the file itself. Note that the "[Visualizer]" section is currently not available for xbox 360 targets but is VERY powerful for other targets.

enjoy and feel free to ask questions.

Friday, May 20, 2011

Incoming Topics (what should I blog about next?)

So i've not had much time recently for various reasons and i apologize to those who were following avidly. I should however have some time very soon and wanted to give people the change to decide which topic i talk about... so go for it... comment away (these topics are for x86, x64 and xbox360 tho i cannot directly discuss xbox360 almost all code applies to all 3 targets)

  • Extending "Watch Window" Vizualization
  • Visual Studio debugger plugins
  • How to get YOUR object to construct first
  • Creating a "Premain" function that runs before global constructors
  • Suggestions?