The demise of the "Low Level" programmer.

[updated 8/9/2011]

When I started programming many of the elements we take for granted now, did not exist. There was no DirectX and not many compatible libs were available for the free compilers of the day. So I had to write my own code for most basic programs, keyboard handlers, mouse handlers, video memory accessors, rasterizers, texture mappers, blitters… the programs I wrote then were 100% my own code and I had to be able to handle anything and everything.
Personally I’ve always been interested in what was going on under the hood so this suited me just fine. I always dug into the details and I almost always end up programming as close to the bone ON the hardware (or OS) as I possibly can both to eek out as much performance as possible AND to satisfy my own hunger for knowledge.
Combine the two and what you get today is someone who enjoys spending 5 days making that single function 20x faster, who revels in reducing the memory footprint of the primary data structure by 1 byte per element across the entire program whilst simultaneously writing a pre-caching system to avoid the special case issues…. who… well you get the picture… I’m an OCD level sport optimizing geek. To such a degree that I keep a notepad by my bed to take notes when I wake up from a “bug fix” dream or a “eureka optimization” doze…yes i’m that sick.
Over the last decade I’ve been involved in the hiring process at many studios and in more recent years I’ve noticed a pattern. Knowledge of what is generally considered “low-level” programming is waning. Many programmers know enough to get through a C# or C++ test, but don’t understand something as basic (and important) as the behavior of memory or god forbid a cache. They don’t seem to grasp that one must understand the native environment you’re working in before going ahead and writing a program to run within it. The intricacies of floating point vs fixed point math are completely lost on them as the term “fixed point” brings about a blank stare; floating point numbers are best right?. I once mentioned bit shifting to an experienced engineer of 10 years and was devastated by the complete lack of basic understanding.
It depresses me that so much of what I consider to be essential is simply not being taught anymore. I’m not talking about assembly language per se; even those of us who used to spend hours writing assembly now more often opt to use intrinsics built into compilers to avoid the stress and complication. What I’m talking about is simply the understanding of WHAT is happening when someone does i++ and not ++i, why one might opt to stripe a memory copy/set in certain circumstances.
So here goes… a list of things I believe all console programmers (and recommend to all programmers as good reading) should fully understand with links to educate where possible. (feel free to suggest more/better links)

• Floating Point Numbers
• They are very useful but often used in situations where they simply don’t suit the solution the programmer is attempting to write. The following links should provide some background and info on where they are not so useful, what the pitfalls are and sometimes even how to avoid them.
• http://www.cprogramming.com/tutorial.html#fptutorial
• http://www.johndcook.com/blog/2009/04/06/numbers-are-a-leaky-abstraction/
• http://www.codeproject.com/KB/recipes/float_point.aspx
• http://drdobbs.com/184402741?pgno=4
• http://users.tkk.fi/jhi/infnan.html
• Fixed Point Numbers
• Fixed point math is mildly old school but it is VERY useful both to understand its makeup and to use. Sadly because it is considered old school many of the online sources are out of date.
• http://x86asm.net/articles/fixed-point-arithmetic-and-tricks/
• http://gameprogrammer.com/4-fixed.html
• Processor Cache Behavior / Memory
•  http://www.akkadia.org/drepper/cpumemory.pdf
•  http://en.wikipedia.org/wiki/CPU_cache
• http://igoro.com/archive/gallery-of-processor-cache-effects/
• Bit Shifting
• http://www.cprogramming.com/tutorial/bitwise_operators.html
• useful hacks (use carefully)
• http://graphics.stanford.edu/~seander/bithacks.html
• http://stackoverflow.com/questions/539836/emulating-variable-bit-shift-using-only-constant-shifts
• http://guru.multimedia.cx/avoiding-branchesifconditionals/
• Branch Prediction
• This may be lower level that people think they need to go… but they’d be wrong. Understanding how the hardware you’re programming for treats branches can affect performance to a HUGE degree… far more than most programmers may appreciate re: death by a thousand cuts
• http://cellperformance.beyond3d.com/articles/2006/04/background-on-branching.html
• http://igoro.com/archive/fast-and-slow-if-statements-branch-prediction-in-modern-processors/
•  http://www.k8gu.com/ece.umn.edu/documents/classes/ece362-branch-prediction.pdf
•  http://www.cs.ucr.edu/~gupta/teaching/203A-09/My6.pdf
• Sorting
• This isn’t really low-level but something i consider “basic” and its an area where many programmers are simply lacking in understanding. Do yourself a favor and play around with this link, read the links it sends you to for each algorithm and try to grasp when each might be used, the properties as described and the next time you need to sort something… consult it.
• http://www.sorting-algorithms.com/
• another good link with sub links: http://corte.si//posts/code/visualisingsorting/index.html
• a funny one (but still strangely useful), the bubble sort dance:  http://www.youtube.com/watch?v=lyZQPjUT5B4

So there you go, I’ve likely missed some aspects that should make the list but if you can grasp the above then you’re more likely to get noticed at interviews and you’ll certainly be a better programmer for it.

Edit 8/8/2011 – It seems I didn’t do a very good job at explaining the core audience for the article, I apologize for that. The programmers I would like to see learning and absorbing these “low level” details are those who would see themselves working in console games. The HW is fixed for long periods of the time and resources become scarce usually within 1 game cycle; understanding how to better utilize resources becomes a key part of the job.
While i believe that most programmers would benefit from understanding more about the medium they work in it is certainly not required
Hopefully you enjoyed the article in the spirit it was intended.
Andy Firth

Working with Instant Messenger.

Personally i could not work without instant messenger (IM) in all of its guises. I use trillian on my main dev machine to talk to friends, colleagues and peers around the world. I do it without thinking and i do it without negatively affecting my work; it is a tool of life and work. I use it to discuss tech within the game/engine, to setup meetings, to discuss scheduling issues, to setup tests... essentially anything work related (and some not work related things).

Recently i did a poll on a well populated games dev only forum. I started the poll based entirely on a single conversation with a friend at R* San Diego who said that R* do not allow IM usage. After a few hours it became apparent that around 10% of developers do not allow IM usage in any form, the other 90% largely given open access.

So why is it some companies simply don't allow it?

It seems that IM usage is a worry to the powers that be who think maybe details that should not be discussed outside of work might be discussed over IM which could indeed be considered a security leak if the receipient is not employed by said company, on the other hand what about internal only IM. I would ask, how likely is it that someone would spend the time (and money) to infiltrate another persons messenger account in order to log the conversations that would require that person to be logged on in the first place. I don't know how easy/difficult this is (and maybe my opinion would change if i knew) but it seems very unlikely that this is happening and thus i question the decision to remove such a useful tool from developers.

I also heard some rumblings that certain companies believe IM is a distraction and they need to remove any and all of those lest the peons get too engrossed in something other than work. Personally i think that people who use IM as a distraction at the expense of their work will likely find another distraction if / when IM is removed.

In the past i've worked for Activision and EA, two of the largest publishers, both of whome encourage IM usage. EA has an SIP setup internal to each studio that provides access to ALL. ATVI don't actively help people setup IM but they do not discourage it either, many of the studio level heads use it. It is a useful element of everyday developer life.

So, my opinions are known... what are yours?

First!!

So I've been reading blogs for a while now of various people and today a friend of mine asked if i had one... upon hearing i didn't he went ahead (with my permission) and published on his blog some of my thoughts and helpful hints regarding Vista64... I had always thought blogs took a long time however his post took about 30 seconds longer than it took for me to tell him my info...

so i figured its high time i made one myself... hopefully i post something useful here soon.