Sunday, December 14, 2008

The demise of the "Low Level" programmer.

[updated 8/9/2011]

When I started programming many of the elements we take for granted now, did not exist. There was no DirectX and not many compatible libs were available for the free compilers of the day. So I had to write my own code for most basic programs, keyboard handlers, mouse handlers, video memory accessors, rasterizers, texture mappers, blitters… the programs I wrote then were 100% my own code and I had to be able to handle anything and everything.
Personally I’ve always been interested in what was going on under the hood so this suited me just fine. I always dug into the details and I almost always end up programming as close to the bone ON the hardware (or OS) as I possibly can both to eek out as much performance as possible AND to satisfy my own hunger for knowledge.
Combine the two and what you get today is someone who enjoys spending 5 days making that single function 20x faster, who revels in reducing the memory footprint of the primary data structure by 1 byte per element across the entire program whilst simultaneously writing a pre-caching system to avoid the special case issues…. who… well you get the picture… I’m an OCD level sport optimizing geek. To such a degree that I keep a notepad by my bed to take notes when I wake up from a “bug fix” dream or a “eureka optimization” doze…yes i’m that sick.
Over the last decade I’ve been involved in the hiring process at many studios and in more recent years I’ve noticed a pattern. Knowledge of what is generally considered “low-level” programming is waning. Many programmers know enough to get through a C# or C++ test, but don’t understand something as basic (and important) as the behavior of memory or god forbid a cache. They don’t seem to grasp that one must understand the native environment you’re working in before going ahead and writing a program to run within it. The intricacies of floating point vs fixed point math are completely lost on them as the term “fixed point” brings about a blank stare; floating point numbers are best right?. I once mentioned bit shifting to an experienced engineer of 10 years and was devastated by the complete lack of basic understanding.
It depresses me that so much of what I consider to be essential is simply not being taught anymore. I’m not talking about assembly language per se; even those of us who used to spend hours writing assembly now more often opt to use intrinsics built into compilers to avoid the stress and complication. What I’m talking about is simply the understanding of WHAT is happening when someone does i++ and not ++i, why one might opt to stripe a memory copy/set in certain circumstances.
So here goes… a list of things I believe all console programmers (and recommend to all programmers as good reading) should fully understand with links to educate where possible. (feel free to suggest more/better links)
So there you go, I’ve likely missed some aspects that should make the list but if you can grasp the above then you’re more likely to get noticed at interviews and you’ll certainly be a better programmer for it.

Edit 8/8/2011 – It seems I didn’t do a very good job at explaining the core audience for the article, I apologize for that. The programmers I would like to see learning and absorbing these “low level” details are those who would see themselves working in console games. The HW is fixed for long periods of the time and resources become scarce usually within 1 game cycle; understanding how to better utilize resources becomes a key part of the job.
While i believe that most programmers would benefit from understanding more about the medium they work in it is certainly not required
Hopefully you enjoyed the article in the spirit it was intended.
Andy Firth


Wolfgang Engel said...

Hey Andy, what about some links on what we all can do to learn the low-level stuff?

Rich Fife said...

What ticks me off is when an interviewee becomes visibly angry when you ask them a low level question. As if you're some sort of old coot bringing in irrelevant crap.

Andy Firth said...

While we've not had someone get mad per se... we have seen irritation that we might dare ask questions about bit shifting or cache behavior

Byron Atkinson-Jones said...

I recently had this kind of issue when trying to explain to a very high level coder why they should be careful of the size of their classes so that they fall neatly into d-cache lines and I found myself having to explain what a cache was.

The most worrying thing is the look of disbelief that any low level optimization can have any impact on the speed of the code and then the back-tracking a few days later when they have run out of options and try your suggestions and find they work. It's happening far too often now and I dread the day when we hit a really nasty bug and the dev has to go to the assembly view to see what the compiler produced.

Rich Fife said...
This comment has been removed by the author.
hcpizzi said...

I don't think every programmer should have that knowledge, specially nowadays when all games use some kind of script language. But obviously those who deal with performance critical code should know what's going on at the low level whenever you're writing C++ code (both in terms of memory layout and machine code generated)and its implications in the target architecture. But I think that the focus will shift towards higher level, algorithmic and data structure optimizations, that are scalable in today's (and tomorrow's) landscape of ever increasing core counts. Some of the knowledge still applies, of course, and I always say that the more you know, the better. Just my 2c :)

Andy Firth said...

while i agree with your comments regarding someone using a scripting language to program say direct gameplay, it has to be said that these days almost ALL code is performance critical. Some is obviously MORE critical that others but the point still holds... Accessing an array of data our of order or even worse, backwards/interleaved, is a very bad thing for modern CPU's and without said "knowledge" a programmer would not know why... (this is an example directly from our test), their code could run much much faster and the limits their original code placed on its users could be much more forgiving.

I've seen games massively limited on say "actor" pathing due to simple data access order algorithmic issues; purely cache related. The programmer didn't "get" the cache was the issue and spent all of his time optimizing math routines which didn't help at all...

Andy Firth said...

sorry Wolf I keep meaning to come back to this and post some links up but we are about to hit alpha and its all hectic; you know the deal.

namar said...

This is an undeniable truth, and i solely believe, that if you are to have a job title with "programmer" in it, this is mandatory.

I'm a new breed, and realize that many from this new generation are not taking programming to that "religious" level and it is sad. Again, might i remind that these are programmers, and that it is implied in their "pro-fession" to be knowledgeable. And yes, even the script programmer needs to know what's happening. For if we start opening the door of, what's important and what's not, then where's the end!

Sometimes in life, better keep a tight leash than an open one.

Speaking for myself, i'm versed and continue to learn, by tackling those small projects, and pushing them to the limit. As Andy, spending time with a function might seem trivial but the knowledge gained, will make the difference between a programmer and "The" coder.