Certification? Please, no.

I want to respond to this post over at Raganwald. Rather than trying to fit some overarching scheme together, it’s just going to be a collection of random vignettes and rants. Much easier to write.

First- if you have twenty years of experience, heck, if you have five years of experience, and a prospective employer is wondering where you got your degree, flee in terror. That is not a place you want to work- unless you’re applying for a job in HR. Five years of experience in the industry is a much better judge of what you are or are not capable of than the fanciest- or plainest, or even lack of- degree. Some of the best programmers I know have never quite managed to finish that bachelor’s yet. And some of the worst programmers I know have PhDs in Comp Sci. I should also mention that some of the other best programmers I know have PhDs in Comp Sci, and some of the other worst programmers I know don’t have bachelor’s yet either. In fact, my experience has been that advanced degrees have little correlation, positive or negative, on quality of the programmer.

The other correlation I have noticed is that the “commercial” certifications do have a strong correlation with competence- a negative one. “MCSE” stands for “Must Consult Someone Experienced”.

The problem, I think, is one of culture. Go back in time to, say, the 50’s or 60’s. Back then you had two clear social tiers. You had the white collar workers- who wore suits and ties and shined shoes, drank wine or martinis, worked in nice, clean, airconditioned offices and made a lot of money and managed people. And you had blue collar workers, who wore jeans and tshirts and boots, drank beer, worked in ugly, uncomfortable factories or outside, made less money, and generally managed (moved, manipulated) things. Even the socialization with coworkers rules were different- blue collar social structures were generally much flatter, and after-work socialization was common (“the whole line, even the line foreman, would go out for a beer after work”). White collar society had a much stricter social heirarchy.

And the nice thing about blue collar workers, from the point of view of white collar workers, was that they were interchangeable and expendable. They were generally very easy to fire and replace. Don’t like the way the guy on line 3 is looking at you? He’s gone, bring in the next guy. This was even more pronounced back in the late 19th, early 20th century when these castes were developing, before Unions got going in a major way. Back when Henry Ford was a model of sensitive, caring management, because his hired goons only beat up union organizers and rabble rousers, instead of killing them.

Now we have this weird third class- the informational technical class. In many ways they have the social markers of being blue collar- they wear jeans and tshirts and boots (“just in case a mountain should crop up in the middle of the server room”), drink beer, and generally socialize like blue collar folks. On the other hand, they work in clean, comfortable offices and make a lot of money, like the white collar people. This new class is neither fish nor fowl, and I think there is a definate pushback. In the 80’s, there was an attempt to make this new class true white collar. Starting with you need to wear a suit and a tie to work, like a good little white collar drone. That didn’t last long.

So now I think there is a push in the other direction. Fine, if you won’t be forced to be white collar, you get to be forced to be blue collar. Starting with being replaceable. This, I think, is the great attraction certifications like MCSE has to those of a certain mindset- it brings with it an illusion of replaceability. Don’t like the way that programmer is looking at you? He’s gone. Bring in the next certified developer.

It doesn’t actually work that way, for reasons that are obvious to anyone who has been in the industry for more than about two weeks. But there is still the attraction there. And I don’t think any serious discussion of certification can be done without dealing with this issue- any movement to certify developers will be co-opted by the “old gaurd white collar” people and turned into a means to reinforce the desired social order.

That said, I think the beer and popcorn discussion of “if I were to design a programmer certification test” is still interesting (especially if it doesn’t lead to actually trying to implement the certification)- as what the topic really is, is “this is what I think is central to programming.”

Which allows me to segue back into discussing the meat of Raganwald’s post.

His certification test would cover (generalizing a little bit here):

• testing


• testing


• testing


• design for testing


• testing


• testing tools


• testing metrics


• testing methodologies

Um, OK. If you’re certifing a QA expert. But I thought we were certifing software developers.

Or, let me rephrase the question. What makes a good software developer? Testing? No.

I used to have a compsci teacher (I think it was compsci AP back in high school), who, whenever I was wrestling with some particularly hard bug, would ask me “if you didn’t mean the program to do that, why did you program it to do that?” At the time, I mainly wanted to beat him over the head with a chair when he said that, but with several decades of experience behind me, I now understand what he was trying to say- that the best way to prevent bugs is to don’t create them in the first place. And that testing is generally not the way you do this. To use Raganwald’s own analogy, an emphasis on testing for software developers is like an emphasis on cooks being able to recognize and treat anaphylactic shock. Professional QA people are not unlike the food tasters the kings of old days used to hire. While accidentally poisoning the king’s food taster was less of a big deal than accidentally poisoning the king, it was still a big deal.

By the time a bug hits testing, it’s a big deal- and the later in the testing cycle it is, the bigger the deal it is. I notice that all the developers who are all about waterfall design being a bad idea are all in favor of waterfall testing. With the exception of unit testing in test-infected cultures, testing is something that happens after the software is more or less “done”. Because the more extensive the testing is, the more expensive it is.

And you can’t make the argument that testing makes for better programmers. Programmers know that their code will be tested at some point or another- as the true final level of testing takes place when people use your program. And most programmers do not really want to write broken code- they just don’t know how not to. So they adopt a dangerous fatalism- bugs are going to happen anyways, testing (either unit testing, QA testing, or user “testing”) will discover them anyways, so why fight fate? Or as the old lady said, if it’s going to happen anyways, lie back and enjoy it.

Which brings up another point. Everything I learned in college, I needed to know in the real world. But not everything I needed to know in the real world I learned in college. And all the best programmers I know of have at least most of a bachelor’s degree. A compsci degree is not the worthless thing most people seem to assume it is. Heck, most of the theory I’ve learned, I’ve learned post-college. The problem, I think, is one of time. At least back in the 80’s, CS departments had to assume people entering the college had never used a computer before, let alone programmed anything before. So first they had to teach you how to long on, move around, edit files, etc. And then teach you introductory to programming, and basic theory and practice. Very basic, because there is only so much time.

Imagine, if you will, if the English department had to assume that students entering the program had never seen a book before, and did not know how to read. And that they had to first spend three years teaching the students to read and write, and basic grammar, before in their last year they could read “Hamlet” and “Moby Dick”.

The reason I bring this rant up is that he’s missed the whole point of static typing. Especially in the context of producing working, usefull code. But this isn’t surprising, given Raganwald’s “fair and balanced” look at static vr.s dynamic typing. And, just like that “fair and balanced” cable channel, the participants of the debate are carefully choosen to create an predetermined winner. I know this because the debate was between C/C++/Pascal (implicitly)/Java/C# on one side, and the dynamic languages like Ruby and Python on the other. Which makes the debate about as legitimate as the Hannity and Colmes show. Weirdos from Air America need not apply.

You probably have guessed (if you know me and/or if you’ve read this blog) that I’m going to bring up Ocaml here, and I am. Because static typing slots right into the code correctness argument. It wasn’t until OO programming became popular and heavily used that gang-of-four design patterns came to be understood- and it wasn’t until I started working in an environment of professional Ocaml programmers that I really started to understand the design patterns that real static typing allows/encourages you to use.

For example, which of the following program constraints can be enforced by Ocaml’s static type system:

• That a given variable or parameter must be an int, and not a char or string or something


• That all code paths of a function must return the same type.


• That all possible variants of a type must be dealt with- for example, that a list has no elements or at least one element must both be handled.


• That the read_async function only takes file handles opened with the async flag set, while the close function can take file handles with the async flag set or not set.


• That the buffer returned from read_async can not be accessed until the asynchronous read is completed.


• That a Postgresql cursor can only be created and accessed with a transaction.

This is, of course, a trick question- the right answer is “all of them” (the third from the last- read_async vr.s close- is handled by phantom types, the last two by monads. I’ll cover this in a later post).

The mantra here at Jane Street is “make illegal states impossible to represent”. Or, to quote my old CS teacher, “if you don’t mean it do that, don’t make it possible to do that!” And more so than any testing, even unit testing, static types are on the front line for doing that.

So what do I think makes a great software developer? I’m still not sure- I’m still answering that question myself. One thing I have realized recently. And that is that our job is not to produce working code. It’s a heretical idea, but it really isn’t. Our job, as software developers, is to solve problems. One (admittedly common) way of doing that is writting some code. But an equally valid way would be to buy a software package which already solves the problem, or figuring out some way for the company to just avoid the problem in the first place. Testing, and static typing, are about producing working code. But a focus on that level misses the wider context the truely great programmers work in- and the problems and aspects of that wider context.

Such as figuring out what the real problem is in the first place. That one I’m still getting a handle on. And goes solidly into the list of things college did not teach me about. And fits on no test I can envision, except real life.