Friday, June 26, 2009

A Mind is a Terrible Thing to Waste

I have just seen a presentation about Agilists and Architects, which resonated with an old thought of mine: the waste of experience in the software development process.

Writing software – programming – is something that takes a huge amount of experience to get right. You have to know the language, of course. You have to know it well enough to avoid getting things wrong because of an unknown feature or characteristic of the language. You have to know the language libraries, frameworks, and development tools. This is the very basic.

You have to know patterns, and you have to know anti-patterns too. You need experience to realize that a pattern apply to a problem, or that some solution is inching toward an anti-pattern.

You have to know about security, about protocols, about distributed systems, about parallel systems, about algorithms – of so many types I can’t even list them here. You have to know about databases, persistence, XML, MVC, DSL, regex, grammars, state machines… we could go on and on. You have to know usability.

You have to know about programming paradigms. That’s easy, there’s only four or five of them. Or maybe more, depending on who’s definition you are using. And, of course, each paradigm has many variations preferred by one language or another – and who has to deal with a single language these days?

You have to know computers, and experience here will bring you understanding of performance issues. Experience will also let you identify bugs, or avoid them altogether.

Then, of course, you have to know CMMI, PMBoK, FPA or any of dozens standards and best practices and processes for software development, quality, testing, team working, etc.

Finally, of course, you have to know how to communicate with other people, understand the business needs, know the existing systems, the explicit and implicit rules, and, if possible, why it exists in the state it is, why certain decisions were taken, what were the conditions that might change and make it interesting – or even necessary – to take another path, or to avoid certain paths.

Each of those things you know or don’t have a direct impact on the quality of the code you write. Sure, you can write software with only a small glimpse of all this knowledge, but many times you would have written better, more efficient and more maintainable code if you only knew there was a word to what you were doing. Whole classes might have crumbled to a single library call. You might have avoided code which is intrinsically insecure, protocols which are unfixable. Hell, you might have avoided trying to solve problems known to be unsolvable.

While deep knowledge is clearly the domain of experts in a field, all those domains come together through the hands of the generalist, who has to know something exists before he can use it.

So there are architects, who are meant to have all that experience, and lay down the trail for those inexperienced to follow. Only, of course, it doesn’t work that way. The sheer level of detail required in developing software preclude architects from being able to know or influence but a thin, though important, layer of all there is.

It couldn’t be other way, in truth – if it were possible to write a precise specification of what the software should do, what should be its structure, how it should go about doing what it is supposed to do, well, if it were possible to write such a specification, then we could feed it to the computer and be done with it. In fact, we can write those specifications – they are called “source code”. Source code is not what a computer executes. It’s a specification detailing how to produce the instructions the computer will follow to provide the service required.

In short, or, rather, in conclusion, programming experience results in code with fewer bugs, that satisfy more completely the users needs, easier to adapt to new needs, and in a shorter time.

So why the hell does any company even contemplates hiring “junior” programmers? No, let me rephrase that. Why is the average experience of programmers working – coding – for corporations so low? There is an obvious – to me – inversion of value going on there. And looking at companies that make software to be sold as service or at retail – as opposed to producing software to someone else’s specs – I don’t see much disagreement.

I can see a few reasons why that might be true, of course. First, just because someone acquires experience doesn’t mean he or she learns new stuff. And the transition from mainframe to personal computing pretty much proved that can be detrimental – when people cling to what they know against the unknown, instead of avoiding what they know to be wrong, or taking the better one of known alternatives.

So, corporations “solve” the problem by making experienced people into architects and managers. Being a manager has a quality of its own, and having experience writing software does not really add anything to it. Being an architect takes you too far from where you can make most use of your experience.

Hopefully, if you are lucky enough to work somewhere software architects have meaningful influence on the software development process, you might get programmers to avoid huge mistakes. And if your corporation is really lucky, you’ll do so without critically impairing the software development process with decisions detached from the reality of the problems being faced.

I was happy, then, with the ideas presented by Rebecca Parsons and Martin Fowler. It pains me to see so much experience and knowledge wasted because of misguided – or misinformed – processes.


  1. I was going to quote Steve McConnell, but this blog stupid comment software (not blogger's default, I believe) prevented me from pasting.

    The gist of the argument is that studies of programmers productivity and code quality found no correlation of either variable with experience. In other words, the time an individual has spent as a programmer is a terrible predictor of performance.

  2. Pretty much everything in this blog, and certainly everything about the comment system, is blogger's default.

    I suppose you are quoting
    "The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968). They studied professional programmers with an average of 7 years’ experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1; the ratio of debugging times over 25 to 1; of program size 5 to 1; and of program execution speed about 10 to 1. They found no relationship between a programmer’s amount of experience and code quality or productivity."

    This 1968 study, which aimed at figuring out if "on-line" programming (ie, direct access to a time-sharing system) was better than "off-line" programming (ie, writing code on a punch card system and then scheduling it for batch execution), and considered a grand total of 21 programmers, evaluated "productivity" as hours spent debugging and "quality" as CPU time spent by the programmer. And, by the way, off-line systems won on quality.

    That study actually found a correlation between quality and experience, contrary to what the quote might lead one to believe. Now, the study was divided in two. In the first one, 12 "experienced" programmers of about 7 years of experience were compared, and in the second 9 "trainee" programmers were compared. The programmers of the two experiments were given different problems.

    This is simply inadequate to measure performance/quality vs experience. The proper way to measure that is to evaluate a set of programmers repeatedly at different ages.

    If you chose instead to measure different programmers of different experience, you have to give them the SAME problem, not problems of different difficulties (as was done).

    Then, if the individual variability is up to 28x, as found in the study, then the set of programmers studied is too small to evaluate differences due to other factors -- including experience -- as the study itself mentions repeatedly.

    Then, there is the question of criteria. As a matter of fact, there is no accepted measurement for quality.

    But what I find most interesting is this study, is that it found no correlation between programmer's experience and the VARIABILITY of the measured quality/performance. Ie, experience does not makes quality/performance more uniform.

    It also found that, while trainee programmers performance showed correlation with Basic Programming Knowledge Test Scores, experienced programmers did not! That is an unequivocal indication that experience changes performance.

    Now, it I doubt Steve McConnell thinks anything of the sort you imply. That would mean Code Complete is a waste of time, as you can't gain anything through experience. Also, I have seen other studies indicating a correlation between experience and quality/performance. But I'm open to hearing about -- and discussing -- any studies showing otherwise.

  3. I didn't mean to say that experience counts for nothing. Peter Norvig's "Teach yourself programming in ten years" was an important reading for me, personally, and the strawman that "you can't gain anything from experience" is ludicrous. Take a single, motivated, programmer now and a few years from now, and I'm sure he'll be a better programmer in the future.

    That said, I do not believe that experience - counted as "ammout of time spent programming" - is a good predictor of an individual's performance relative to others. In other words, taken two random software developers, one much more experient than the otter, and chances are just as good that the less experient will outperform the more experient than the opposite.

    I enjoyed most of your post, but I think you conflated aquired knowlege and experience. The sad fact is that most working programmers (that I have known, i.e. this is anectodal evidence) just don't bother to keep learning as their careers settle down. And the few that do already outperform the average in their early years.

    PS: I maintain that there is something wrong with this comment box, maybe it has to do with blogger version or something :)

  4. Hey, don't blame me for the comment box! :-)

    Well, it seems we are in agreement, then, but that I did not make myself clear enough.

    I do not claim that you can predict which programmer is best from the time they have spent programming. I'm simply saying experience makes a programmer better -- statistically! :-)

    Some do so only marginally, sadly, but I'm not interested in them.

    What I'm interested in is having those programmers who did grow with experience actually get to put that experience to use where it will most benefit their employers: at programming.

    I sometimes use an analogy with medicine. It seems that, if you thought of IT as medicine, we put those most experienced to take patient's histories, and the newbies to do surgery.