How Wrong Can One Guy Be?

I just caught this article in the Seattle Weekly. The author is a former Microsoft employee who now embraces the Mac platform and now has predictions of doom and failure for his former employer.

So how can the supposed failures of Microsoft be sensationalized? Let me count the ways:

Why are Microsoft products so endlessly frustrating to use? Even techno-geeks like me get annoyed by Windows. I’m tired of spending the first 10 minutes of my day rebooting just so I can get to work. Microsoft Outlook 2003, the latest version of the company’s e-mail and calendar software, hangs for me about once a day, requiring me to restart my PC. I also have a problem with Word 2003: Whenever I bullet a line of text, every line in the document gets a bullet. Asking Windows to shut down is more of a request than a command-it might, it might not. And recently, Internet Explorer stopped opening for me.

I have seen this kind of system behavior before, and its name is virus. For Pete’s sake, I’ve been using Windows since 3.1 and I have NEVER seen a system fail that often or that severely in the absence of a virus or a hardware problem. Sure, you get the occasional blue screen (and when I say occasional, I’m talking every couple months at worst), but…Internet Explorer won’t open at all? And that’s Microsoft’s fault? Look at your PC, my man.

But in the first five minutes on my new Mac, I was surfing the Internet, sending e-mail, and ripping a CD. OS X has been a breath of badly needed fresh air after Windows.

This strikes me as the thinking of somebody who, having embraced a new computing paradigm, is now trying to over-justify his change by finding fault in his former paradigm. I mean, Microsoft is far from perfect, but if we’re going to bash them, let’s bash them for things they’re actually at fault for.

Jeff, here’s some news for you: it is OK to decide that you prefer the Mac platform to the Wintel platform. No, really – it’s OK. This is a free country and if you love your G5, more power to you. But what is NOT OK is writing up hyperbole and inaccuracies in order to justify the change.

The 5 Pitfalls of Estimating a Software Project

Updated June 27, 2016

People often ask me why estimating a software project is so fraught, why software projects are chronically late and/or over budget. Naturally, I always take that question as an opportunity to brag about my current level of skill at estimating projects.

It took until I was 6 years into my career to learn the art of estimating a software project so that it was acceptably accurate.  Along the way I discovered a set of behaviors that always lead to blown estimates and broken budgets. Avoiding these pitfalls will put any organization on the road to more accurately estimating a software project.  This will lead to happier clients and profitable projects.

Before we get started on the list, I want to make one thing clear:

Estimating a software project well is hard.

Know-it-all articles that tell you “just do this and everything will work out!” annoy me, and they probably annoy you, too.  Prefacing any kind of advice-giving with “just” is reductive and unhelpful.  Understand that what I’m offering you here are my particular experiences, offered in the spirit of “maybe try this?” rather than “just do this!” – because I acknowledge that what we’re trying to do when we develop estimates is hard.  In other words, these 5 pitfalls have been true for me and for the teams I’ve worked on, and I suspect they’ll be true for you too.  Ultimately, it’s up to you to try to fit them to your experiences as a developer and see if they work for you.

All that said, here are the 5 behaviors that I’ve seen consistently contribute to bad estimates:

1) Allowing non-technical staffers to give estimates.

You think I’m about to launch into a typical software developer’s tirade about how talent-less the people on the “business side” of the house are.  I’m not a typical software developer, though.  I value the business side of the house as much as I do the engineering side.  That said, a non-technical employee estimating a software project makes no sense.  You wouldn’t allow a software engineer to commit the company to a marketing plan of his own design, would you?  Of course not. Despite this, non-technical staffers giving technical estimates happens all the time.

To be clear – NOBODY should be estimating a software project except for the person who is actually going to perform the work.

Specialize or Die

Remember Adam Smith? He wrote an little book entitled The Wealth of Nations, maybe you’ve heard of it. In this book, he argued for the specialization of labor, using a pin factory of the day as an example:

…where each of a dozen workers engaged in only one part of the process of manufacture, so that together they produced far more pins than if each worker produced whole pins; the price of pins then fell, and more pins could be used by more people.

When businesses violate the principle of specialization of labor with regard to creating their product, trouble is soon to follow. It is far better to allow the specialists to do work that falls within their specialty than it is to placate a client by giving a totally fictional estimate.  Estimating a software project is specialized work, and should be treated that way.

Be Graceful Under Pressure

Because of the Iceberg Effect, a salesperson out in the field (assuming you work for an org large enough to have salespeople out in the field), who is under great pressure to close deals, is likely to think “Well, that’s just a couple of screens and buttons, it shouldn’t take very long“.  So the salesperson gives the client a number, and the number may as well be 1 hour or 1 million hours, because it cannot possibly have any connection to reality unless it comes from the person who has to build the thing being estimated.

Let your salespeople sell, your marketers market, your executives execute, and your developers develop…estimates, that is.  If you happen to be a small shop where people wear multiple hats, that’s just fine so long as the person who has to build it is also the person providing the estimate.

2) Being afraid to look in the mirror.

Most development shops I’ve interacted with do not do any “post-mortem” reviews at the end of an over-budget, past-due project (or even a successful project, for that matter). Therefore, they never learn what went wrong, never disseminate information that can other staffers avoid what happened, and thus promptly repeat the exact same mistakes. This causes the next project to be over-budget and past-due, and that project doesn’t get reviewed either, so the same mistakes are made on the project after that, and so on. Even seasoned developers can fall into this trap.

Don’t be afraid to take a look in the mirror and ask “what did I do wrong?” – this applies to everyday life as much as it does to software development. Always analyze your projects for both mistakes to correct and best practices to carry forward. Failure to do so is the equivalent of flying a 747 without visibility or instrumentation through the Himalayas; way too much risk for most client’s tastes (and checkbook). And again, if you’re a small shop without much in the way of formal review processes, take notes as you go; all you have to do is document the things that went poorly, and the reasons why.  When estimating a software project, having this kind of info at your disposal will help you create a better, more accurate estimate of the work to be done.

3) Underestimating design time and debugging time.

Many software developers are not in tune with The Business, they are in tune with The Thing – that is, they focus tightly on the act of construction and sometimes miss the other, equally vital, things that go into a successful project. In my experience, design is one of the most commonly underestimated things.  Yes, I just said that only the person who builds the thing can estimate the thing, and that counts for design work, too.  This is why it’s important for design and dev to work together when estimating a software project, rather than being silo’d away from each other or, even worse, being actively at odds with one another.  I’ve seen it happen in some dysfunctional orgs, and it is as ugly as it is counter-productive.

Figuring out how to build something takes more time than it took to figure out what to build. On the other end of things, debugging and refining and polishing up the code takes more time than anyone ever anticipates – in fact, in all my years in this business, I have never seen an accurate debugging estimate when estimating a software project. Never. Not once – not even my own!

Budgeting 50% of total development time for debugging is absolutely reasonable, if not necessary. Any developer who fails to accept this is still in the Age of Unlimited Wisdom (more on that later).

4) Inadequate/unclear requirements.

Before estimating a software project, you have to do some requirements-gathering.  There are many ways to do this, but gathering requirements at all is difficult. Gathering requirements well is rare, and gathering requirements in the correct quantity and level of detail before asking a developer to start work is just this side of miraculous. I would estimate that 90% of my pre-consultancy projects suffered from inadequate requirements.

Developer vs Analyst

Every developer has war stories about dealing with unbelievably poorly-written specs.  Every developer has war stories about requirements with more holes in it than Swiss Cheese.  Most developers like to blame the Business Analyst for this.  To be fair, most Business Analysts have war stories about OCD-affected developers.  Stories abound about devs who suffer from analysis paralysis without impossibly perfect specs.  That’s just the nature of the beast sometimes.

It is tempting to rush the requirements gathering process. It is tempting to just assume the answer to questions that are not addressed in the requirements. Mapping a client’s business process is hard, and very few people want to do it – those that actually relish the task are of a rare and strange breed. Uncovering and documenting every relevant hidden nuance of how a company operates is difficult.  This is why estimating a software project is difficult – it encompasses other things that are difficult.

Satisfying the Stakeholders

It’s downright hard to ensure that the high-level details are an accurate representation of what needs to be built. Inaccuracies will creep in if you let them.  This is doubly true if you are in a rush.

By slowing down – just a little bit – and making sure that the requirements gathering is done right (as in, not too detailed and not too vague) before construction begins, everyone will end up happier and the project is infinitely more likely to be a success.

5) Taking too large a bite from the apple at once.

I cannot count the number of times I’ve watched in horror as a Project Manager approached a junior programmer, asking “How long will it take to do X?”, only to hear “Not very long – maybe Y hours” in response from the young know-nothing.  My horror, you’ll note, stems not from the fact that the junior developer was a know-nothing (we all start someplace), but rather from the fact that the Project Manager should know better than to put a junior dev on the spot for something like that.  It’s bad for estimating a software project, it’s bad for whatever downstream politics the project is likely to run afoul of, and it’s exploitative of the junior developer.

Finding the Right Side of the Line

Regardless of what feature X might be, there’s only one correct answer to “how long will it take?”.  That answer is, “Let me see the requirements”. If you don’t have requirements, you need to get some, even if they’re super-lightweight. Even then, you can’t commit to anything better than a “plus or minus 50%” estimate until everyone has agreed that the requirements are on the right side of the 80/20 rules in terms of accuracy. Even then you cannot start estimating a software project until you break down those requirements down into bite-sized chunks.

Making Your Project Chunky

That, my friends is the secret weapon of estimating a software project – the more atomic you can make the requirements, the more accurate your resulting estimate can be.  This is why I love user stories – they’re super atomic (maybe too atomic for estimation purposes, but we can argue about that another day).  That said, the truth is that most people are lazy. Analyzing requirements is almost as difficult as gathering them, and again – very few people want to do it. Those that do will consistently deliver more successful projects than those who do not.

I’m not saying anything here that people in the industry don’t already know. But I am saying a lot of things that people in the industry don’t do. I wish that were not the case, but I can only control my own projects.


Photo Credit: Luis Fernández García (license)

Yes, RSS Is On The Way…

…although I can’t imagine why anybody would want an RSS feed to this poor, neglected blog of mine. Still, to the few of you who have eMailed me asking if I’m going to set up a feed, the answer is “yes, as soon as I have a spare evening”.

I am also going to implement a simple discussion board when I get a chance.

Beaten To The Punch…Again!

I remember waaay back in 1996 when I thought to myself “Self, it sure would be cool if somebody would market computers with brightly-colored cases. Gee, maybe I’ll do it!” I never took action. Two years later the iMac was born.

Then there was the time when I thought to myself “Self, wouldn’t it be spiffy if these was a way to seamlessly swap files with anyone over the internet without having to use eMail?” Yeah, the idea was so spiffy that Napster did it just a few years later.

Well, it’s happened again. Just a handful of months ago, I thought to myself “Self, why are there so many people who know how to write code, but completely fall down on every other aspect of developing software? Maybe I should write a book about that!” And lo and behold, Mike Gunderloy beats me to the punch with Coder to Developer: Tools and Strategies for Delivering Your Software.

Of course, Mike being a legend in the industry and me being – well, nobody in the industry, odds are that no one would have ever read such a book if I had written it, whereas everyone in the industry is likely to at least skim a book with Mike’s name on it. So that is a good thing – I’m sure tons of coders are going to read this and learn something useful. Heck, I’m going to read it and will probably find it useful!

Now, wouldn’t it be cool if someone could invent a way for everyone to communicate via their computer? You know, we could have these things called "pages" that everyone could read, using a program called a “browser”…

Gmail Mania Grips the Masses

It’s official…Gmail invites are the hottest thing since Willy Wonka’s golden tickets. You show me any message board on the internet and I will show you at least one “Beg for a GMail Invite” thread. Things get even deeper if you hit the blog of any Google employee. I am ashamed to admit that I actually contributed to one of these “me too” threads. What a chump.

Why is this so hot? After all, it’s only free eMail, and we’ve all got free eMail already.

I’ll tell you why it is so hot: Google has street cred unlike any other company out there right now. Their technology is proven and their corporate mantra resonates with the geek demographic. In short, we implicitly trust Google to deliver the goods. I challenge anyone to name something Google has offered that sucks.

Nobody knows when GMail will be officially launched, but until that golden event takes place you can bet there will be people continuing to scour the internet in search of extra invitations.

And all I have to say about that is…invite, please? 😉