Understanding the Size of the "Job"
Note that I put the word Job in quotes. This is really important! The job is the deliverable, not the process intended to produce the deliverable. It doesn't matter how well a team works a process, if the deliverable is either inadequate or not cost effective, they failed. Too many organisations have geared themselves to working processes instead of delivering solutions.
In most cases where projects go so horribly wrong, the cause can be traced back to the starting point. If the Job itself isn't clearly defined, even at a high level, then the person or people managing it can't possibly know how big the project should be. Equally, how would they know who to put in charge, or later on, if the initial proposals are even appropriate or if progress is adequate?
Getting it wrong at the start can mean the difference between spending tens of thousands or tens of millions, even worse (see examples). The fact is, quality and appropriateness start dropping as soon as the number of people involved increases beyond the minimum number required to deliver a solution. Too many chefs and all that. Worse still, getting the architecture wrong, a common mistake when getting size wrong, can lead to a high maintenance, clumsy and costly solution.
Click here for examples...
What Makes Size so Difficult to Quantify?
Certain scenarios involve a great deal of unknowns, technologies, inputs and outputs. That doesn't mean that they are big jobs (no pun intended). Dividing an on-line solution, for example, into application tiers, listing a few technology options, security features and privacy concerns, will produce pretty much the same diagrams and process flows for a simple application as for an incredibly complex one.
Presenting a solution this way, to provide a "high-level" overview, is exactly the sort of thing that results in loss of perspective. Customers, and managers, that don't have sufficient knowledge and experience won't be able to tell the difference between a project that can be delivered by two or three people in a couple of weeks, and one that requires a much larger team and a year or two to complete.
Even a simple login process, broken down into its constituent steps, looks like a big deal to the uninitiated. Despite the fact it shouldn't take more than a couple of hours to build, or more than likely is already part of a code library.
The problem then is two-fold. First, having non-technical people negotiating a solution will generally make a mountain out of a molehill, or vice versa. Second, these people are far more likely to be exploited by greedy, less scrupulous solution providers, internal and external, simply because their lack of technical understanding makes them an easy target.
Lack of Technical Skills and Relevant Experience
This is a consequence of not getting the right person. Managers without technical skills tend to either oversimplify or overcomplicate everything. They share responsibility for decision making because they lack the knowledge and experience necessary to make important decisions themselves.
People unable to at least get a good feel for the basic architecture required, have a tendency to over-staff. In most cases, unless a solution is unusually large, a single programmer with a good set of requirements and access to one or two subject matter experts is all that's required.
Many solutions are delivered miserably by teams comprising multiple programmers, data analysts, business analysts, subject matter experts, testers, project managers and administrators. Not necessarily because of lack of skill, but because having too many people involved guarantees delivery will be sub-optimal at best.
Not Involving Developers from the Start
Administrators, accountants in particular, lack vision. Many times we, the developers, hear the words "It's hard to visualise without something to look at.". Wire-framing and story-boards play an important part in refining requirements, but if the audience can't visualise they won't be able to quantify or contribute.
Without good technical direction and input anything can happen. Look at the UKs National Patient Database for example. An estimated £10 billion spent on procrastination and failure. A small team could have developed the software in a year or so for less than a few million.
Pandering and Telling Porkies
Large projects are usually kicked off with a flurry of bold statements, reassurances and promises. Nothing is too much, takes too long or can't be done. This is just amateur theatrics, intended simply to win budget, and is the most toxic way to start a project.
Most customers want an adequate, cost effective solution delivered in a timely manner. Most managers (administrators), think customers need to be shielded from the realities of solution delivery, manipulated and kept away from the development team. This is wrong and results in a dramatic mismatch between expectations, potential and what's being delivered.
When this happens the "managers" panic and waste considerable technical resource seeking solutions and excuses for problems that should never have arisen. Worse still, they try to de-scope less important features to compensate for wasted time and resources.
How to Find a Size that Fits
Involve Developers from the Start
Chances are, a developer will have done something similar before and be able to provide a far better feel for the size of a job than anyone else involved.
A developer sees things from a different perspective and will ask the most important questions early in the process. Questions others might think insignificant, but whose answers have far reaching impact on the way a solution evolves.
This helps avoid the most costly omissions and mistakes, the ones requiring rework and restructuring. Project milestones are easier to define and schedule, the solution is cleaner and the delivery team has better direction.
Anyone between the delivery team and the customer is likely to bring little more than noise to the process. In situations where the customer is unable to describe the problem in full, developers, SMEs and real analysts are far more likely to coax out good detail.
Not "Managing" Expectations, AKA Not Lying to the Customer)
Essentially, the closer the relationship between developer and customer the better.
Packaging anything to look simpler than it actually is is a big mistake. If a customer, internal or external, gets the impression something is no big deal they will a) expect to pay less, b) expect it a lot quicker than can is practical, and c) when delays and problems do occur, assume either they were lied to at the start or the delivery team is not adequate.
Rather than brush over potentially problematic aspects of a job, be honest and keep the customer in the loop. This way they'll have more realistic expectations and, if required, make resource available to discuss and help clarify more complex issues. Doing this at the start, before and during the production of detailed requirements, will produce a much better understanding of size and scope.
It's better to over estimate than under estimate. Customers are never disappointed by an early delivery that comes in on, or under, budget. It is what it is, most people can deal with that a lot better than they can disappointment.
Ideally the person, or people, commissioning the work should either have a very good understanding of what's required, or consult with someone independent who does. Reality checks are essential, and more importantly... if you are the customer, make it clear that you will hold the supplier to the promises they make. This way they will be reluctant to guess, shy away from potential issues and underestimate to get approval.