Mark Hayes, 2nd August 2019.

Producing and Refining Software Solution Requirements

Requirements are everything. They may well evolve over time, but when they do then developers should be controlling both process and quality (after all, aren't the developers the customers here?).

This doesn't mean developers should be producing requirements, rather they should be providing reality checks, guidance and feedback. If something isn't clear to the developer, the best time for clarification is while it's fresh in the minds of the people producing the requirements.

Without adequate requirements any kind of forecast is simply guesswork. Asking "How long will it take?", when "it" hasn't been defined adequately is pointless and unprofessional. A good developer might reply "What do you mean by it?" or "I need more information", a bad or inexperienced one will provide an estimate.

In particular, when replacing or modifying an existing solution, make sure any limitations and problems associated with it are documented from the start. For example, in relation to inputs and outputs, data integrity, manual processes, reporting, security or privacy concerns etc.

Understanding the limitations and failings of an existing solution is a huge step forward in understanding the expectations of what a replacement should deliver.

Why are Requirements so Problematic

They're Being Produced by the Wrong People, Often in Isolation

Despite the importance of getting the requirements right, it's generally the case that the person producing them doesn't have either the experience or skills, often both, to do the job properly. This is often because:

  1. It's not much fun and nobody appropriate wants to do it.
  2. It's not considered important enough to pull the right person away from their normal duties.
  3. The task is assigned to a junior, or someone with no experience, because an administrator thinks it will be good experience for them.
  4. The task is assigned to someone completely inappropriate because an administrator wants to sneak them into the team.
  5. Producing quality requirements requires a great deal of skill, knowledge and experience. Nobody senior wants to admit this, doing so would bring to light the fact that there's often nobody suitable.

Also, this is a cultural issue. Many people, on their way to seniority in these institutions, rapidly move between departments and teams. It's actually encouraged as part of employee development. However, the real impact of this is two fold.

First, teams become heavily reliant on the people or person that have been around for a long time and, for whatever reason, are not on the path to seniority. These are the people who are good at their jobs, they understand why things are done the way they are and know all the processes, inputs and outputs.

Ironically, these are often the people who are most likely to benefit from new systems and process improvements. Ironic because a) they are the least likely to have the time to help improve or communicate existing processes, and b) they are most likely the only ones that add any significant value when it comes to producing and refining requirements.

Secondly, the staff that do move around regularly, despite all the networking, don't actually become proficient in anything other than networking. They are only briefly part of a team and have only a limited understanding of the its purpose and processes. These are the people that often get tasked with producing requirements. These are generally the ones to be avoided at all cost.

Business process analysis is a skill that requires training. It's as much a profession as programming or accounting. There are proven approaches, methods and formats for documenting and designing data structures, processes and interfaces. Business knowledge alone doesn't compensate for not knowing them.

The fact is, a large portion of requirement related roles are filled by people who are simply friendly, smart and available.

Lack of Quality Control

Requirements tend to be accepted as is. No one wants to pick through them looking for problems, go back to the customer asking for more and then have to chase detail.

In many cases requirements are so bad the majority of the developers efforts are spent either trying to decode them, refine or correct them. So bad they could more effectively be replaced by a single sentence, despite having used up considerable resource and time.

Lack of Visualisation Skills (again)

Being able to visualise the end result is crucial. People that can't are not suitable for producing requirements. These are the people who chant things like "It's hard to know what we want before we've something to look at.", or "Build it and we can tweak it later.". If the person producing the requirements can't visualise, the requirements will be incomplete, muddy and fragmented.

Lack of Communication

Regardless of who produces requirements, an analyst, subject matter expert or developer, it is essential that they not be produced in isolation. Many times I've been given requirements that include a piece of functionality dependant on something else that's not included. For example, a report that includes data not described anywhere other than the report definition.

Another problem area is problem areas, sounds obvious enough, but... In many cases the more complex, or abstract, aspects of requirements are simply skipped over because no one is prepared to communicate the fact they don't understand. Essentially communication is stifled by the unwillingness to raise issues or say something that might be unpopular.

Lack of Existing Documentation

Despite company policy, and in some areas regulatory requirements, a large number of organisations don't have accurate up to date documentation for existing processes. Often there's nothing that can be given to an analyst or developer that doesn't need to be written up first. This is incredible given the the amount of resource directed at administration.

Despite this obviously weak starting point, it may come as a bit of a surprise to learn that this situation doesn't necessarily change following delivery of a new system.

The problem stems from the fact that most processes change and get refined over time, yet little or no effort is put into updating related documentation.

All process documents should therefore be tested in a proper working environment. That means, tested by someone who might actually work the process but has not seen it before. If it's not possible for a user to work a process, correctly and completely, with nothing but the relevant process document to go on, then the process document is inadequate.