What is it that distinguishes the practice of software systems development from other creative professions? Is the act and process of producing software distinctive from other forms of production? In a recent sample of 28 software producing or servicing firms in Ireland covered organisations ranging from 20 individuals to thousands (companies ranged over SMEs to government agencies and multi-national corporations), team sizes ranged from 3 to around 30 (typically from 5 to 10) and software codebase size ranged from 100k to several millions of LOC (see below).
Table: Product size versus team size; Ireland, 2010 (Higgins et al, unpublished).
To put this in context; consider an academic research conference with around 100 papers being presented. Each paper consisting of approximately 6,000 words, equating to perhaps 800 lines of text mapping down to about 600 sentences or fact statements. Let us assume that the knowledge content of the collected conference proceedings is roughly 100 papers x 600 fact statements per paper; equating to 60,000 lines of code equivalent. Contrast this with a fairly typical team of software engineers of anywhere from 5 to 30 people managing the active compilation and on-going design of a software product of 100K LOC. This is the software equivalent of somewhere between 150 to 200 conference papers. Basically, small software development teams are involved in a collective intellectual exercise commensurate with the output of 150+ academic researchers.
To put it in another context, have a look at the graph of codebase size visualisation informationisbeautiful.net. Interestingly codebase sizes begin to approach the numbers of lines of decoded gene DNA in organisms...
What developers actually do
"Developers solve problems at all levels between the ‘whole project’ level and the ‘one line of code’ level... we break complex problems into simpler sub problems. We use this reductionist approach to deal with problems that are too large to handle otherwise."Raccoon proposes a simple cycle of problem definition, development, solution and stability as a motif for the entire business of producing software (below). Status quos can be thought of as known stable points, perhaps something that compiles without errors while also implementing some subset of what is eventually required. He posits that the activities of the problem solving loop actually shape the large-scale structure of development work, regardless of what the espoused methodology or lifecycle is. This based on a constitutive connection between “one line of code and the entire project” (Raccoon, 1995). Raccoon argues for the analogical use of Chaos theory as an underlying theory for systems development projects; simple self-similar processes giving rise to complex emergent behaviour.
(Raccoon, 1995)
In practice developers find themselves working within fractal problem solving loops, in which you may work on all levels; from developer the 'whole program level' right the way through to the 'one line of code' level. Developers use all their skills at all times throughout the project, skills encompassing requirements analysis, design, coding and maintenance.
Raccoon describes the process as essentially a focus on lines of code, on interpreting and problem solving. His insight is that the work of development revolves around problem solving centred on code and data. The 'line of code' means actual software code but can include a requirements statement or project tasks. As each ‘line of code’ changes its impact propagates from one end to another of a project. This brings attention back to the end-to-end nature of design and code and what it addresses. There is a continuum from technical foundations (code, architectures, frameworks, 3rd party tools) through to user visible screens and performance. Its consequence is to highlight the necessity for users, developers and technology to converge and agree for any status quo to be achieved or reach. The other consequence is that any status quo is simply an intermediate point, one of many status quos (see below).
Figure: Relationships between risk/stability activities (adapted from Raccoon, 1995)
A consequence of the Raccoon's analysis is to claim that traditional ideas about the connections between phases of a lifecycle like waterfall or spiral are not true depictions of the state of a project (see below). He provides four life cycle archetypes ranging over the 'simple but rigid' to 'complex but adaptable'. The waterfall or SDLC are termed sequential; modified waterfall is overlapping; spiral is lingering; chaotic is all-at-once or code-and-fix.
Figure: Four life cycle archetypes; the relationships between analysis, design, implementation and maintenance (from Raccoon, 1995)
"Developers interleave design, implementation and testing as they work on different pieces of the project. It is not possible to isolate these phases from each other…each phase involves setting goals, carrying out the goals, and maintaining the results."He concludes that life cycles are just our own perspectives “on the state of a project rather than any essential truth about the state of the project” (Raccoon, 1995) The basic problem solving loop (Figure 7) is self evident but it is also intended to apply to the activities at the whole program level, component level, function level and ‘one line of code level.’ Raccoon sees this as an intrinsic process happening regardless of the lifecycle phase we consider the project to be in at any time. The same basic learning loop applies also during the analysis and requirements phase, design phase, implementation phase, and maintenance. Indeed ‘phase means perspective,’ it is not a predetermined state.
(Raccoon, 1995)
Most lifecycles are simplifications of the real-world situation, although they may not acknowledge this as such, however the truth is that real development projects are complex, messy and unstable. It turns out that the very act of programming is in fact a signature or leitmotif for the whole process of developing software (Raccoon, 1995). While this is very much a developer’s view of development projects rather than a user’s or managers, the perspective aims to give a more truthful description of their roles too.
