The move to agility involves refocusing on practices and discourse. An introduction to the principles and moves in methods termed ‘agile.’ A review of Extreme Programming (XP), the Agile Manifesto, and Scrum.
"We need to make our software development economically more valuable by spending money more slowly, earning revenue more quickly, and increasing the probable productive lifespan of our project. But most of all we need to increase the options for business decisions."
(Beck, 2000)
AGILE PRACTICE
Towards the end of the 90’s and early 2000’s saw the emergence of so-called Agile or improvisational models, including Extreme programming (Beck, 2000), Agile Development (Highsmith, 2002) and derivative approaches like Lean Software Development (Poppendieck and Poppendieck, 2003). In 1999 Eric Raymond (Raymond, 1999) posited that there were two diametrically opposite strategies evident for organising and engaging in design work; the Cathedral way and the Bazaar way. The then current radical movements in the software industry centred on Open Source software, extreme programming (Beck, 2000) and agile methodologies (Highsmith, 2002), were all characterized by frequent iterations, dynamic planning, intensive testing and making releases available regularly. The Open Source and Agile movements represented the Bazaar way for software development. The use of strict lifecycle models or organisational control frameworks (e.g. CMMI, RUP, ISO9001 style frameworks) was emblematic of the Cathedral way of software development.
Figure: Manifesto for Agile Software Development (source: agilemanifesto.org)
Agile methods have since completely transformed practitioner understanding of how to organise software and high-tech development. These practitioner-oriented methods assume software development occurs in response to early, frequent, feedback. This in turn requires management commitment to allow and enable plans to
evolve continuously. Agile methods have so successfully capture the development imagination that the CMMI and RUP (among others) have attempted to incorporate ‘Agility’ within their own meta-narratives.
THE AGILE MANIFESTO
In 2001 a group of developers gathered in the Lodge at the Snowbird ski resort in the Wasatch mountains of Utah. Inspired perhaps by Richard Stallmans's GNU manifesto (
www.gnu.org 1985) or Mitch Kapor's Software Design Manifesto in 1990 (for an extract see
hci.stanford.edu), Seventeen of them put their names to the "Manifesto for Agile Software Development" (
agilemanifesto.org). The 'Manifesto' had two significant effects, it launched the 'Agile' turn in software development by offering a broad sketch of principles and a vision for the values of software development. It also played within an industry 'meme' of emancipation, inspired by other idealistic manifestos written for the software industry and beyond. Some of these interventions were intended for industry mindshare or commercial gain, others ironic or humorous (
soa-manifesto.org,
www.waterfallmanifesto.org,
www.halfarsedagilemanifesto.org,
failmanifesto.org,
manifesto.softwarecraftsmanship.org,
www.relisoft.com,
Library Software Manifesto).
Figure: Principles of Agile Software (source: agilemanifesto.org)
EXTREME PROGRAMMING
In 1999 Kent Beck introduced the world to the idea of Extreme Programming (XP) (Beck, 1999). He presented an interesting and compelling vision of the process of programming that appeared to offer substantial, almost radical, benefits if adopted or adapted into organisations.
Beck’s paper and subsequent book (Beck, 1999; Beck, 2000) provided explanations and actual cases and generated such interest that it has since taken on the appearance of a 'movement' in among professional software engineers. Such is the respect with which it is held that management and teams should take XP evaluation seriously, if only to establish a position on it on a principled basis. The culture of XP is based on the following four values: Communication, Simplicity, Feedback and Courage. In practical terms the challenges that XP address are characterised in terms of the traditional variables of project management: Cost, Time, Quality and Scope. These aspects describe the mind-set and the degrees of freedom you operate within when you work XP.
Distinguishing Characteristics of XP
The defining features of XP were set out by Beck as follows:
To learn from early tangible feedback from short development and delivery cycles; that is to develop and release often.
Incremental planning is therefore essential if learning from early release is to be fed back into the development cycle. The software development project or plan therefore needs to evolve continuously.
The schedule should be flexible to enable the team to implement new ideas, measure their cost and test their benefit, and then reset the schedule.
All tests should be written before coding. One way of understanding this is that tests are written as code is written and changed as code is changed.
All tests need to be automated and run as often as possible. What this means is that all the tests can be run 'at will' but at the very least with each build of the software.
Communication must become the very heart of development; communication can be many things but is perhaps the key practice of XP.
Design therefore needs to be reviewed continuously. XP rejects the idea that design is a one-shot up-front “design phase” that ceases prior to coding, nor is there a place for one-dimensional roles like analyst, architect or test engineer. Everyone does some design and the design is continuously evolving.
Finally, all coding needs to be collaborative. Collaborative coding implies that code is subjected to continuous to peer review and contributions through XP's emblematic process 'pair programming'. The consequence of this that responsibility is also shared. No one 'owns' the code yet everyone 'owns' the code.
XP'S FOUR VALUES
Communication Is done through source, test and code, in comments and other artifacts. It r requires real commitment, and can be in-your-face at times; it can be uncomfortable, unsettling. Communication requires engagement in all spheres, written, verbal, non-verbal, in-code, on walls. Kent's catchcry to 'embrace change' can be read as 'embrace conflict' too.
Simplicity is as aesthetic appreciation of what code and the design becomes. As a philosophical principle for design it can be used like a knife to continuously pare the program to necessary essentials. Simplicity requires eternal vigilance as is in a sense the driving dynamic behind the practice of 'refactoring'.
The very idea of feedback is integral to an XP style of working. Feedback is supposed to be real world and is exemplified by Beck's request that you bring the customer into the development environment, have the customer/user sitting and working beside you. Customers are necessary because developers are ignorant, ignorant in this case because you (the developer) are not an expert in the customer's domain and you can not know what the customer really needs, in spite of attempts to codify and capture in requirements documents. (Note: sometimes neither does the customer but that’s where eXtreme succeeds). In the same way that the customer does not know about your domain of expertise (coding).
And Courage. Right! But think again; coding, recycling, throwing it away after you have learnt how to do it right, taking an idea and writing/testing some stuff compile/debug, then using it, throw it away and start again. Thinking of programming as something miraculous, making something intangible tangible, bringing thoughts to substance in a program, thinking of programming as an inherently creative and therefore unknowable a-priori; it takes courage and perhaps a certain amount of luck to get it right.
THE VARIABLES
XP, like other systems development approaches, attempts to maximise project cost, time, scope and quality: Four variables of mixed types, uncertain (even changing) definition. If correctly modeled this problem would involve finding the maxima/minima on a complex perhaps fractal multi-dimensional surface?
Cost can increase or decrease and depends on the availability of money, people, hardware, tools. Time is generally limited and linked to estimation, other resources, need etc. Timeframe is also a concern, how long will the software be used for, is maintenance required, are people available, is the product dependent on other product release cycles. Scope should be variable, the customer can of course demand features, the goal however is to deliver on the features you really need now, leave the other stuff till later. Don’t deliver now what you can put off to a later iteration.
And Quality; the one variable that shouldn’t be treated as a variable and also the one perhaps most difficult to define. Quality is often an indicator of the success or failure of our ability to balance the dynamic interactions between cost, time and scope. Compromising quality undermines and destroys the values XP aspire's to; a craftsman strives for quality, the inherent value and appreciation of things made for use.
XP PRACTICES AND RULES
Many of these practices and rules have since become accepted as general professional practice on systems development projects. Many projects already apply some of these rules but XP aims to use ALL the rules as they reinforce each other. At least half of this list is accepted as general-programming-best-practice. Evidence of the following practices and rules defines whether a workplace is employing XP or not.
Small Releases: Start programming, check your progress against your goals, correct your direction and continue programming, repeat until you finish!
Metaphor: A simple design analogy or descriptive rubric that describes the software in a language that people from very different backgrounds can share.
Particularly useful for the customer to illustrate the problem in terms that are meaningful, e.g. “the IP packet routing and subnet bridge device” becomes “networking networks”, or “the host server proxy and daemon product” becomes a “portal”, or “the object request broker architecture for distributed applications” becomes “middleware”.
Simple Design: Focus on the real business problem, then design to meet that need, not anticipated needs or “really neat stuff” that some hypothetical future customer might find useful. You’ll probably get the first (and second) one wrong anyway so don’t invest unnecessary physical and intellectual capital into something you may throw away once, twice or three times.
Testing: You write the code, you write the test (this is a “plural” you, see pair programming to follow).
The test is written before you code, automate the new test, run it often to boost your confidence.
On-site Customer: Have full-time access to a customer or someone representing a customer working on the team, on-site with you as you code up the features.
Coding Standards: Each programmer accepts the collective coding standard to support the all the source code, by making the style and formatting conform to a common standard so that the code is accessible to all past, present and future programmers that will touch the project.
No arguments about the position of curly braces or indentation, layout of functions etc.
Pair Programming: Can be roughly approximated as peer review or code review. Recall the Open Source pardigm “with enough eyeballs all problems are trivial”. The heart of XP, if you aren’t pair programming you aren’t doing XP. This (and the Planning Game) is the mark of a true XP project.
Collective Ownership: Has huge cultural and behavioural implications, this implies behaviour and practices akin to open source development. Everyone has access to the codebase, therefore everyone can (in principle) become expert in the architecture and ‘theories’ of the design of the software. But with ownership comes responsibility, instead of zones of ownership you have zones of greater or lesser expertise, ideally these average over the team, expanding over time or disappearing when members leave. The ideal outcome is to have sufficient knowledge embedded in practices, experience, memories and knowledge of the team members developing and maintaining the product.
Continuous Integration: Requires modern and usually open tools for managing source code, e.g. Subversion, CVS, git etc.
40 Hour week or Sustainable Pace: A reasonable expectation on the part of workers and the business.
Refactoring: Coupled with attention to classic Patterns (architectural motifs), is a way to introduce simplicity of design on a continuous basis. Without refactoring bug fixes and the addition of new features or enhancements leads to increasing cruft, entropy and fragility. Refactoring removes what is termed 'cruft', it restores order and returns the software to stability.
The Planning Game: The planning game consists of periodic meetings development and the customer (or the customer’s representative) to explore the requirements and track progress. It facilitates incremental planning, reprioritisation and change. The plan and the software evolves continuously and visibly. The Planning Game involves the Business (customer) and Development (supplier) sitting at the same table. The game itself consists of three moves, exploration, commitment, and after an iteration, steer. Within an iteration the game is continued via Stand-up Meetings with the developers and the on-site customer present. The same moves can play out (exploration, commitment, and steer) but we can also follow how well the developers implement the 'user stories', verify the value and usability of completed stories, track progress and problems, and attempt recovery.
Figure: The relationship between XP's key practices.
Agile work environments are supported by a host of tools and equipment. The
Soft Infrastructure requires good source control tools, test frameworks, build frameworks, email, news, messaging etc. An effective physical infrastructure includes best in class computer workstations, large (and multiple) monitors, accessible desk space, shared workspaces and personal areas, meeting rooms, whiteboards, display walls (information radiators) etc. The social environment has its own qualities, typically a strong espirit d'corps, pizza and social outings, sports at lunch and other social/work activities. The key point is that these values, rules, culture and practices support each other, weakness in one is covered by strength in another.
XP'S BENEFITS AND RISKS
XP is a distinctive attempt to shift from the general situation where decision-making authority resides in management teams and at key development milestones in Test/QA teams. It facilitates a shift in power and responsibility towards the engineers themselves in the product teams. As Beck’s opening introduction states...
"Extreme Programming turns the conventional software process sideways. Rather than planning, analyzing, and designing for the far-flung future, XP programmers do all of these activities—a little at a time—throughout development."
(Beck, 1999)
One way of reading XP is that it is an attempt by software engineers to exert or reassert a claim of power (autonomy, authority etc) over the production of the software. While Beck claims the effect of XP is limited to just the software engineering domain, we can conclude that if engineering teams adopt XP practices (and principles) then they will inevitably impact the other business divisions interacting with particular software development teams. The consequence therefore of adopting XP on even one team is to raise the possibility of ‘viral’ process occurring within the organisation, one that may alter the locations of control and decision-making throughout the organisation with unpredictable consequences. Like any alteration to the power balance within an organisation change itself is often subversive to current management knowledge and understanding. These are radical claims which, if we follow Beck's advice, may end up leading us to an unknown place and so a caveat; when attempting to adopt any new business process or methodology, members of the organisation need to carefully consider the potential impacts.
