The blame is not for failure

Jørgen Vig Knudstorp: At LEGO, Growth and Culture Are Not Kid Stuff
"The blame is not for failure, it is for failing to help or ask for help"

Grant Freeland interviews Jørgen Vig Knudstorp (Boston Consulting Group video)

Exercise: Construction

DesignConstruction from Allen Higgins on Vimeo.

Preamble:

1. Do the Lego Mindstorms familiarisation exercise first (see slides)
2. Then have individuals make their own sketches of a lego robot that can guard a "base",

Objective:
To experience the different activities and issues that can arise in teams constructing a shared object in accordance with a well specified design.
To enable people to get to grips with a particular technology ecosystem to enable reasoned judgement and design thinking relevant to other related exercises.

Preparation:

1x copy of the design/construction document per group.

1x LM construction box.

Index cards, pencils, paper.

Instructions (45”):

e.g. A group race to construct a driving base.
Provide a brief introduction to the construction environment and capabilities of the tools used. 

Outputs:

A completed driving base.

Interjections:

• At intervals team members have to 'rotate' around the workbench. 
• About 15' into the exercise pause and reflect how they have ended up organising themselves. Would you choose a different arrangement? Roles? Spatial arrangement? Move things around? Leadership? Democratic?
• About halfway to completion offer the colour booklet.
• About 30' into the exercise pause and reflect. Ask 'what is the design? The guide? Which page? The finished product? The small details? The ideas? Did anyone notice errors? What was the impact of colour? Any improvisation? Coping with missing parts?

Debrief:

See Exercise: follow up on construction

Previous results:

Class of 2014/15 FT 

Group ID	Time (mins)
a	1 hr 13'
b	non-finish

Class of 2013/14 PT 

Group ID	Time (mins)
a	24'
b	non-finish
c	29'
e	27'
f	non-finish
g	non-finish
h	non-finish
i	23'
j	non-finish

Class of 2013/14 FT 

Group ID	Time (mins)
a	non-finish
b	non-finish
c	non-finish
e	20'
f	non-finish
g	'1'? non-finish
h	non-finish
i	31'
j	non-finish

Class of 2012/13 

Group ID	Time (mins)
a	20
b	35
c	40
e	29
f	non-finish
g	42
h	47
i	40
j	34

Class of 2012 

Group ID	Time (mins)
1	non-finish
2	non-finish
3	33
4	40
5	33
6	25
7	28
8	34

I couldn't resist "Why Your Employees Should Be Playing With Lego Robots"

Arguing for play-like activity in work (not just using Lego to communicate ideas) by Colin Lewis...

"Participants from all backgrounds gain key team building skills through collaborating closely at every stage of ideation, innovation, deployment, evaluation and scaling. At the end of the training teams are required to present their ideas and results, building effective communication skills." (Lewis, 2014)

Requirements (SDLC)

Part of successfully designing and maintaining a high tech system is having a strong top down vision. In the absence of vision nobody has a clear idea of what belongs and what doesn’t.

"It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them." Steve Jobs, quoted in BusinessWeek (25 May 1998) cross ref. (link)

THE USER AND REQUIREMENTS
Understanding the requirements process. An overview of the need for requirements and how they can be gathered and managed for systems development.

If you have a product or if the product is still just a concept you must be able to answer the question: “what goal does the user attain by using your ‘offering’ or product?”

INTRODUCTION
Why do we need to talk about requirements? Aren’t requirements usually self-evident? Are the really valuable requirements obvious or easily captured?

Product management needs to be grounded in a disciplined approach to managing high tech design and development. The product or project manager, right down to the programmer, must be able to answer the question "how do I know when I'm done?" However requirements may may not capture a clear or coherent statement of the problem we need to solve. Requirements can state outwardly perceived problems but in ways that are value laden and suggestive of remedies (perhaps as band-aids) rather than as simple statements of issues that could be addressed in . In these cases requirements encroach on design and may result in the worst aspects of incremental change. (Foote and Yoder, 2000).

Identifying the real problem is often the problem (Curtis et al., 1988). The requirements process is therefore more often a process of mutual discovery; uncovering and discovering the need and use for a high tech system.

Three simple questions offer a revealing insight into the maturity and capability of the organization:

1. Show me a statement of the requirements your product delivers?
2. How were the requirements gathered?
3. From where does each requirement originate?

User requirements and analysis is often considered to be the starting point for the systems development process but it is rarely a straightforward process of transforming a list of requirements into a product. It is more often a process of intervention and change that impacts “personal and group efficiency, effectiveness, job satisfaction and the quality of working life.” (Mumford, 1985) Edith Mumford suggested that the best possible approach for design teams is to involve users with the developers as part of the requirements capture and design process. To create a situation where user requirements can be clarified and defined in the absence of solving underlying technological considerations. Requirements definition should therefore focus on user problems and needs before looking at current tasks. Identify the goals before looking at the minutia of the current list of tasks undertaken to overcome or satisfy the goals.


AN UNDERLYING THEORY OF REQUIREMENTS?
Why have a requirements process and what are its distinctive aspects? In an ideal world the developer and the user would communicate directly and perfectly. The user would be able to perfectly articulate some need, the developer would understand the need and be able to create and deliver it. However in the real world our understanding of needs and ability to communicate them is imperfect. In addition, in the real world, there isn’t a one to one correspondence between users and developers. A successful product may have hundreds, thousands, if not millions of users (Figure below, a). Conversely a product development team size may be anywhere between five and a hundred people (rarely more). The more users there are the less likely all their various needs can be articulated, communicated, implemented and satisfied. The more users there are the less likely the developers can interact and communicate with them on a one to one basis. The immediate role of an analyst may be quite simply to buffer or manage interaction and communication between users and developers.

The challenge of such a role is its potential to further distort the process of communicating between users and developers (Figure below). But simply adding an analyst to the equation doesn’t address the core challenge of large user communities interacting with small developer communities (Figure below, b). Consider the following three cases as ways of organizing interaction between users and developers. Case I (Figure below, I) corresponds to direct communication between developer and user, case II (Figure below, II) to the business analyst as a buffer between and two, and case III (Figure below, III) to open communication between developers and selected users. The solution to addressing large user communities is to select a representative subset of users and work with them (Figure below, c). As a general rule of thumb we could assume the number of representative users will be of the same size (or order or magnitude) as the size of the analyst/developer team (Figure below, d). The risk with a product requirements document is that it lies between the user and the designer; its use may both communicate and mis-communicate. The key to success is to selectively enable direct interaction to clarify and validate both the requirement and its delivery.

The ETHICS approach
There are many requirements management frameworks most of which are basically templates and checklists for gathering and recording a variety of user-oriented data. Mumford's ETHICS framework is a useful approach to organizing participative problem solving initiatives for systems development. The acronym is also intended to convey that an ethical orientation is also necessary to allow and enable the analyst to contribute to good systems design resulting in increasing “the positive factors and eliminate or reduce the negative factors” (Mumford, 1985) for user interest, effectiveness, job satisfaction and commitment. Mumford's ETHICS approach to structuring systems requirements specification consists of question challenges and resolving activities.

The risk however with a requirements gathering process cast in this format is that it can expand into an open-ended discovery of a range of problems that weren’t previously articulated. You quickly loose yourself in changing, evolving goals. This is amplified by users awareness of the coupling between requirements and design. They understand fully that implementation of new technology impacts and challenges the target organization. New IT alters the day-to-day organization of work, job satisfaction and corporate mission. Impacts and challenges may be both positive and negative thus Mumford’s emphasis on the ethical treatment of people. In this case changing people’s tasks and responsibilities is predicated (and made valid) on the basic assumption that the exercise is to produce changes that improve the conditions, effectiveness and satisfaction of work.


METHODS FOR GATHERING REQUIREMENTS
The end result of any process of requirements capture should be a body of raw data (interviews, statements, recordings, diagrams, images, documents, objects). The requirements are a distillation of that data as lists, categories and descriptions that are wherever possible, traceable back to its originator or source for validation.

Contemporary methods for gathering and managing requirements range from formal negotiated approaches like ETHICS to informal (though still negotiated) processes evident in newer Agile methods (Cockburn, 2002). New methods for organizing systems development such as Extreme Programming (Beck, 1999) and Scrum (Schwaber, 2004) create distinctive roles for the customer or product owner at the heart of the development process (Figure below). Kent Beck popularized the idea that a development team needs an ‘on-site customer.’ Ken Schafer asserts the need for a ‘Product Owner,’ someone who speaks for the customer authoritatively and with ownership. Scrum’s insistence on a product owner is essential as the process is driven by value and therefore it is crucial to identify the person who is responsible for the product’s ROI and who is therefore responsible and accountable for decisions such as what the requirement is and which requirement to deliver first. The ideas of an on-site customer and a product owner are designed to simplify the communication process between users and developers. However they say little if anything about how to choose the users, communicate or interact with them.

Interaction Design (IxD), which has grown in popularity among HCI/CHI designers, directly addresses the methodological challenge of gathering requirements from user communities. Interaction design positions the role of the interaction designer rather than ‘analyst’ as the central figure bridging the gaps between users and developers. A more open interpretation of the role of Analyst or Requirements Analyst is anyone who is involved in requirements gathering and analysis. Various business titles could be applied to the same role: Product Manager, Marketing Analyst, Researcher, Interaction Designer, etc.

Product requirements can be thought of as a rather unique kind of shopping list, a shopping list written by (more often on behalf of) the user, and written for (usually by) the developer to deliver (Figure above). Taking the analogy further the requirements shopping list is for a shop where the shelves are initially empty because the things the user wants haven’t been made yet. Alternatively there may be something on the shelf that approximates what the user wants but it’s not quite right and needs to be customized. To compound this seemingly odd situation we may also find that product requirements may be written (created) by someone who is neither the customer (user) nor the designer (developer). In this situation those charged with requirements capture have a lot of responsibility and power to influence the design process. Product requirements lie between the user and the designer and act as a communication device between the two. The requirements document is merely a representation of a potentially unbounded set of product requirements therefore the process used to create the representation is perhaps more significant that the document itself.

METHODS FOR ELICITING REQUIREMENTS
Methods for gathering requirements from user communities at the early stages of systems development will usually consist of research and data gathering methods that are more exploratory and interpretive that conditional/quantitative (Cooper et al., 2007). After all, you won't know what questions you need to ask or the relevant information until you have undergone a process of open-ended learning and discovery.

A similar categorization of methods attuned towards requirements gathering is presented by Moggridge (2006). Specific methods are indicated for specific situations. Observational and recording methods are indicated when requirements or needs are initially unknown, latent, unvocalised, or problems of physical ergonomics etc. Survey, focus groups and expert opinion are useful for situations where explicit opportunities exist. Conversely the research method may be indicated by the size of the group being addressed. Statistical techniques are essential for assessing large markets or populations of users. Interpretive methods allow the researcher identify and explore concrete cases of use and associated goals.

THE RISKS OF REQUIREMENTS PROCESSES
Requirements are distillations of information, selective presentations of potentially vast unbounded wish lists and issues. The person charged with requirements capture has the power to influence the design process and therefore has a responsibility to carefully record and preserve the data from which selection is based. Creating and managing requirements involves discriminating and selecting information. It carries an onus to be truthful, fair, accurate, factual and impartial. Research data should wherever possible be preserved and indexed so that requirements can be traced back and associated with the users or situations they were generated from.

The task of requirements gathering is all the more difficult because what is learnt was previously unknown or unarticulated. A requirement is often tentative and uncertain. Furthermore, the statement of a requirement does not necessarily equate with its solution. Requirements may be functional and concrete (the product does X, Y and Z) or they may be non-functional and abstract (the product is fast, easy to use, secure). Requirements capture demands a level of attention and rigor to data gathering, storage, management and presentation equivalent to that of academic research. Whenever possible data should be verifiable, traceable, reproducible, and useful. Raw data is the body of evidence and material necessary for reasoned decision-making in the design process. It is no coincidence therefore that many of the methods and techniques for gathering and analyzing data are derived from academic research methods. Among these research tools qualitative research methods have achieved particular prominence in the technology design community (Cooper, 2004, Cooper and Reimann, 2003, Moggridge, 2006).
The value of qualitative research techniques is that they both complement quantitative methods and enable access to information that is impossible to capture via quantitative approaches. More importantly, qualitative research methods involve the researcher analyst directly with the user community and open the possibility for gestalts and insights that quantitative assessment may overlook.

CASE: A vignette on the value of qualitative research (Cooper et al. 2007)
“In one particularly illustrative example, we were asked by a client to perform a user study for an entry-level consumer video-editing product for Windows users. An established developer of video editing and –authoring software, the client had used traditional market research techniques to identify a significant business opportunity in developing a product for people who owned a digital video camera and a computer but hadn’t connected the two yet.
In the field, we conducted interviews with a dozen users in the target market. Our first discovery was not surprising – that the people who did the most taping and had the strongest desire to share edited versions of their videos were parents. The second discovery, however, was quite startling. Of the 12 people whose homes we visited, only one person had successfully connected his video camera to his computer, and he had relied on the IT guy at work to set it up for him. One of the necessary preconditions of the success of the product was that people could actually get video onto their computers to edit, but at the time it was extremely difficult to get a FireWire or video capture card functioning properly on an Intel-based PC.
As a result of four days of research, we were able to help our client make a decision to put a hold on the product, which likely ended up saving them a considerable investment.” (Cooper et al., 2007)

CONCLUSIONS
Contemporary ideas of user-driven adaptation, technology-in-use, and situated use imply that high tech development efforts remain in a perpetually unfinished state. IT, ICT and high tech products more generally are constantly being adapted for use by developers as they observe or become aware of users exploring and adapting technology to fulfil their own needs. We might think of the process of modern product development as being in a kind of constant surveillance with the producer or developers receiving a steady stream of reports from the market place, to try to better understand everyday use as well as user driven innovation. This open-ended orientation to technology requires both developers and users to be agile, as each creates conditions of possibility for shifting the meaning and functions of technology. Such agile approaches assume that technology is typically not used in isolation but that it becomes embedded in users’ everyday practices and lives. These processes of appropriation are creative processes of identifying novel forms of use or shaping the conditions of use.

REFERENCES

Gathering and managing requirements

User requirements and analysis is often considered to be the starting point for the systems development process. There are many requirements management frameworks most of which are basically templates and checklists for gathering and recording a variety of user-oriented data.

Examples:

• Atlassian's Confluence/Jira offers a sophisticated holistic model for capturing, storing, presenting requirements for future development. See this example from the Confluence/Jira tutorial (link).
• A typical/conventional/traditional requirements document; source - a student engineering project (link)

A selection of typical requirements documents.

Some musings on requirements:
Product requirements can be thought of as a rather unique kind of shopping list; a shopping list written by (more often on behalf of) the user, and written for (usually by) the developer to deliver. Taking the analogy further; the requirements shopping list is for a shop where the shelves are initially empty because the things the user wants haven’t been made yet. Alternatively there may be something on the shelf that approximates what the user wants but it’s not quite right and needs to be customized. To compound this seemingly odd situation we may also find that product requirements may be written (created) by someone who is neither the customer (user) nor the designer (developer). In this situation those charged with requirements capture have a lot of responsibility and power to influence the design process. Product requirements lie between the user and the designer and act as a communication device between the two. The requirements document is merely a representation of a potentially unbounded set of product requirements therefore the process used to create the representation is perhaps more significant that the document itself.

Links of interest?
https://svpg.com/the-end-of-requirements/

Supporting Exercise?

Share a file or post a link to an example of an actual product requirements page or document. Note. Examples out there might be titled 'pitch' or 'design' (e.g. a game design document), rather than 'requirements' but the substance of the subject matter described in the document will be features, needs, constraints etc.

Guindon Design Experiment

Based on designing and building a cantilever beam using spaghetti and sticky tape.

Detailed protocol available at:

https://managingdesignanddevelopment.blogspot.com/2010/09/guindon-design-activities.html

Goal:

Develop an understanding of empirical design and development work as it unfolds over time.

Method:

The experiment will run for ~30 minutes.
At 1 minute marks check one or more activities you underwent in the last period from the following list:

5. "Scenario level"
4. "Requirement level"
3. "High level solution"
2. "Medium level solution"
1. "Low level solution"
0. "Key ideas (lightbulb moments)"

 Results:

At the end of the experiment take a photo of your cantilever experiment and post it to your socials. Potential tags...

"#designing #designprocess #designcollaboration #softwaredesign #digitalinnovation #guindondesignchallenge #thinking-aloud #researchmethods"

Discussion:

Reflect on your graph. Can you relate your findings to Raccoon's Chaos Model?

Guindon Design Notes:

In the late 1980s Raymonde Guindon designed an experiment to observe software designers at work while they were engaged in the process of creating a solution to a set problem. The software engineers, working individually, adopted a ‘thinking-aloud’ protocol and were observed directly by the researcher and videotaped for analysis.
As a consequence of these studies Guindon developed an understanding of design and development work as it unfolds over time; it is in fact a chaotic process of learning and reflection through trial and error. In essence the process of creating a solution to an ill-structured problem is itself un-structured, at least in the simplistic sense of being a planned, logical process moving from high level design to low level implementation in a smooth orderly manner. In fact the observations lead Guindon to the conclusion that software design work is largely underdetermined (Guindon, 1990).

“opportunistic decomposition is better suited to handle the ill-structuredness of design problems… top-down decomposition appears to be a special case for well-structured problems when the designer already knows the correct decomposition. .” (Guindon, 1990)

Guindon’s study demonstrated empirically that top-down design doesn’t occur as such in design work, or at least it doesn’t occur in a linear sequence from top to bottom. This has implications for lifecycles and frameworks that impose linear or staged phase structures based on the concept of top-down design-to-development processes.

Reference: Guindon, R. (1990) Designing the Design Process: Exploiting Opportunistic Thoughts. Human-Computer Interaction, 5, 305-344.

Writing a precis

The commentary or précis of a reading/article conveys what you understood, learnt, and how you might use the knowledge. Consider expanding your commentary to include a section for a critical or analytical interpretation, i.e. what is the intention of the authors, who is the audience, how valid are the claims?

Style #1. Simple Q&A pattern...

• Q: Who are the authors?
• Q: What is your key takeaway from this article?
• Q: Can you highlight one key quote for the audience?
• Q: What do you think is the value or importance of this article?
• Q: So where are we today in terms of this topic?
• Q: another question?

Style #2. Written paragraph or section pattern...

• Sentence 1:Name of author, genre, and title of work, date in parenthesis; a rhetorically accurate verb (such as "claims," "argues," "asserts," "suggests"); and a THAT clause containing the major assertion or thesis statement in the work. 
• Sentence 2: An explanation of HOW the author develops and supports the thesis, usually in chronological order. 
• Sentence 3: A statement of the author's apparent purpose, followed by an "in order to" phrase. 
• Sentence 4: A significant quote from the paper used in a sentence.

Writing tips:

Focus on the article being reviewed, not so much on other readings, books, articles etc.

Please do identify key quotes from the article. These a short statements or at most a sentence or two that distil some essential aspect of the article. A key quote is used: to point to the authors' evidence or claims; to make a justification for your own arguments; to act as a foundation for your own ideas. However, there must be clear delineation between the authors' content and your use of it.

• For quotes: use quotation marks followed by cite.
• For paraphrasing: follow with cite.
• For extracts and transformations like lists and tables: explain source followed by cite.
• When reviewing, do not quote the author's quotes of other authors. Instead, quote an original passage written by the author of the article you are reviewing.

Please use double quotation marks and page number to identify "the quoted text" p. 23. You could apply one of the standard citation methods if you like e.g. Harvard style:

• (Surname et al., Publication Year, p.#)
• (Surname et al., Publication Year, pp.#-range)

Something like "some text" (Surname et al, 2033, p.7), or similar according to the citation standard required for the document.

Relax and play

Take timeout and feel free to play some of the Board Games in the Smurfit restaurant...

1. Ticket to Ride
2. Saboteur
3. Hanabi
4. Ubongo – (completely in german)
5. Bang!
6. Love Letter 
7. Dixit
8. Carcassone + Carcassone Expansion box 
9. Stratego
10. Risk

Exercise: World Café (and Word Cloud)

World Café exercise

The World Café: A method to learn from and harness the power of groups.

https://theworldcafe.com/key-concepts-resources/world-cafe-method/

4/5 people per group max

#1. Find words. Individual activity – 7” quiet, write notes.

#2. Share. Connect, cluster, name, move, organize, link

  This happens on a shared wall/whiteboard

#3. Assign champions.

#4. 10”/round. Others move around over three rounds. Champion gives each a voice. Champion facilitates a conversation. Champion helps scribe.

  First move; second move; third move.

#5a. 3” Champion synthesis. Devise a single key discovery and debrief to the wider group.

#5b. Alternate synthesis. 30" arrange a coffee break for the participants and the champions to come together to harness the materials that were gathered during the group rounds. Create a distillation/synthesis by drawing, sketching, writing, and/or typing up, a sense-making dialogue or cartoon or flow diagram or combination of all, to present by way of debriefing to the wider group. Will you ask them to create a combined synthesis or separate?

Word Cloud. An extra step, visualising words

What are the challenges of digital innovation?

Enter your response in the following form.

https://forms.gle/mUuGq7pTcnXxhs2T7

Admin to paste and share the results in the word cloud generator (free and requires no download).

Exercise: NATO conference proceedings

approximate time: 1h15'

Objectives (5') - S1-S2 

To produce and present a critical analysis
To conduct independent research
To experience and reflect on group work

Transition (5')

Identify groups

Provide readings

Group work starts (20') - S3 

• Critically evaluate one of the articles provided.
• Preparing a group review (without visual props). 
• 20' to read and prepare of which 5' quiet time.

Without giving too much guidance up front simply ask the students to spend 20 mins critically evaluating their article and then each group can present their critique for a maximum of 2 mins (without powerpoint) to the rest of the class. After all of the presentations there should then be approximately 15 mins for the lecturer to provide some feedback based on the key touch-points.

Presentation delivery (30')
10x 2-minute presentations followed by one quick Q&A on the subject matter

---

Articles

Document 1: From Naur & Randell "NATO Conference on Software Engineering," 1968 (link)

1. Group Discussion: User Requirements (pp 40-43)
2. Group Discussion: The Nature of Software Engineering (pp 19-23)
3. Group Discussion: Software Engineering Management and Methodology (pp 24-30)
4. Group Discussion: Design and Production in Software Engineering (pp 31-32)
5. Keynote Speech, by A. J. Perlis (pp 135-137)
6. S. Gill: Thoughts on the sequence of writing software (186-187)

Document 2: From Randell & Buxton "Software Engineering Techniques; NATO Conference Proceedings," 1969 (link)

1. Group Discussion: Case Histories; A Survey (pp 41-42)
2. Group Discussion: Apollo Programming Support (pp 43-47)
3. Group Discussion: The Electronic Switching System (pp 48-50)
4. Group Discussion: Software Engineering Education (pp 61-66)
5. R. M. Needham: Software engineering techniques and operating system design and production (pp 111-113)
6. R. M. Needham and J. D. Aron: Software engineering and computer science (pp 113-114)
7. J. I. Schwartz: Analyzing large-scale system development (pp 122-136)

---

Thematic Discussion (10")

• What can we take from these passages?
• What were they concerned about in the 1960s?
• Are old concerns still contemporary issues? Why?
• What did they think would solve these problems? Based on what knowledge?
• Was there agreement as to the problems? The solutions?
• Is the work we do today and the ways we manage it essentially different or only accidentally different?
• In what ways is the work then and now similar?

---

Class Discussion (10')

What is critical analysis?

• Evaluating
• Subjective
• Persuasion
• Evidence
• Scientific
• Political 

Did typical roles arise? What were they?

• Manager
• Timekeeper
• Recorder/checker
• Sceptic
• Big boss
• Lurker
• Facilitator

Were roles assigned or volunteered for?
Did people change roles? Why?

Did each member have a voice, make an impact?

What was the dynamic (over time)?

• Initial analysis
• Independent research
• Synthesis
• Chaos
• Lost in the desert
• A cavalry charge

Did the group...

• Present a brief and cogent piece? 
• Add value - illustrate, relate etc?
• Reflect and critically evaluate?

e.g. (t: critical analysis - 30s) / (t: synopsis + 30s) + insightful analysis + impactful conclusions.

Wrap up - S4 - S5 - S6 - S7 - S8 - S9 - S10

---

Further reading

A process for combining self-directed and group-based learning can be organised as follows. Note, groups should adapt and modify the steps to suit the round style and conditions. (Schwartz et al., 2001) 

1. First encounter a problem ‘cold’, without doing any preparatory study in the area of the problem.
2. Interact with each other to explore their existing knowledge as it relates to the problem.
3. Form and test hypotheses about the underlying mechanisms that might account for the problem (up to their current levels of knowledge).
4. Identify further knowledge gaps or learning needs for making progress with the problem.
5. Undertake self-study between group meetings group to satisfy identified learning needs.
6. Return to the group to integrate the newly gained knowledge and apply it to the problem.
7. Repeat steps 3 to 6 as necessary.
8. Reflect on the process and on the content that has been learnt.

The Seven Jump or Maastricht process offers a similar template for structuring small-group tutorial learning. (Grave et al., 1996)

1. Clarify unknown terms or concepts in the problem description.
2. Define the problem(s). List the phenomena or events to be explained.
3. Analyse the problem(s). 
• Step 1. Brainstorm. Try to produce as many different explanations for the phenomena as you [can] think of. Use prior knowledge and common sense.
• Step 2. Discuss. Criticize the explanations proposed and try to produce a coherent description of the processes that, according to what you think, underlie the phenomena or events.
4. Formulate learning issues for self-directed learning.
5. Fill the gaps in your knowledge through self-study.
6. Share your findings with your group and try to integrate the knowledge acquired into a comprehensive explanation for the phenomena or events. Check whether you know enough.

Alternatively the MacMaster ‘triple jump’ represents three main stages for student-driven problem investigation: initial analysis, independent research, and synthesis. Each stage consists of a series of activities (not necessarily taking place in sequence).

1. Initial analysis: identify problems, explore extant knowledge, hypothesise, identify knowledge gaps
2. Independent research: research knowledge gaps
3. Synthesis: present findings – relating them to the problem(s), integrate learning from others, generate a synthesis, self-assessment of learning process, repeat ‘triple jump’ if needed.

References:

• GRAVE, W. S., BOSHUIZEN, H. P. A. & SCHMIDT, H. G. (1996) Problem based learning: Cognitive and metacognitive processes during problem analysis. Instructional Science, 24, 321-341.
• SCHWARTZ, P., MENNIN, S. & WEBB, G. (Eds.) (2001) Problem-Based Learning: Case studies, experience and practice, London, Routledge.

Technology journalist John Sterne launches the Irish Tech Archives

The Irish Tech Archives https://techarchives.irish/ 

"The core mission of TechArchives is to create and preserve the stories surrounding Ireland’s long and convoluted relationship with information technology.

Karlin Lillington of the Irish Times writing about the announcement of this fantastic initiative by Technology journalist John Sterne to record personal accounts of veterans of the industry recalling the history of Irish Tech.

 https://www.irishtimes.com/business/technology/net-results-the-tale-of-ireland-s-technology-history-1.2695334

Exercise: The (daily) standup

1. What did I accomplish yesterday?
2. What will I do today?
3. What obstacles are impeding my progress?

The combination of regular stand-up meetings, story-cards and a task-board seem to be a particularly powerful enabler for the other elements of Agile teams. What we termed the ‘task-board process’ confers both visibility and responsibility

“What I think about the task-board, and I feel it myself, is that the engineers have a hell of a lot more autonomy now. In what they do, there is much less control about what we do now, we pick things off the board, ourselves and we drive them ourselves right through to the end”

To start off we adopted the following guidelines influenced by one of Dublin's early Extreme Programming consultancy groups EXoftware - now part of emergn. The guidelines helped us start to get into the habit of having a daily stand-up and to avoid some of the weeds that inevitably sprout up around new organisational practices when they appear to start succeeding. The 'weeds' are things that others try to piggyback, slipstream, coat-tail onto anything that succeeds in getting people in one place and paying attention.

Rules of stand-up meetings as follows…

• One of the team calls the rest to convene the stand up meeting
• Everyone gathers at the task board (conference remote people in by phone or skype)
• No interruptions
• Keep story to less than 60 seconds
• Start story with StoryName
• Walk up to the board and point to the cards you are referring to.
• Ask for assistance if required
• All dialogs to expand in break-out meetings after the stand-up
• Each person must stand up to the task board and indicate the story-card they are describing
• Adoption of stand-up meetings will negate the need for the weekly opening meeting
• The last stand-up meeting of each week is the “big meeting”. It will be followed by the weekly group meeting (operational focus as per previous opening and closing meetings).

People can look at our rules and find faults or things to improve, indeed so did we and as a consequence the practice or flow of the meetings changes over time, sometimes for the good sometimes for the worse. Reading through Martin Fowler's reflections on daily stand-up meetings offers numerous points of comparison and critique for us to contrast, interpret and perhaps change our own practice (see link below).

Further Reading
Martin Fowler has given considerable thought to the dynamics, quirks, irritations, failings and other aspects of programmer stand-up meetings (martinfowler.com).

Exercise: Experiment with a research method

Exercise: Experiment with a research method

Materials:
Copies of the research method protocol template. Use the template to structure your experiment with one or more of the research protocols based on the cards from IDEO.

Choose at least one protocol, but even better if you try more than one, if you have the interest or time available.
Context:
Identify an object or context for field work. Possible examples include:

• Stairs
• Doorways
• Queuing in a café
• Anywhere with a queue
• Parcel drop services
• Anything with a touchscreen
• UCD directional maps online and on-campus
• Dublin Bus website 
• ATM machine 
• Train/LUAs ticketing 
• Supermarket Self-service checkout machines 
• Copier machines at the Copi-Print system 
• UCard Top-up 
• Printer
• Kindle or other E-reader 
• Brightspace (from student’s perspective) 
• Alarm System 
• Airport Self-service Checkin 
• Tesco’s Online Shopping website 
• UCD Library catalogue 

Time:
Schedule field study observations over several days.

Instructions:

1. Write your name and the research context on the form
2. Investigate and propose your own version of the procedure for the research method assigned to you. Base this procedure on readings or your own creative extrapolation of the description on the IDEO methods card.
3. Conduct a trial run of the procedure on yourself first, then involve another willing participant, and another...
4. Make hand-written notes, photos and recordings for later analysis.
5. Keep a record of evidence gathered.
6. Make an archive of data/records (paper sketches and notes, digital folder, website).
7. Present your findings and analysis at the next class.

Evaluation:

In class discussion of findings. 

Questions to ask:

1. Did you write the method protocol (recipe/steps)?

2. Did you identify a method reference (source literature/inspiration)?

3. Have provided a sample/example of data you gathered?

4. Did you try to analyse or interpret the data? (may need to apply some tool or technique)

5. What was learnt from it all? Did I learn anything? Can you decide if the method is feasible, practical, suitable for your project?

Research Protocols

Names and descriptions for the IDEO Method Cards (IDEO, 2003)
Ideo (2003) IDEO Method Cards: 51 Ways to Inspire Design. William Stout.

LEARN

Research Method	Description
Activity Analysis	A description of project relevant dynamics: actions, interactions, tasks, and objects of achieving goals.
Affinity Diagrams	Represent or diagram clustering of design elements with activities, goals, obstacles. Proximity, dependence and relationships
Anthropometric Analysis	Human factors or ergonomics to assess project relevant use factors.
Character Profiles	Personas. Archetypes of real people with real goals, lifestyle, behaviour, identities.
Cognitive Task Analysis	List, summary of all available sensory inputs decision points and actions.
Competitive Product Survey	Collect, compare, and conduct evaluations of extant and competitive products.
Cross-Cultural Comparisons	Personal accounts of differences in situations, behaviour and artefacts in different national or cultural settings.
Error Analysis	Capturing the things that actually go wrong in the project relevant setting.
Flow Analysis	The flow of information in existing and/or new system.
Historical Analysis	Identify trends and cycles of product use, customer behaviour, market, and practice. Relate to timeless goals
Long-Range Forecasts	Narratives of future scenarios complete with social and technological trends to predict behaviours.
Secondary Research	Summary analysis of existing sources and 3rd party data on project relevant areas.

LOOK

Research Method	Description
A Day in the Life	Catalogue a person’s whole day without necessarily focusing on project relevant aspects.
Behavioural Archaeology	Look at use, wear, the detailed arrangement or organisation of use objects in their use setting.
Behavioural Mapping	Map position, movement, and use of space over time.
Fly on the Wall	Observe in context without interfering.
Guided Tours	Ask the user to guide you through project relevant spaces and activities.
Personal Inventory	Ask the user to reflect on and describe the things they view as important or significant
Rapid Ethnography	Participate and experience it first-hand with the user for as long as possible.
Shadowing	Tag-along with people through the day.
Social Network Mapping	Notice the relationships between people, groups. Look for identity, profession, culture, and connections.
Still Photo Survey	Build up a visual record of key use and interaction moments over time.
Time-Lapse Video	A way of summarising activity over time, use of time, space, and location.

ASK

Research Method	Description
Camera Journal	A written and visual diary of project relevant circumstances and activities.
Card Sort	Organise cards spatially in ways that make sense. To expose mental models of device or system.
Cognitive Maps	Create a map of an existing or virtual space. The pathways they know and navigate, mental models.
Collage	Self created collage of arranged images. Used to help verbalise complex or unvocalised themes.
Conceptual Landscape	Sketch and juxtapose social and behavioural constructs from participants. People’s mental model.
Cultural Probes	A visual journal for participants to build up themselves. A self generated reflection. Gathered and compared across many participants.
Draw the Experience	Ask participants to visualise and draw the experience in their own way, with their own associations, order, relationships, theories.
Extreme User Interviews	Evaluate (ask) users at extreme ends of market (early adopters, power users, beginners) to highlight their issues.
Five Whys?	Ask “why?” in response to five consecutive answers. Expose/uncover deeper attitudes perceptions.
Foreign Correspondents	Elicit inputs from others (snowball sample) to build up varied cultural and environmental contexts.
Narration	Users perform tasks and achieve goals while describing aloud. Talk aloud protocol. Stream of consciousness.
Surveys & Questionnaires	Targeted questions to assess design, usage, interaction characterises and perceptions of users.
Unfocus Group	Gather a range of tools or materials and get diverse user group to create things relevant to the design or project.
Word-Concept Association	Users associate words with design. Cluster user perceptions to evaluate design features & concepts. Value and priority.

TRY

Research Method	Description
Behaviour Sampling	Snapshot people’s activities at different times. How do pervasive products intrude in your lives?
Be Your Customer	What is it like to purchase your product? Actual experience of searching, buying, consuming, disposing.
Bodystorming	Act out scenarios with many people, using space, place, sequence, queues, etc. Test performance in context.
Empathy Tools	Experience the range of users capability for involvement under real conditions.
Experience Prototype	Mock up a rough but workable prototype to simulate the experience of using the new product.
Informance	Act out scenarios observed in the field to interpret and analyse in the lab. Builds shared understanding and good for solution formation.
Paper Prototyping	Rapid paper based mock-ups that are manipulated to demonstrate functionality. To articulate design concepts with users.
Predict Next Year’s Headlines	Involve users in future design possibilities. Futuristic, wishing, unmet needs. Help separate what is needed now from what can wait.
Quick-and-Dirty Prototyping	Assemble a very rough mock-up of a new feature or product to help start and refine a design.
Role-Playing	Identify actors/stakeholders involved in design use, enact real activities in real or imagined context.
Scale Modelling	Simulation technique to test arrangements of space, place, context.
Scenarios	A character-rich story, to communicate (simulate) and test a plausible story in probable context.
Scenario Testing	Show depictions of possible future scenarios. Share reactions, refine concepts.
Try it Yourself	Like Microsoft’s famous ‘eating our own dog food’ being the first user.

General comments on doing field research

On learning about methods

On methods; I recommend you identify a published research example and adapt it. And look carefully at the example's own bibliography too. The methods literature is broad. You will take ownership ofdiscovering your methods' background, select an informing literature and decide for yourself about method suitability etc.

Each of us is expected to delve into the literature on the different methods available and on adapting these findings to our own topic. The reasons for this are various but primarily because research methods for product design and scientific studies is a very broad area with huge variation and there is no way to teach methods without limiting or biasing your own research journey (see comments below on the overuse of surveys and questionnaires). Educationally I expect each researcher to identify, study and develop expertise in the methods they employ. By taking ownership of this you become authentically involved in the process of professional and scholarly research. You will discover the background to various methods, select the publications that inform your own research design, decide through experimentation what methods are suitable and feasible for your own research context.

Some comments by way of general orientation to a research exercise:

• Be curious and open to discovery
• Goal oriented analysis often yields interesting findings.
• Be open to identifying insights, deep insights, particularly "what I learnt".
• Look out for the unexpected, things that are puzzling, aberrations. 
• Self-reporting can be hugely insightful, this can be researcher self-reporting or respondent/subject self-reporting.
• Don't get side-tracked by 'annoyances' when there is an elephant in the room.
• When making recommendations or advising on solutions
• Contrast with comparable experiences, goals, solutions from other industries.
• Contrast with completely incomparable solutions from completely different settings.
• Avoid 'selling' a new tech system as if it is a holy grail or nirvana solution. All tech systems have their failings, you may simply be changing the source of your pain, not taking it away.
• There is huge value in a quick-dirty prototype. 
• Use paper sketches, paper prototypes, mock-ups, from low to medium to high fidelity.
• Design ideas (sketches, mock-ups, prototypes) are used for feedback, for learning.
• If a design proposal is quite narrow, if so then it must be very focused, well argued, and insightful.
• NO DANCING BEARS! For an explanation see Alan Cooper's book "The Inmates are Running the Asylum" (search link)
• Respect your research subjects' privacy-identity
• It's generally seen as good practice to anonymise your interviewees if putting information in the public domain unless it is absolutely necessary for some reason.
• I recommend redacting personal identities from reports and papers.
• Be aware of copyright and attribution
• You cannot post other people's copyright material online.
• If you 'copy-paste' content from someone else make sure you surround it with quotation marks and include a citation or attribution.
• Do please make posts on your websites that are your own writing. 
• Be aware of and comply with copyright law and conventions for fair use, attribution etc.

On Surveys and Questionnaires

I am not particularly interested in student projects that use surveys or questionnaires as the main or even as supporting research methods. 

Surveys and questionnaires are a tertiary research method and rely on the laws of large numbers and/or access to large representative population pools. Survey/questionnaire methods are explicitly closed-ended in that they imply limited sets of allowable data/responses. Unfortunately these methods are rarely accompanied with a necessary introspection by the researcher of prior assumptions, or reflection/statement of researcher's epistemological/ontological assumptions that may skew or predispose the design of data collection to produce implicit results. A research design may in fact generate (produce, reify) the very objects it seeks to reveal.

It is also rare that we see well justified and well designed questionnaires or surveys. In fact these methods require that that the researcher has already studied (through literature review) or actually conducted extensive primary empirical research and/or carried out medium scale studies before resorting to survey/questionnaire. The findings of prior studies provide the justifications for determining what, why and who to ask. There will be clear connections between foundational research findings and the very design of a survey/questionnaire instrument. The content and sequence of each question is there for a valid research reason. The method is an inherently closed style of questioning and is therefore a kind of 'forcing function'. It produces limited responses responses to focused questions and therefore carries the risk that you will only detect what you expect to find, the method is quite literally self-determining and often results in poor science.

The design of surveys and questionnaires must needs be subjected to scrutiny, refinement and quality checking in order to avoid problems ranging from avoiding leading questions through to establishing construct validity. For example, is it essential for the research that you capture the respondents gender? Why? Is this justified? If so how many categories are you providing? Male/Female? Does the respondent have to answer or can they opt out? If so how does this affect the data and your analysis of it?

Surveys and questionnaires have their uses, for example where the concepts are well defined, not subject to misunderstanding (ref. construct validity), and the questions being addressed can usefully be asked of a large sample population. Therein lies my last bugbear with survey/questionnaire, sample size. A survey with 8 respondents is technically useless unless the whole population is 8. Statements based on small sample sizes are likewise in general useless unless you are sampling small absolute populations. Plus, survey responses are often presented in ways that mislead us as to their significance, for example, "100% of survey respondents answered yes" creates a different impression from "8 survey respondents answered yes".

So when does a survey sample size reach statistical relevance? The answer to this question depends on the margin of error threshold you seek to pass. Notice when talking about populations we generally refer to the sample size, a subset of the population. The implication being that we don't expect to be able to contact every single member of a target population, merely a subset. So what sample size do we need to reach to make reasonable inferences about the entire population? The margin of error or confidence level desired are in fact functions of population size; see "survey population/sample size calculator" for further information.

Survey population-sample size calculators

The Difference between Citing References and Field Data

References and citations should relate to the subject matter being written about. Field data and interviews etc. gathered from a field study site does not constitute reference material in the same way; in this case a quote from the data is simply data not a citation. Similarly survey results are only included where relevant to the arguments or analyses being made, the entire survey with responses should not be appended to the paper. Research instruments and the gathered results may, if desired, be included in an appendix at the end of the document.

Personal Reflection

1-page appendix item "Personal Reflection"

The aim of a personal reflection is to give the student an opportunity to relate a personal understanding of the course. To highlight not just the described learning outcomes but also draw attention to challenges and areas of difficulty. Think of it as a statement of what you determine to be the key learnings and contribution of the course. It can be critical, highlighting gaps etc. Ultimately it is a personal statement of your own (perhaps new or changed) perspective on the subject, new understandings, difficulties, and insights.

Grading criteria:

The Personal Reflection is authentic, critical, supported by evidence and descriptive, conveying your own personal learning insights.

• A single page, approximately 500 words.
• Is it original? Is it your own work? (this is a basic requirement)
• Are the insights and learning described authentic? Does it honestly communicate your personal learning on taking this class?
• Is it critical? Critique isn't a bad thing. It challenges your own and others, even the subject itself. Consider prior understandings, misunderstanding, new knowledge, or changes in understanding?
• Are statements supported with examples? For example, comments or reflections on the homework tasks, the project, themes and subject matter?
• Core concepts? At the very best the reflection offers a compelling account of the significance of some of the key ideas arising in the course.

On the subject of research writing, methods, data, analysis, assumptions...

On the subject of research writing, methods, data, analysis, assumptions... Some ideas for structuring your own research ideas, and useful tips about research design.

Working Title: Initially, phrase a research question as the title of the paper (you can change it later).
Abstract: Restate and expand on the research question in the abstract (you can change it later when you have analysed your findings).

Research Access: Make good use of your personal access to your contacts, projects or companies, past or present for providing data.

Gather data and working backwards:
What I mean by this is that you will almost certainly end up changing/revising the research question as you go along, and the abstract will need to be revised at the end. The working title and abstract written at the beginning was just a first stab at the paper.
A research question is a prerequisite and precursor to a research project. Having a question puts the focus on a few things; the kinds of data you might expect to gather which could be anything from interviews, observations, documentary/documents, to literature review/desk research etc. etc.
A (tentative) research question usually implies particular kinds data, so ask, what kind of data does the question imply? What constitutes evidence for the phenomena under investigation? What data will (or might) provide the kinds of evidence that could be used to make justifiable statements about the research context?
A clear idea of the kind of evidence and data sought leads us to consider the types of data capture methods (research methods) that can be used to produce data or the evidence sought. This in turn indicates a typical overall research model, research design and research approach. Sometimes a programme of research will involve many of these approaches. For example, literature review, a distinctive family of desk based text research, is generally a prerequisite for all other research activities; evident in an introduction or positioning section of a paper, or constitute the whole research project itself. In very general terms, the gamut of research designs or models includes - but is not limited to:

Essay: Purely theoretical, elaborating a conjecture, speculation, conceptual, philosophical argumentation, thought experiment, word games, word play, rhetoric, logicism, formalism, constructionism.
Review, meta-analysis of prior research, selective literature review, systematic literature review.
Empirical: descriptive, idiosyncratic, case-study, naturalistic observation, questionnaire survey.
Correlational-causal: studies adopting systems model views, inputs factors, outcomes, case-control study, observational study, survey, structural equation modelling.
Statistical meta-analytic: research derived from other research, findings based on aggregations of other research.
Semi-experimental: research styled on intervention in settings, field experimentation not amenable to laboratory control, quasi-experiment, trials, living labs.
Experimental: classical closed system model of scientific discovery, reproducible experiment, blind experimentation, random assignment experiments.

(partially derived from the Wikipedia article on Research Design):

These various research models involve or require differing philosophical commitments and assumptions or beliefs around the nature reality and the world (worldview, ontology), and of the nature of knowledge (epistemology is the theory and nature of knowledge; objective, subjective, social). Usually it is sufficient to simply acknowledge the stance adopted for the purpose of the research, be it: interpretive, critical, critical realism, naive realism, post-modern, deconstruction, construction, positivist, aesthetic, utilitarian, speculative, qualitative, quantitative.

Improve the draft:

Commence your paper with an introductory/positioning piece that incorporates a short selective review of relevant recent literature critically addressing the topic areas you are working with.

Provide a presentation to peers or colleagues if possible, talking through your ideas, the data, your analysis and findings. Doing so will inevitably help refine how well you communicate the story, your message, convincing evidence, the key points, highlight your contribution and findings.

Use a Template

Start using a scientific conference template for writing up. I recommend you write using a journal or conference template. Two examples in the IS field below...

This is an example from the ECIS 2015 conference...

(http://www.ecis2015.eu/participation/submissions.html)

This is a pointer to the live HICSS template in Word format or in LaTeX format on the HICSS conference website. See the "Author Instructions" section...

(https://hicss.hawaii.edu/authors/)

The HICSS conference style in LaTeX is provided/shared via Overleaf. Overleaf is an online writing/editing service that uses LaTeX, the defacto standard for most scientific research writing...  

Further reading

Links to various related conference templates below:
A LaTeX and a Word version of an ECIS Template - from Muenster 2015 - recommended!!
A LaTeX version of an ICIS Template - Ryan Schuetzler - a bit gnarly. A Word version of an ICIS Template - it's Word :-(