Monday, 6 November 2023

Gathering and managing requirements

User requirements and analysis is often considered to be the starting point for the systems development process. There are many requirements management frameworks most of which are basically templates and checklists for gathering and recording a variety of user-oriented data.

Examples:
  • Atlassian's Confluence/Jira offers a sophisticated holistic model for capturing, storing, presenting requirements for future development. See this example from the Confluence/Jira tutorial (link).
  • A typical/conventional/traditional requirements document; source - a student engineering project (link)
hightechrequirements
A selection of typical requirements documents.
Some musings on requirements:
Product requirements can be thought of as a rather unique kind of shopping list; a shopping list written by (more often on behalf of) the user, and written for (usually by) the developer to deliver. Taking the analogy further; the requirements shopping list is for a shop where the shelves are initially empty because the things the user wants haven’t been made yet. Alternatively there may be something on the shelf that approximates what the user wants but it’s not quite right and needs to be customized. To compound this seemingly odd situation we may also find that product requirements may be written (created) by someone who is neither the customer (user) nor the designer (developer). In this situation those charged with requirements capture have a lot of responsibility and power to influence the design process. Product requirements lie between the user and the designer and act as a communication device between the two. The requirements document is merely a representation of a potentially unbounded set of product requirements therefore the process used to create the representation is perhaps more significant that the document itself.

Links of interest?
https://svpg.com/the-end-of-requirements/

Supporting Exercise?
Share a file or post a link to an example of an actual product requirements page or document. Note. Examples out there might be titled 'pitch' or 'design' (e.g. a game design document), rather than 'requirements' but the substance of the subject matter described in the document will be features, needs, constraints etc.

Wednesday, 11 October 2023

Guindon Design Experiment

Based on designing and building a cantilever beam using spaghetti and sticky tape.
Detailed protocol available at:
https://managingdesignanddevelopment.blogspot.com/2010/09/guindon-design-activities.html

Goal:

Develop an understanding of empirical design and development work as it unfolds over time.

Method:

The experiment will run for ~30 minutes.
At 1 minute marks check one or more activities you underwent in the last period from the following list:
5. "Scenario level"
4. "Requirement level"
3. "High level solution"
2. "Medium level solution"
1. "Low level solution"
0. "Key ideas (lightbulb moments)"

 Results:

At the end of the experiment take a photo of your cantilever experiment and post it to your socials. Potential tags...
"#designing #designprocess #designcollaboration #softwaredesign #digitalinnovation #guindondesignchallenge #thinking-aloud #researchmethods"

Discussion:

Reflect on your graph. Can you relate your findings to Raccoon's Chaos Model?

Guindon Design Notes:

In the late 1980s Raymonde Guindon designed an experiment to observe software designers at work while they were engaged in the process of creating a solution to a set problem. The software engineers, working individually, adopted a ‘thinking-aloud’ protocol and were observed directly by the researcher and videotaped for analysis.
As a consequence of these studies Guindon developed an understanding of design and development work as it unfolds over time; it is in fact a chaotic process of learning and reflection through trial and error. In essence the process of creating a solution to an ill-structured problem is itself un-structured, at least in the simplistic sense of being a planned, logical process moving from high level design to low level implementation in a smooth orderly manner. In fact the observations lead Guindon to the conclusion that software design work is largely underdetermined (Guindon, 1990).
“opportunistic decomposition is better suited to handle the ill-structuredness of design problems… top-down decomposition appears to be a special case for well-structured problems when the designer already knows the correct decomposition. .” (Guindon, 1990)

Guindon’s study demonstrated empirically that top-down design doesn’t occur as such in design work, or at least it doesn’t occur in a linear sequence from top to bottom. This has implications for lifecycles and frameworks that impose linear or staged phase structures based on the concept of top-down design-to-development processes.

Reference: Guindon, R. (1990) Designing the Design Process: Exploiting Opportunistic Thoughts. Human-Computer Interaction, 5, 305-344.

Wednesday, 20 September 2023

Writing a precis


The commentary or précis of a reading/article conveys what you understood, learnt, and how you might use the knowledge. Consider expanding your commentary to include a section for a critical or analytical interpretation, i.e. what is the intention of the authors, who is the audience, how valid are the claims?

Style #1. Simple Q&A pattern...
  • Q: Who are the authors?
  • Q: What is your key takeaway from this article?
  • Q: Can you highlight one key quote for the audience?
  • Q: What do you think is the value or importance of this article?
  • Q: So where are we today in terms of this topic?
  • Q: another question?

Style #2. Written paragraph or section pattern...
  • Sentence 1:Name of author, genre, and title of work, date in parenthesis; a rhetorically accurate verb (such as "claims," "argues," "asserts," "suggests"); and a THAT clause containing the major assertion or thesis statement in the work. 
  • Sentence 2: An explanation of HOW the author develops and supports the thesis, usually in chronological order. 
  • Sentence 3: A statement of the author's apparent purpose, followed by an "in order to" phrase. 
  • Sentence 4: A significant quote from the paper used in a sentence.

Writing tips:

Focus on the article being reviewed, not so much on other readings, books, articles etc.

Please do identify key quotes from the article. These a short statements or at most a sentence or two that distil some essential aspect of the article. A key quote is used: to point to the authors' evidence or claims; to make a justification for your own arguments; to act as a foundation for your own ideas. However, there must be clear delineation between the authors' content and your use of it.

  • For quotes: use quotation marks followed by cite.
  • For paraphrasing: follow with cite.
  • For extracts and transformations like lists and tables: explain source followed by cite.
  • When reviewing, do not quote the author's quotes of other authors. Instead, quote an original passage written by the author of the article you are reviewing.

Please use double quotation marks and page number to identify "the quoted text" p. 23. You could apply one of the standard citation methods if you like e.g. Harvard style:

  • (Surname et al., Publication Year, p.#)
  • (Surname et al., Publication Year, pp.#-range)
Something like "some text" (Surname et al, 2033, p.7), or similar according to the citation standard required for the document.


Tuesday, 19 September 2023

Relax and play

Take timeout and feel free to play some of the Board Games in the Smurfit restaurant...
  1. Ticket to Ride
  2. Saboteur
  3. Hanabi
  4. Ubongo – (completely in german)
  5. Bang!
  6. Love Letter 
  7. Dixit
  8. Carcassone + Carcassone Expansion box 
  9. Stratego
  10. Risk


Exercise: World Café (and Word Cloud)

World Café exercise

The World Café: A method to learn from and harness the power of groups.


4/5 people per group max

#1. Find words. Individual activity – 7” quiet, write notes.

#2. Share. Connect, cluster, name, move, organize, link

  This happens on a shared wall/whiteboard

#3. Assign champions.

#4. 10”/round. Others move around over three rounds. Champion gives each a voice. Champion facilitates a conversation. Champion helps scribe.

  First move; second move; third move.

#5a. 3” Champion synthesis. Devise a single key discovery and debrief to the wider group.

#5b. Alternate synthesis. 30" arrange a coffee break for the participants and the champions to come together to harness the materials that were gathered during the group rounds. Create a distillation/synthesis by drawing, sketching, writing, and/or typing up, a sense-making dialogue or cartoon or flow diagram or combination of all, to present by way of debriefing to the wider group. Will you ask them to create a combined synthesis or separate?

Word Cloud. An extra step, visualising words

What are the challenges of digital innovation?

Enter your response in the following form.

Admin to paste and share the results in the word cloud generator (free and requires no download).

Monday, 28 August 2023

Exercise: NATO conference proceedings

approximate time: 1h15'

Objectives (5') - S1-S2 

To produce and present a critical analysis
To conduct independent research
To experience and reflect on group work

Transition (5')

Identify groups
Provide readings

Group work starts (20') - S3 

  • Critically evaluate one of the articles provided.
  • Preparing a group review (without visual props). 
  • 20' to read and prepare of which 5' quiet time.
Without giving too much guidance up front simply ask the students to spend 20 mins critically evaluating their article and then each group can present their critique for a maximum of 2 mins (without powerpoint) to the rest of the class. After all of the presentations there should then be approximately 15 mins for the lecturer to provide some feedback based on the key touch-points.

Presentation delivery (30')
10x 2-minute presentations followed by one quick Q&A on the subject matter


Articles

Document 1: From Naur & Randell "NATO Conference on Software Engineering," 1968 (link)
  1. Group Discussion: User Requirements (pp 40-43)
  2. Group Discussion: The Nature of Software Engineering (pp 19-23)
  3. Group Discussion: Software Engineering Management and Methodology (pp 24-30)
  4. Group Discussion: Design and Production in Software Engineering (pp 31-32)
  5. Keynote Speech, by A. J. Perlis (pp 135-137)
  6. S. Gill: Thoughts on the sequence of writing software (186-187)

Document 2: From Randell & Buxton "Software Engineering Techniques; NATO Conference Proceedings," 1969 (link)
  1. Group Discussion: Case Histories; A Survey (pp 41-42)
  2. Group Discussion: Apollo Programming Support (pp 43-47)
  3. Group Discussion: The Electronic Switching System (pp 48-50)
  4. Group Discussion: Software Engineering Education (pp 61-66)
  5. R. M. Needham: Software engineering techniques and operating system design and production (pp 111-113)
  6. R. M. Needham and J. D. Aron: Software engineering and computer science (pp 113-114)
  7. J. I. Schwartz: Analyzing large-scale system development (pp 122-136)

Thematic Discussion (10")
  • What can we take from these passages?
  • What were they concerned about in the 1960s?
  • Are old concerns still contemporary issues? Why?
  • What did they think would solve these problems? Based on what knowledge?
  • Was there agreement as to the problems? The solutions?
  • Is the work we do today and the ways we manage it essentially different or only accidentally different?
  • In what ways is the work then and now similar?

Class Discussion (10')

What is critical analysis?
  • Evaluating
  • Subjective
  • Persuasion
  • Evidence
  • Scientific
  • Political 
Did typical roles arise? What were they?
  • Manager
  • Timekeeper
  • Recorder/checker
  • Sceptic
  • Big boss
  • Lurker
  • Facilitator
Were roles assigned or volunteered for?
Did people change roles? Why?
Did each member have a voice, make an impact?

What was the dynamic (over time)?
  • Initial analysis
  • Independent research
  • Synthesis
  • Chaos
  • Lost in the desert
  • A cavalry charge
Did the group...
  • Present a brief and cogent piece? 
  • Add value - illustrate, relate etc?
  • Reflect and critically evaluate?
e.g. (t: critical analysis - 30s) / (t: synopsis + 30s) + insightful analysis + impactful conclusions.

Wrap up - S4 - S5 - S6 - S7 - S8 - S9 - S10


Further reading

A process for combining self-directed and group-based learning can be organised as follows. Note, groups should adapt and modify the steps to suit the round style and conditions. (Schwartz et al., 2001) 
  1. First encounter a problem ‘cold’, without doing any preparatory study in the area of the problem.
  2. Interact with each other to explore their existing knowledge as it relates to the problem.
  3. Form and test hypotheses about the underlying mechanisms that might account for the problem (up to their current levels of knowledge).
  4. Identify further knowledge gaps or learning needs for making progress with the problem.
  5. Undertake self-study between group meetings group to satisfy identified learning needs.
  6. Return to the group to integrate the newly gained knowledge and apply it to the problem.
  7. Repeat steps 3 to 6 as necessary.
  8. Reflect on the process and on the content that has been learnt.
The Seven Jump or Maastricht process offers a similar template for structuring small-group tutorial learning. (Grave et al., 1996)
  1. Clarify unknown terms or concepts in the problem description.
  2. Define the problem(s). List the phenomena or events to be explained.
  3. Analyse the problem(s). 
    • Step 1. Brainstorm. Try to produce as many different explanations for the phenomena as you [can] think of. Use prior knowledge and common sense.
    • Step 2. Discuss. Criticize the explanations proposed and try to produce a coherent description of the processes that, according to what you think, underlie the phenomena or events.
  4. Formulate learning issues for self-directed learning.
  5. Fill the gaps in your knowledge through self-study.
  6. Share your findings with your group and try to integrate the knowledge acquired into a comprehensive explanation for the phenomena or events. Check whether you know enough.
Alternatively the MacMaster ‘triple jump’ represents three main stages for student-driven problem investigation: initial analysis, independent research, and synthesis. Each stage consists of a series of activities (not necessarily taking place in sequence).
  1. Initial analysis: identify problems, explore extant knowledge, hypothesise, identify knowledge gaps
  2. Independent research: research knowledge gaps
  3. Synthesis: present findings – relating them to the problem(s), integrate learning from others, generate a synthesis, self-assessment of learning process, repeat ‘triple jump’ if needed.

References:

  • GRAVE, W. S., BOSHUIZEN, H. P. A. & SCHMIDT, H. G. (1996) Problem based learning: Cognitive and metacognitive processes during problem analysis. Instructional Science, 24, 321-341.
  • SCHWARTZ, P., MENNIN, S. & WEBB, G. (Eds.) (2001) Problem-Based Learning: Case studies, experience and practice, London, Routledge.

Wednesday, 25 January 2023

Technology journalist John Sterne launches the Irish Tech Archives

The Irish Tech Archives https://techarchives.irish/ 
"The core mission of TechArchives is to create and preserve the stories surrounding Ireland’s long and convoluted relationship with information technology.

Karlin Lillington of the Irish Times writing about the announcement of this fantastic initiative by Technology journalist John Sterne to record personal accounts of veterans of the industry recalling the history of Irish Tech.

 https://www.irishtimes.com/business/technology/net-results-the-tale-of-ireland-s-technology-history-1.2695334

Thursday, 1 December 2022

Exercise: The (daily) standup


  1. What did I accomplish yesterday?
  2. What will I do today?
  3. What obstacles are impeding my progress?

The combination of regular stand-up meetings, story-cards and a task-board seem to be a particularly powerful enabler for the other elements of Agile teams. What we termed the ‘task-board process’ confers both visibility and responsibility
“What I think about the task-board, and I feel it myself, is that the engineers have a hell of a lot more autonomy now. In what they do, there is much less control about what we do now, we pick things off the board, ourselves and we drive them ourselves right through to the end”
To start off we adopted the following guidelines influenced by one of Dublin's early Extreme Programming consultancy groups EXoftware - now part of emergn. The guidelines helped us start to get into the habit of having a daily stand-up and to avoid some of the weeds that inevitably sprout up around new organisational practices when they appear to start succeeding. The 'weeds' are things that others try to piggyback, slipstream, coat-tail onto anything that succeeds in getting people in one place and paying attention.

Rules of stand-up meetings as follows…

  • One of the team calls the rest to convene the stand up meeting
  • Everyone gathers at the task board (conference remote people in by phone or skype)
  • No interruptions
  • Keep story to less than 60 seconds
  • Start story with StoryName
  • Walk up to the board and point to the cards you are referring to.
  • Ask for assistance if required
  • All dialogs to expand in break-out meetings after the stand-up
  • Each person must stand up to the task board and indicate the story-card they are describing
  • Adoption of stand-up meetings will negate the need for the weekly opening meeting
  • The last stand-up meeting of each week is the “big meeting”. It will be followed by the weekly group meeting (operational focus as per previous opening and closing meetings).
People can look at our rules and find faults or things to improve, indeed so did we and as a consequence the practice or flow of the meetings changes over time, sometimes for the good sometimes for the worse. Reading through Martin Fowler's reflections on daily stand-up meetings offers numerous points of comparison and critique for us to contrast, interpret and perhaps change our own practice (see link below).

Further Reading
Martin Fowler has given considerable thought to the dynamics, quirks, irritations, failings and other aspects of programmer stand-up meetings (martinfowler.com).


Wednesday, 16 November 2022

Exercise: Experiment with a research method

Exercise: Experiment with a research method

Materials:
Copies of the research method protocol template. Use the template to structure your experiment with one or more of the research protocols based on the cards from IDEO.

Choose at least one protocol, but even better if you try more than one, if you have the interest or time available.
Context:
Identify an object or context for field work. Possible examples include:
  • Stairs
  • Doorways
  • Queuing in a café
  • Anywhere with a queue
  • Parcel drop services
  • Anything with a touchscreen
  • UCD directional maps online and on-campus
  • Dublin Bus website 
  • ATM machine 
  • Train/LUAs ticketing 
  • Supermarket Self-service checkout machines 
  • Copier machines at the Copi-Print system 
  • UCard Top-up 
  • Printer
  • Kindle or other E-reader 
  • Brightspace (from student’s perspective) 
  • Alarm System 
  • Airport Self-service Checkin 
  • Tesco’s Online Shopping website 
  • UCD Library catalogue 
Time:
Schedule field study observations over several days.

Instructions:
  1. Write your name and the research context on the form
  2. Investigate and propose your own version of the procedure for the research method assigned to you. Base this procedure on readings or your own creative extrapolation of the description on the IDEO methods card.
  3. Conduct a trial run of the procedure on yourself first, then involve another willing participant, and another...
  4. Make hand-written notes, photos and recordings for later analysis.
  5. Keep a record of evidence gathered.
  6. Make an archive of data/records (paper sketches and notes, digital folder, website).
  7. Present your findings and analysis at the next class.

Evaluation:
In class discussion of findings. 
Questions to ask:
1. Did you write the method protocol (recipe/steps)?
2. Did you identify a method reference (source literature/inspiration)?
3. Have provided a sample/example of data you gathered?
4. Did you try to analyse or interpret the data? (may need to apply some tool or technique)
5. What was learnt from it all? Did I learn anything? Can you decide if the method is feasible, practical, suitable for your project?

Research Protocols

Names and descriptions for the IDEO Method Cards (IDEO, 2003)
Ideo (2003) IDEO Method Cards: 51 Ways to Inspire Design. William Stout.

LEARN

Research MethodDescription
Activity AnalysisA description of project relevant dynamics: actions, interactions, tasks, and objects of achieving goals.
Affinity DiagramsRepresent or diagram clustering of design elements with activities, goals, obstacles. Proximity, dependence and relationships
Anthropometric AnalysisHuman factors or ergonomics to assess project relevant use factors.
Character ProfilesPersonas. Archetypes of real people with real goals, lifestyle, behaviour, identities.
Cognitive Task AnalysisList, summary of all available sensory inputs decision points and actions.
Competitive Product SurveyCollect, compare, and conduct evaluations of extant and competitive products.
Cross-Cultural ComparisonsPersonal accounts of differences in situations, behaviour and artefacts in different national or cultural settings.
Error AnalysisCapturing the things that actually go wrong in the project relevant setting.
Flow AnalysisThe flow of information in existing and/or new system.
Historical AnalysisIdentify trends and cycles of product use, customer behaviour, market, and practice. Relate to timeless goals
Long-Range ForecastsNarratives of future scenarios complete with social and technological trends to predict behaviours.
Secondary ResearchSummary analysis of existing sources and 3rd party data on project relevant areas.

LOOK

Research MethodDescription
A Day in the LifeCatalogue a person’s whole day without necessarily focusing on project relevant aspects.
Behavioural ArchaeologyLook at use, wear, the detailed arrangement or organisation of use objects in their use setting.
Behavioural MappingMap position, movement, and use of space over time.
Fly on the WallObserve in context without interfering.
Guided ToursAsk the user to guide you through project relevant spaces and activities.
Personal InventoryAsk the user to reflect on and describe the things they view as important or significant
Rapid EthnographyParticipate and experience it first-hand with the user for as long as possible.
ShadowingTag-along with people through the day.
Social Network MappingNotice the relationships between people, groups. Look for identity, profession, culture, and connections.
Still Photo SurveyBuild up a visual record of key use and interaction moments over time.
Time-Lapse VideoA way of summarising activity over time, use of time, space, and location.

ASK

Research MethodDescription
Camera JournalA written and visual diary of project relevant circumstances and activities.
Card SortOrganise cards spatially in ways that make sense. To expose mental models of device or system.
Cognitive MapsCreate a map of an existing or virtual space. The pathways they know and navigate, mental models.
CollageSelf created collage of arranged images. Used to help verbalise complex or unvocalised themes.
Conceptual LandscapeSketch and juxtapose social and behavioural constructs from participants. People’s mental model.
Cultural ProbesA visual journal for participants to build up themselves. A self generated reflection. Gathered and compared across many participants.
Draw the ExperienceAsk participants to visualise and draw the experience in their own way, with their own associations, order, relationships, theories.
Extreme User InterviewsEvaluate (ask) users at extreme ends of market (early adopters, power users, beginners) to highlight their issues.
Five Whys?Ask “why?” in response to five consecutive answers. Expose/uncover deeper attitudes perceptions.
Foreign CorrespondentsElicit inputs from others (snowball sample) to build up varied cultural and environmental contexts.
NarrationUsers perform tasks and achieve goals while describing aloud. Talk aloud protocol. Stream of consciousness.
Surveys & QuestionnairesTargeted questions to assess design, usage, interaction characterises and perceptions of users.
Unfocus GroupGather a range of tools or materials and get diverse user group to create things relevant to the design or project.
Word-Concept AssociationUsers associate words with design. Cluster user perceptions to evaluate design features & concepts. Value and priority.

TRY

Research MethodDescription
Behaviour SamplingSnapshot people’s activities at different times. How do pervasive products intrude in your lives?
Be Your CustomerWhat is it like to purchase your product? Actual experience of searching, buying, consuming, disposing.
BodystormingAct out scenarios with many people, using space, place, sequence, queues, etc. Test performance in context.
Empathy ToolsExperience the range of users capability for involvement under real conditions.
Experience PrototypeMock up a rough but workable prototype to simulate the experience of using the new product.
InformanceAct out scenarios observed in the field to interpret and analyse in the lab. Builds shared understanding and good for solution formation.
Paper PrototypingRapid paper based mock-ups that are manipulated to demonstrate functionality. To articulate design concepts with users.
Predict Next Year’s HeadlinesInvolve users in future design possibilities. Futuristic, wishing, unmet needs. Help separate what is needed now from what can wait.
Quick-and-Dirty PrototypingAssemble a very rough mock-up of a new feature or product to help start and refine a design.
Role-PlayingIdentify actors/stakeholders involved in design use, enact real activities in real or imagined context.
Scale ModellingSimulation technique to test arrangements of space, place, context.
ScenariosA character-rich story, to communicate (simulate) and test a plausible story in probable context.
Scenario TestingShow depictions of possible future scenarios. Share reactions, refine concepts.
Try it YourselfLike Microsoft’s famous ‘eating our own dog food’ being the first user.

General comments on doing field research

On learning about methods

On methods; I recommend you identify a published research example and adapt it. And look carefully at the example's own bibliography too. The methods literature is broad. You will take ownership ofdiscovering your methods' background, select an informing literature and decide for yourself about method suitability etc.

Each of us is expected to delve into the literature on the different methods available and on adapting these findings to our own topic. The reasons for this are various but primarily because research methods for product design and scientific studies is a very broad area with huge variation and there is no way to teach methods without limiting or biasing your own research journey (see comments below on the overuse of surveys and questionnaires). Educationally I expect each researcher to identify, study and develop expertise in the methods they employ. By taking ownership of this you become authentically involved in the process of professional and scholarly research. You will discover the background to various methods, select the publications that inform your own research design, decide through experimentation what methods are suitable and feasible for your own research context.

Some comments by way of general orientation to a research exercise:

  • Be curious and open to discovery
    • Goal oriented analysis often yields interesting findings.
    • Be open to identifying insights, deep insights, particularly "what I learnt".
    • Look out for the unexpected, things that are puzzling, aberrations. 
    • Self-reporting can be hugely insightful, this can be researcher self-reporting or respondent/subject self-reporting.
    • Don't get side-tracked by 'annoyances' when there is an elephant in the room.
  • When making recommendations or advising on solutions
    • Contrast with comparable experiences, goals, solutions from other industries.
    • Contrast with completely incomparable solutions from completely different settings.
    • Avoid 'selling' a new tech system as if it is a holy grail or nirvana solution. All tech systems have their failings, you may simply be changing the source of your pain, not taking it away.
    • There is huge value in a quick-dirty prototype. 
    • Use paper sketches, paper prototypes, mock-ups, from low to medium to high fidelity.
    • Design ideas (sketches, mock-ups, prototypes) are used for feedback, for learning.
    • If a design proposal is quite narrow, if so then it must be very focused, well argued, and insightful.
    • NO DANCING BEARS! For an explanation see Alan Cooper's book "The Inmates are Running the Asylum" (search link)
  • Respect your research subjects' privacy-identity
    • It's generally seen as good practice to anonymise your interviewees if putting information in the public domain unless it is absolutely necessary for some reason.
    • I recommend redacting personal identities from reports and papers.
  • Be aware of copyright and attribution
    • You cannot post other people's copyright material online.
    • If you 'copy-paste' content from someone else make sure you surround it with quotation marks and include a citation or attribution.
    • Do please make posts on your websites that are your own writing. 
    • Be aware of and comply with copyright law and conventions for fair use, attribution etc.

On Surveys and Questionnaires

I am not particularly interested in student projects that use surveys or questionnaires as the main or even as supporting research methods. 

Surveys and questionnaires are a tertiary research method and rely on the laws of large numbers and/or access to large representative population pools. Survey/questionnaire methods are explicitly closed-ended in that they imply limited sets of allowable data/responses. Unfortunately these methods are rarely accompanied with a necessary introspection by the researcher of prior assumptions, or reflection/statement of researcher's epistemological/ontological assumptions that may skew or predispose the design of data collection to produce implicit results. A research design may in fact generate (produce, reify) the very objects it seeks to reveal.

It is also rare that we see well justified and well designed questionnaires or surveys. In fact these methods require that that the researcher has already studied (through literature review) or actually conducted extensive primary empirical research and/or carried out medium scale studies before resorting to survey/questionnaire. The findings of prior studies provide the justifications for determining what, why and who to ask. There will be clear connections between foundational research findings and the very design of a survey/questionnaire instrument. The content and sequence of each question is there for a valid research reason. The method is an inherently closed style of questioning and is therefore a kind of 'forcing function'. It produces limited responses responses to focused questions and therefore carries the risk that you will only detect what you expect to find, the method is quite literally self-determining and often results in poor science.

The design of surveys and questionnaires must needs be subjected to scrutiny, refinement and quality checking in order to avoid problems ranging from avoiding leading questions through to establishing construct validity. For example, is it essential for the research that you capture the respondents gender? Why? Is this justified? If so how many categories are you providing? Male/Female? Does the respondent have to answer or can they opt out? If so how does this affect the data and your analysis of it?

Surveys and questionnaires have their uses, for example where the concepts are well defined, not subject to misunderstanding (ref. construct validity), and the questions being addressed can usefully be asked of a large sample population. Therein lies my last bugbear with survey/questionnaire, sample size. A survey with 8 respondents is technically useless unless the whole population is 8. Statements based on small sample sizes are likewise in general useless unless you are sampling small absolute populations. Plus, survey responses are often presented in ways that mislead us as to their significance, for example, "100% of survey respondents answered yes" creates a different impression from "8 survey respondents answered yes".

So when does a survey sample size reach statistical relevance? The answer to this question depends on the margin of error threshold you seek to pass. Notice when talking about populations we generally refer to the sample size, a subset of the population. The implication being that we don't expect to be able to contact every single member of a target population, merely a subset. So what sample size do we need to reach to make reasonable inferences about the entire population? The margin of error or confidence level desired are in fact functions of population size; see "survey population/sample size calculator" for further information.
Survey population-sample size calculators

The Difference between Citing References and Field Data

References and citations should relate to the subject matter being written about. Field data and interviews etc. gathered from a field study site does not constitute reference material in the same way; in this case a quote from the data is simply data not a citation. Similarly survey results are only included where relevant to the arguments or analyses being made, the entire survey with responses should not be appended to the paper. Research instruments and the gathered results may, if desired, be included in an appendix at the end of the document.

Personal Reflection

1-page appendix item "Personal Reflection"

The aim of a personal reflection is to give the student an opportunity to relate a personal understanding of the course. To highlight not just the described learning outcomes but also draw attention to challenges and areas of difficulty. Think of it as a statement of what you determine to be the key learnings and contribution of the course. It can be critical, highlighting gaps etc. Ultimately it is a personal statement of your own (perhaps new or changed) perspective on the subject, new understandings, difficulties, and insights.

Grading criteria:

The Personal Reflection is authentic, critical, supported by evidence and descriptive, conveying your own personal learning insights.
  • A single page, approximately 500 words.
  • Is it original? Is it your own work? (this is a basic requirement)
  • Are the insights and learning described authentic? Does it honestly communicate your personal learning on taking this class?
  • Is it critical? Critique isn't a bad thing. It challenges your own and others, even the subject itself. Consider prior understandings, misunderstanding, new knowledge, or changes in understanding?
  • Are statements supported with examples? For example, comments or reflections on the homework tasks, the project, themes and subject matter?
  • Core concepts? At the very best the reflection offers a compelling account of the significance of some of the key ideas arising in the course.

Tuesday, 8 November 2022

On the subject of research writing, methods, data, analysis, assumptions...

On the subject of research writing, methods, data, analysis, assumptions... Some ideas for structuring your own research ideas, and useful tips about research design.

Working Title: Initially, phrase a research question as the title of the paper (you can change it later).
Abstract: Restate and expand on the research question in the abstract (you can change it later when you have analysed your findings).

Research Access: Make good use of your personal access to your contacts, projects or companies, past or present for providing data.

Gather data and working backwards:
What I mean by this is that you will almost certainly end up changing/revising the research question as you go along, and the abstract will need to be revised at the end. The working title and abstract written at the beginning was just a first stab at the paper.
A research question is a prerequisite and precursor to a research project. Having a question puts the focus on a few things; the kinds of data you might expect to gather which could be anything from interviews, observations, documentary/documents, to literature review/desk research etc. etc.
A (tentative) research question usually implies particular kinds data, so ask, what kind of data does the question imply? What constitutes evidence for the phenomena under investigation? What data will (or might) provide the kinds of evidence that could be used to make justifiable statements about the research context?
A clear idea of the kind of evidence and data sought leads us to consider the types of data capture methods (research methods) that can be used to produce data or the evidence sought. This in turn indicates a typical overall research model, research design and research approach. Sometimes a programme of research will involve many of these approaches. For example, literature review, a distinctive family of desk based text research, is generally a prerequisite for all other research activities; evident in an introduction or positioning section of a paper, or constitute the whole research project itself. In very general terms, the gamut of research designs or models includes - but is not limited to:
Essay: Purely theoretical, elaborating a conjecture, speculation, conceptual, philosophical argumentation, thought experiment, word games, word play, rhetoric, logicism, formalism, constructionism.
Review, meta-analysis of prior research, selective literature review, systematic literature review.
Empirical: descriptive, idiosyncratic, case-study, naturalistic observation, questionnaire survey.
Correlational-causal: studies adopting systems model views, inputs factors, outcomes, case-control study, observational study, survey, structural equation modelling.
Statistical meta-analytic: research derived from other research, findings based on aggregations of other research.
Semi-experimental: research styled on intervention in settings, field experimentation not amenable to laboratory control, quasi-experiment, trials, living labs.
Experimental: classical closed system model of scientific discovery, reproducible experiment, blind experimentation, random assignment experiments.
(partially derived from the Wikipedia article on Research Design):

These various research models involve or require differing philosophical commitments and assumptions or beliefs around the nature reality and the world (worldview, ontology), and of the nature of knowledge (epistemology is the theory and nature of knowledge; objective, subjective, social). Usually it is sufficient to simply acknowledge the stance adopted for the purpose of the research, be it: interpretive, critical, critical realism, naive realism, post-modern, deconstruction, construction, positivist, aesthetic, utilitarian, speculative, qualitative, quantitative.

Improve the draft:
Commence your paper with an introductory/positioning piece that incorporates a short selective review of relevant recent literature critically addressing the topic areas you are working with.

Provide a presentation to peers or colleagues if possible, talking through your ideas, the data, your analysis and findings. Doing so will inevitably help refine how well you communicate the story, your message, convincing evidence, the key points, highlight your contribution and findings.

Use a Template

Start using a scientific conference template for writing up. I recommend you write using a journal or conference template. Two examples in the IS field below...

This is an example from the ECIS 2015 conference...

This is a pointer to the live HICSS template in Word format or in LaTeX format on the HICSS conference website. See the "Author Instructions" section...

The HICSS conference style in LaTeX is provided/shared via Overleaf. Overleaf is an online writing/editing service that uses LaTeX, the defacto standard for most scientific research writing...  

Further reading

Links to various related conference templates below:
A LaTeX and a Word version of an ECIS Template - from Muenster 2015 - recommended!!
A LaTeX version of an ICIS Template - Ryan Schuetzler - a bit gnarly. A Word version of an ICIS Template - it's Word :-(

Wednesday, 19 October 2022

Exercise: Writing an Academic Article

Use technology to manage a bibliography, referencing and in-text citations.
This exercise introduces MS Word style sheets and References...

The following illustrative exercise uses the ECIS template on Google Drive (link).

1. Copy the template from the shared folder to your own computer.
2. Rename your file using the following pattern "Surname_MyResearchProject_YYYY.doc".
For example my own paper is going to be "HigginsEtAl_WorkingInVirtualLight_YYYY.doc". I have used the author convention "SurnameEtAl" as there are three or more authors.
3. From the MS Ribbon "Home" open the Styles Panel. The "Current style" field show the current text style wherever the cursor is in your Word file. Alternately navigate to top menu "Format>Style" for similar.
The current style at cursor location

4. Select section 1 "First level heading" and rename it "Introduction"

5. Paste and match formatting using following unformatted text as new paragraphs for section 1
Critical management studies appreciate that products and services, produced with technologies, by organisations, and the involvement of users, rely upon "actors having formal and symbolic resources for the exercise of... systematic forms of control over organisational participants, and indirectly over other groups and non-human objects"\citep{AlvDee2000aa}. The techniques and skill of management for producing digital goods and services (through software, hardware and systems) at its best aims to resolve this through the delicate, democratic balancing of power, control of resources, shaping of work culture, and leadership \citep{Kid1981aa}. The following brief introduction to the literature positions this study within the broad field of management information systems and seeks to inform further creative, design, and development initiatives.This study looks at...


6. Confirm that the paragraph current style is "Basic text"

7. Select the following text and change its style to "Subtle Emphasis". You many need to filter the style list selection at the bottom of the styles window.
“actors having formal and symbolic resources for the exercise of... systematic forms of control over organisational participants, and indirectly over other groups and non-human objects”
8. With the MS Ribbon "References" active...
Select the LaTeX citation command \citep{AlvDee2000aa} and replace it with the MS Word equivalent citation, i.e. from the References Ribbon select "Insert Citation". You may need to create a new entry in the Word file's Citations collection
Create a new citation source record

Enter a new source record as follows (n.b. add ", p. 7" to be thorough)
MS Word's new citation source editor

9. Similarly replace the LaTex/Bibtex command for Kidder "\citep{Kid1981aa}" with
Adding Tracy Kidder's Soul of a New Machine to the citation list in your MS Word document
10. Now regenerate the bibliography at the end of the draft paper by navigating to the "References" section, and selecting "Bibliography" to insert a new bibliography. You'll end up deleting the previous copy. You'll also need to reapply the style "Reference" to this text
n.b. the bibliography style-type (Harvard - Anglia) and the insert "Bibliography" command


For further background refer to the notes on the term paper at:
https://managingdesignanddevelopment.blogspot.com/2015/04/term-paper-and-presentation-guidelines.html

Writing styles: The term paper is written in an academic style, presenting your background reading, method, research, analysis, theorising and critiquing aspects, for example of the history, situation, processes etc of a particular sourcing context. Consider identifying an exemplary paper that you aspire to emulate or to compare your own paper with.
You must use the specified scientific conference template for the term-paper. Choose between either the LaTeX or Word template from the ECIS 2015 conference. Copies are available on (Google Drive link). By using and sticking with the ECIS template your paper will automatically conform with the scientific format guidelines for that conference.


Self-assessment for this exercise:

  1. Did you upload the file (e.g. word file)?
  2. Did you use the correct template?
  3. Did you add a new source record into the document?
  4. Did you insert a citation into the text of the document?
  5. Did you regenerate the bibliography/references at the end of the document? 

Cultural probe method exercise

"Some complex design challenges involve people of different cultures, languages, and societies where traditional research approaches won't help us adequately emphathize with their experiences." (Battarbee et al., 2012: p. 7)

The cultural probe method encourages participants to make a visual journal and reflection of their encounter with a culture context/experience. Probes gathered and compared across many participants offer the possibility to provide both generalisable information and deep insights (potential for broad and/or deep learning).

The following exercise provides a 'flavour' of the method:

Suggested protocol to conduct a cultural probe as follows:
  1. Take a trip to a local shop, market, or a specialist food store.
  2. Purchase 1 inexpensive food item you never tried before (1x photo).
  3. Research how to prepare it to eat or use it as an ingredient.
  4. Follow the recipe to prepare the food and test it.
  5. Write a paragraph-note summarising the exercise.
  6. Findings: Captured in (at least) a 1+ photo and (at least) a 1+ paragraph-note.
Reflections from students:
"The homework’s cultural probe was an assignment that made me reflect on the concept of design empathy and the importance of investing yourself into new contexts and open your senses. The bad designs homework has for example made me look at the world in a different ways. I now notice things from a design perspective that I otherwise wouldn’t have noticed before."

"The cultural probe exercise was not a simple cooking exercise, for a first-time chef like me; it was an interest-oriented, research-based, process designed operation with excellent results."

Thursday, 6 October 2022

Exercise: (b) Design a text-free search interface

Some hand-drawn ideas and doodles...

The goal of this exercise is to familiarise students with interface design approaches and tools. First by using pencil/pen and paper. Second by using the Balsamiq wire-frame tool to mock-up your design.

Objective
To practice creating design artifacts to display and test technology use/interaction ideas.

Material
A4 paper, pencil or pen.
Create an account on Balsamiq Cloud (no payment details are needed).
Log In to Balsamiq Cloud
https://balsamiq.cloud/#login

Instructions
1. Use paper/pencil sketch a mockup of a new kind of Internet ‘search page’ that doesn’t use text (at all)! (allocate 10")
2. Collect the sketches and display them to the class; allow the group to provide a brief commentary. (allocate 10" to display all mockups).
3. Next, use Balsamiq to create a digital version of the design. The new design may vary from the paper/pencil sketch. (allocate 15")
4. Discuss the following reflection points. (allocate 10")

Reflection
How did the quality and level of design discussion differ from the earlier exercise?
Comment on how you conceptualised or simulated the 'user' of your design.

References
Snyder, C. (2003) Paper Prototyping: The fast and easy way to design and refine user interfaces, San Francisco, CA, Morgan Kaufmann, Elsevier Science.


(other tools include: wireframe.ccapp.moqups.cominvisionapp.comsketchapp.comAxureJustinmindMarvelapp/POPapp and iRise)



Previous Student Examples

Dilyan's example: Fast booking mock up: Fast booking mock up

Whole class examples in collage:
designdiagrams02

Monday, 3 October 2022

Lean Software Development: An Agile Toolkit

Poppendieck & Poppendieck (2003) borrow the notion of 'lean' from Taiishi Ohno's Toyota Production System approach and adapt the system to software development. The guiding principle behind the idea of 'lean' software development is to eliminate waste in the form of inventory stock and piece work dwell times from any and every aspect of work possible. In borrowing lean principles and terminology, which were developed for physical goods manufacturing, how does 'lean' apply to software's 'perfect' manufacturing environment (perfection in this sense meaning to software's potential for its manufacture of exact copies of an original via digital copying)?

Waste is present in the guise of unnecessary formality and management overhead.
Extensive documentary efforts lead to waste, excessive communication (meetings) lead to waste, excessive processes, interruption, delays and defects all lead to waste in the software process.
Using the 'lean' model requires that we apply tests to work activities to establish their validity.
"A good test of the value of paperwork is to see if there is someone waiting for what is being produced." (Poppendieck & Poppendieck, 2003:5) They note that the most wasteful work around documentation efforts surrounds attempts to produce documents that "contain all of the information that the next person in line needs to know." (p7) Explicit documentation requires infeasible completeness; rely instead on tacit knowledge and direct communication between producer and receiver. The cases given in this book emphasise processes of communication, on incremental delivery, communication with customers, taking care to produce and give what is really needed, delivering simplest possible meaningful feature sets as quickly as possible.

All software is partially done, or rather is always potentially subject to change. Because software is amenable to change and its requirements are difficult to establish unambiguously software development processes are designed to reduce risk. Managing large software developments is an inherently risky process (Royce, 1970). In industry two polar extremes are taken by attempts to address risk: to establish perfect clarity prior to commencing the work, or to get feedback by producing something as early as possible (to give producer and customer a concrete artefact to explore, clarify, negotiate and make sense of. Lean takes the second position as its goal. P&P paraphrase Royce (1970) by stating that "every step in the waterfall process except analysis and coding is waste." (Poppendieck & Poppendieck, 2003:4). However Royce's own position was that "there are two essential steps common to all computer program developments, regardless of size or complexity. There is first an analysis step, followed second by a coding step... This sort of very simple implementation concept is in fact all that is required if the effort is sufficiently small and if the final product is to be operated by those who built it" (Royce, 1970: 328).

Value stream mapping is one way of describing the 'as-is' situation and preparing to change current processes and the balance of effort invested in development. Lean thinking applies the view that early feedback reduces waiting, the goal for which is to free the developer somewhat and the customer too, to give each the opportunity to clarify meaning and learn from each other what is desired, desirable, degrees of freedom and constraints. The implication of providing and giving early feedback is to recast the idea of who controls and manages development, instead control or management figure through involvement in production. Development may at this extreme be better understood as co-production. How then is this heightened communication between customer and developer attained?

References
  • Poppendieck, M. & Poppendieck, T. (2003) Lean Software Development: An Agile Toolkit Upper Saddle River, NJ, USA, Addison Wesley.
  • Royce, W. W. (1970) Managing Development of Large Scale Software Systems. IEEE WESCON. TRW. http://facweb.cs.depaul.edu/jhuang/is553/Royce.pdf

Notes:

SA 1: Is the use of a manufacturing metaphor in what is clearly a field of design and development work strictly applicable?

SA Note 1: The two fields of design and manufacturing are both conceptually and physically different. Certainly software duplication does not require the sorts of controls and efficiencies sought for in scaling production of physical goods manufacture. For digital media the finished product is absolutely the end of the story. More importantly, in the case of software a shift of focus from reproducing a digital good towards the work of designing the good is absolutely necessary. Software development's focus must be upon everything that precedes the production of the first copy of the end product. The manufacturing metaphor has limits when applied to software development however the principles of 'lean' are still relevant and applicable to software's activities: design, analysis, coding etc.