Exercise: Estimating User Stories

This exercise demonstrates task size estimation using ‘story points.’ A story point is a unit-less measure of size and complexity for a story. More important than the estimate is the discussion the group has about the story. This activity helps a group understand what tasks and proposed solutions might mean/be before attempting to implement them.

Objectives
Understand the ‘planning poker’ technique for task/story estimation for high tech development project and to obtain useful estimates for (iteration) planning.

Prerequisites
Familiarisation with ‘user stories’ and the ‘story card’ method for capturing goal driven requirements.
Planning poker cards or index cards with range of numbers (?, 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, & coffee)

Instructions
Budget up to 45 minutes to run the whole exercise (for a class size of ~50). Arrange class into groups from 4 to 7 people.

1. Allocate 1” for everyone to read through the ‘local’ rules for planning poker below.
2. Discuss planning poker rules to clarify understanding.
3. Allocate 2” for everyone to read the scenario carefully (don’t make personal estimates yet).
4. 25” for planning poker rounds.
5. 15” for debriefing discussion.

(Local) Rules for Planning Poker; adapted from (Cohn, 2006)

1. The dealer picks up the next story and asks: “Ok, how many story points will this story take?”
2. Team members think about the story and come up their own estimates. Nobody speaks the estimate just yet.
3. Members place a card from their decks face down on the table, with a value equal to or close to their estimate.
4. When everyone’s card is on the table turn the cards over at the same time.
5. Each person explains what he or she thought the story required, maximum of 3 minutes discussion.
6. Taking account of new information the cards are played again (and perhaps again) until rough convergence is achieved or one developer’s proposal and estimate is agreed upon.

Discussion points

• Did you identify duplicate entries?
• How did you handle user stories that did not follow the “As a… I want to… so that…” pattern?
• Were some user stories too ambiguous to estimate?
• How did the teams make sense of the stated requirement?
• Did the teams seek clarification from the product owner?
• How were divergent estimates treated?
• Describe your ‘feelings’ about the process? Fair, political, honest, dominated, contentious, accurate, shared, personalised, ideas, support, attack…
• How did the team arrive at a shared interpretation of the estimated value?
• Did you attach a concrete meaning for the estimate figure? Dimensionless, people, complex-standalone, similar to or like something else, hours, days, small-medium-large, easy-difficult…

Online version

The online version we'll use is courtesy of PlanningPoker.com's basic free account access.

Register at https://play.planningpoker.com/plans and select the free account option

Only one member of the group needs an account. One member of the group sets up the planning poker session. Those invited should not need to register to play.

Enter the following stories:

First a simple test run...

Organise a class night out under Covid

https://play.planningpoker.com/play/game/xItYl4XolMNg93wpjLy39cHOHssGPvYn

Title: Organise a class night out under Covid

Description: As a member of the class I want organise a social event for the whole class that is fun and interactive and accessible so that we (as a class) get to know each other better.

Acceptance Criteria:

It must...

Involve the whole class

Conform with social distance norms

Be fun

Be accessible 

Be interactive

You provide a Wedding Planning Service

The couple wants you to plan and design a whole wedding experience.

How long will it take you to put together a proposal for the following...

• A wedding experience starting on Christmas Eve.
• Options for the perfect venue - a Castle in the West of Ireland.
• With accommodation and board for approximately 200 guests (around 150 arriving from abroad).
• Arranged airport to venue transportation.
• Providing three separate marriage celebration ceremonies running over three consecutive days, each in a different style: European, Asian, South American.
• Concluding with a morning reception on the last day.
• Offering high impact evening entertainment options.
• Provide Media Service and Management (socials, streaming, video, photography).
• Full Security offering.
• Provide optional sports and tourist activities over 3.5 days.
• Arranged airport transportation for departing guests.

Bridge Design Lesson Plans

• A. Five classic bridge designs 
• B. Bridge failures!
• C. Test stuff to destruction!

Title: A. Five classic bridge designs 

Description: As a teacher I want student groups to construct models of the 5 classic bridge designs so that they have practical hands-on experience of design construction challenges.

Acceptance Criteria:

Proposed lessons to be delivered in blocks of 45 mins for a sample group of students (teenagers).

Student group completes lesson for bridge type 1  in 45 mins

Repeat lessons for bridge types 2-5 each taking 45 mins.

Photographic outputs from each lesson.

Title: B. Bridge failures!

Description: As a teacher I want students to test the weight bearing capacity of different bridge designs so that my students will appreciate the applications for different bridge types.

Acceptance Criteria: 

A lesson plan for a 45 min class

Students to generate photographic evidence and scientific data relevant to bridge performance.

Title: C. Test stuff to destruction!

Description: 

As a teacher I want students to evaluate different building materials and relate them to physical design elements  so that my students will appreciate the suitability of material choices for different bridge elements.

Acceptance Criteria: 

Each proposed lesson to be delivered in blocks of 45 mins

Students to create a table of physical characteristics

Students to create a table of load characteristics

Students to create analytical maps of physical demands for bridge design elements

References
Cohn, M. (2006) Agile Estimating and Planning, Upper Saddle River, NJ, Pearson Education.

On imposing a single estimate.

Mike Cohn from Mountain Goat Software posted a reflection on imposing single estimate for a backlog item that will necessarily involve more than one specialism. He wrote:

"During the heat of a Planning Poker session, it can be very tempting to decide to shorten some of the discussion by using multiple estimates instead of a single value. For example, if testers and programmers are having a hard time coming up with a single estimate for the full work, someone may suggest putting “programmer points” and “tester points” on each product backlog item. This is almost always a mistake."

After reading the full post I think the key idea is that you can't really trust composite estimates, made by several contributors, working on what is ostensibly the same job. Any estimate and re-estimate must be down to the judgement of the person who signs up for the item, even though it is implicit that they will necessarily need to involve other specialisms. As I see it estimation shouldn't be a back-door to task atomisation. The correct reading of estimation is that it offers a process of sharing knowledge and, in conjunction with the standup meeting, shared support. Ulimately we want a person who 'takes it on' also takes responsibility to make sure it gets done done, not 3 or 4 people all pointing fingers at the other to blame for a problem.

A link to Mountain Goat Software's estimation notes

Cost+ estimation

Estimation attempts to address the two great unknowns of complex technological systems development:

How much things cost

and

how much things cost!

Q: Why did I write "cost" twice? 
Because I want to highlight that we're interested in more than monetary cost. Cost can be thought of in terms of the resources available or needed, things like the effort (of knowledgable people), resources we already have or can alter (tools, spatial arrangement, environment), and other things we can acquire (more or newer tools, equipment, space, specialist inputs etc). Linked with this is also the idea of "opportunity cost", the hypothetical cost of not doing something as we pursue one goal over another. However the point of this post is to develop the concepts and understandings needed to perform estimation.

For the moment I'll focus on the first "how much things cost"; the second "how much things cost" is the province of the complex technical sale, value perception and price, an entirely specialised domain, skill and art in its own right.

The question here is "how much will it cost to develop? What is the effort? How long will it take? How many people for how long to produce a nominal deliverable (more or less?)?

Accepted wisdom suggests that you make a guess, double it and hope for the best. Why? Novel valuable high tech requirements are by definition unknowns. And estimating how to produce an as yet unknown, unfinished thing is necessarily an art, not a science. But we don't need to leave it there; there are some approaches and practices, that combined, enable us to make informed 'guesstimates' or scientific wild-ass guesses (SWAG) that help us overcome this bind.

For example, if a requirement is very similar to something we have encountered, resolved or done before then, all things being equal, we use the prior experience as a template to estimate the cost of addressing the new requirement.

But as the new requirement strays into the unknown then risk and uncertainty increases and estimates inexact. Yet even here rigour can be applied, knowledge, expertise and judgement employed to suggest effort, cost and timeframe estimates.

"Estimation as practice" relies on the skill, knowledge, resources and contexts of those involved in a situation. Estimations are 'situated' in the same way that other kinds of knowledge work are situated amongst context, history, place and moments in time.

The methods below offer the rigour of organisational processes, from which estimates can be produced (and refined) for planning.

---

The Planning Poker method is a Delphi technique, an approach that harnesses the collective knowledge experience, judgement and wisdom of team members.

---

Constructive Cost Model (COCOMO), currently titled COCOMO II, is a holistic method for estimating software costs and schedules using generalised software design elements as a cost units. In simple terms you start from a figure of one person-month per function point (incorporating specification, design, test, documentation etc), and refine according to the empirical results your team/organisation achieves.  COCOMO II provides three models for estimation/planning:

1. Application composition; applies complexity-weighted costs for objects like screens (with layout, buttons, input, displays), data tables/schema, reports/outputs, components, objects etc. 
2. Early design; combines function point numbers and sizes with personnel attributes.
3. Post-architecture: includes source lines of code (SLOC), function points, current architecture, code use/reuse, and project history data.

---

Exercise: Planning poker scenario

Scenario: You are a member of a joint committee of Engineers Ireland and the Irish Software Association tasked with developing instructional documentation, training videos and conducting an outreach programme for secondary schools.
Your goal: Excite students about a career in bridge design!
The volunteers are ‘sizing’ the effort needed to deliver educational exercises addressing stated requirements. Mary is facilitating the workshop. Yi and Daniel are engineers.
Yi started; “This kind of estimation technique is called ‘planning poker.’”
“What’s planning poker?” asked Daniel.
“It’s an approach to overcoming some of the difficulties surrounding estimation” said Mary. “The problem of different people anchoring their estimates in others’ figures, you know, when the first person picks a number out of the air, and you go along with it just because it’s out there”.

Yi added; “We have a list of stories captured from the requirements study.” (Table 1)
Mary explained, “Our goal is to size these stories for the next iteration. We’ll be using the story points I talked about before, remember this is just to get a general feel for the size and difficulty of the story before we talk about how long it will take.”

“Shouldn’t I be thinking in terms of how long it’ll take?” said Daniel.
“Not yet,” said May, “this way we’ll get a general feel for the size, complexity, and difficulty of the story, and maybe even agree some of the detail and what the story means.”
Yi spoke up, “each of you take the numbered cards. Think of estimates ranging between ‘small, medium, and large’. We deal with duration after we’ve got a feel for difficulty and complexity.”

Note: Estimation challenges include: anchoring behaviour, task misinterpretation, not understanding one task's relationship to other tasks.


References and Links
Inspired by ‘Agile Estimating and Planning’ (Cohn, 2006).

Planning Poker Workshop

I put this video together to introduce and document the Planning Poker workshop from 2010.

Planning Poker Estimation. from Allen Higgins on Vimeo.

Exercise: Battleship as a metaphor for Plans or Planning

This game may be all you'll ever need to know about project management :-)

You Lost! €-300.000

Exploring the relationship between action and plans, having a plan and planning, design and designing.(note: the game uses a dot "." separator for 'thousands' as per common usage in many countries) 

The Rules of Planning Battleship

• There are 5 ships hidden in the grid - one ship of size 2, two ships of size 3, one ship of size 4, and one ship of size 5.  
• Reveal the ships by taking shots at grid coordinates.
• Each shot costs 10.000.
• You have a balance of 400.000 to spend on shots (i.e. equivalent to taking 40 shots).
• Try to generate a positive return.
• You only receive a payout if a whole ship is revealed. The payout is worth the ship size * 50,000.

Step 1: Design a pre-prepared plan of attack
Individuals produce one or more up-front plans using provided blank sheets.

Step 2a: Play Planning Battleship using the 40 shots per iteration game.

1. Open the playable version of battleship on https://tech.zilverline.com/battleship/public/index.html
2. Enter each of your pre-prepared plan of attacks.
3. Record your results in the survey form (survey form here) - these will be numbers from 1 - 40 and values from -400000 and above that are multiples of 10000.

Step 2b: Play Planning Battleship using the 10 shots per iteration game.

Step 2c: Play Planning Battleship using the 1 shots per iteration game.

Step 2d: Play Planning Battleship using another number of shots per iteration game.

Step 3:
Look at the results (data spreadsheet here)


Discussion:
What is an 'iteration' with respect to the game?

When does it make sense to run a project without feedback?

What is the 'design' with respect to the game?
What is 'strategy'  ?
What is 'project planning' ?
What (if anything) might this exercise highlight for projects in general?

What is the maximum payout? *1. 

What is the maximum cash position possible? *2.

Acknowledgement to Zilver - Zilverblog The Power of Feedback in Scrum:

Zilverline developed this simple version of the Battleship game, written in Javascript, as a tool to explore ideas related to the design and development and dynamics of projects.

• The difference between Plans and Planning
• The value of feedback
• Cost and reward
• The size of an effort versus the payback in terms of information
• The game-like nature of projects (like pinball, the goal is to play again?)

"Board layouts are random and you get 40 shots in total to destroy the enemy’s fleet. After each iteration you get feedback about hits and misses. If you use iterations of 1, you are playing the regular battleship-game. Each shot costs 10.000 and when you sink a ship you get the_ships_size * 50.000 (e.g. the submarine of size 3 will reward you with 150.000). If you keep track of the balance after each iteration, you could also try to get across the idea that stopping after a few iterations might give ‘good enough’ rewards."

The game can be downloaded from the GitHub repository as a zip or you can take a look at our code. Just double click on the index.html (in the public folder) to start a game.

Source code from Zilverline - https://github.com/zilverline/battleship

*1. Max payout from hits will be 5 ships of total size 17 * 50,000 = 850,000

*2.  Max cash position will be 1,080,000 (based on spending exactly 17 shots to gain 850000 and retaining 23 shots worth 230000, resulting in a retained profit of 1080000). Note that the game engine cannot be completed in this scenario.

A reflective journal gathering learning and observations

25% Reflective Assignment. A reflective journal gathering learning and observations based on homework activities completed during term, due at the end of week 12. (Grd)

Deliverables: Write an 12-page reflection paper. The page count does include the References section. The page count does not include the Appendix section.

Graphs, images, tables are only to be indicated by caption and cross-reference text e.g., <<Table 1 goes here >> or << Figure 1 goes here>>. Copies of the actual tables, figures, images will go into the appendix. There is no page limit on the size of the appendix.

Note: Any graphs, images, tables etc. must be placed in the appendix after the References section, they are not to be included in the main body of the paper as they will distort the quantum of writing.

Template: You must use and conform with a specific scientific conference template for the paper using the European Conference on Information Systems (ECIS) template. Copies are available on (Google Drive link). You may choose between either the LaTeX or Word template.

Papers should be submitted as a ".pdf" file. Note: By using the template, you will automatically and naturally be producing a document that complies with the required format (page layout, fonts and font sizes, line spacing, heading styles, reference style etc.).

Knowledge Navigator (Apple 1987)

Considering recent announcements and commentary (e.g. Platformer link) surrounding the arrival of AI agents (early days yet) I thought it was timely to remember the Knowledge Navigator video from Apple, circa 1987. Apple shared its then vision of a "future computing system and how people might use it to navigate worlds of knowledge."

Just search for "Knowledge Navigator" or use the link below.

Knowledge Navigator video

The Guindon Design Experiment

This experiment is an adaptation of the Spaghetti Cantilever Teambuilding Exercise. Organise into large and small groups (from 4 or 5 people to 8 or 9 people).
• Each group will employ a ‘thinking aloud’ protocol as they run the experiment.
• The cantilever designers should highlight key transitions or changes in their thinking about the problem.
One person will act as the researcher, capturing a time-record of the designers’ abstraction level and time at any moment. The researcher will make judgements about the abstraction level. The researcher is not allowed to take part in the design and construction.
• Change the person in the researcher role every 5 minutes to give all team members an opportunity to contribute to the cantilever design and construction.

Resources for 10 groups

Include the following observations on the ‘Design Activity Graph.’
• Scenario thinking
• Requirement thinking
• High level solution thinking/building
• Medium level solution thinking/building
• Low level solution thinking/building
• Key ideas.
• Testing or Review.

Example of one group's design-construction activity chart

Spaghetti Cantilever Exercise
An exercise in design and coordination (adapted from Patrick Stacey’s boundary object seminar). This exercise resonates with Peter Skillman's 'Marshmallow Challenge.'

Allocate at approximately 1 hour to run this exercise. 10" setup and briefing. 30" experiment. 5" extra time. 15" debriefing.

You will need a large space with scattered desks to accommodate the exercise. Tiered lecture theaters are not suitable environments for this activity.

Aims
Practical Aim: Construct a cantilever extending from a surface, such as a table top.
Knowledge Aim: To assess the different activities people engage in open-ended problem solving.

Material
For construction each group is given a pack of spaghetti, a roll of tape, 2 sheets of A4 paper. Pens are for writing and not to be used in the structure!
Each group to be given a sheet of graph paper to capture the team's Guindon graph.
The tutor will need a timer and tape measure.

Competitive dimension/evaluation: Which group will construct the longest cantilever, - it must not touch the floor!
The tutor will need a measuring tape to measure and compare the length of the cantilevers.

Reflection
Ask each group to classify the activities they underwent (perhaps over 4 or 6 distinct kinds of activity)
Ask each group to estimate how much time they spent on each activity.
Ask the groups to reflect on how they won (or lost!) and to reflect on the contributions their different experience, backgrounds, disciplines made to the solution.
Were there collaboration problems? What boundary objects used to make sense of the challenge?
What evidence of design work is available (diagrams, prototypes, experimental trials)?

Class of 2024/25 (25" deadline announced at 20")

Group ID	Test 1	Final Span (cm)
Bananagram	OK	90cm
Whatever	OK	55cm
Noodle	OK	60cm
Goodfellows	OK	80cm
Carbonara	OK	80cm
Creators	OK	80cm
Alpha	Catastrophic collapse	10cm

Class of 2019/20

Group ID	Test 1	Final Span (cm)
NinjaTurtles	OK	86cm
Rogue1	OK	51cm
Rogue2	OK	43cm
DontDrive	OK	78cm
TeamTape	OK	109cm
Waterslide	OK	75cm
FishingRod	OK	108cm

Class of 2013/14

Group ID	Test 1	Final Span (cm)
a	OK	72cm
b	OK	44cm
c	OK	64cm
e	OK	91cm
f	OK	64cm
g	OK	20cm
h	OK	52cm
x	OK	71cm

Class of 2013/14 FT

Group ID	Test 1	Final Span (cm)
a	OK	13cm
b	OK	10cm
c	OK	70cm
e	OK	46cm
f	OK	58cm
g	OK	67cm
h	OK	59cm
i	OK	62cm
j	OK	18cm, 18cm

Class of 2012/13 

Group ID	Test 1	Final Height
a	3/4	56cm
b	OK	71cm
c	1/2	nil
e	1/2	52cm
f	1/4	81cm
g	3/4	46cm (74cm)
h	OK	83cm
i	1/4	83cm
j	3/4	51cm

Product (inc. Service) Design Resources

https://servicedesigntools.org/about

The Service Design Tools site is a wonderfully detailed collection of resources suited to evaluating user contexts, for helping you adopt a `design attitude'. The resource is supported by an on-going project aimed at bridging education, academic research and professional practices.

https://www.designmethodsfinder.com/how-it-works

The design methods finder offers another possible source of inspiration for identifying research (and design) methods to apply to your own design research project.

https://www.nngroup.com/

The Nielsen Norman Group (Jakob Nielsen, Donald A. Norman, Bruce "Tog" Tognazzini) has been behind a sustained drive to professionalise high tech design by defining a new kind of occupation; Interaction Design.

https://useit.com

This is (was) Jakob Nielsen's site for sharing thoughts on the principles of interaction design, events and ID community material.

https://asktog.com

Bruce Tognazzini's site and his take on design principles.

PNP Arcade - an outlet for creators to self-publish print and play games

 Like the title says - PNP Arcade - an outlet for creators to self-publish print and play games. Some games are free to download, others are pay per download. PNP Arcade promotes a Game of the Week, New Games, lists of publishers, and also offers a wealth of support and guidance for new Game Designers, for crafting printable aligned components, cards, boards, on cardboard and plastic, and all the other gritty details of creating a board game of publishable quality. As PNP Arcade sets out on their mission page...

PNP Arcade is looking for games that are printer friendly and can be played with common game components. Games that play well remotely are great too!

Our selection of games are curated therefore not all submissions will be added to the site.

While we wish we could send a personal reply for each submission, we unfortunately cannot guarantee a response due to the number of submissions.

PNP Arcade does not accept games that utilize any form of AI generated art at this time.

Explore the catalogue from PNP Arcade's homepage www.pnparcade.com/

 

DataCamp 2024

Dear class. I am pleased to announce that we have been granted 6 months of unrestricted access to DataCamp for self-directed learning (not for credit). Please be aware, this access is an optional additional feature and not formally part of the MIS41020 course.

DataCamp supports self-directed learning to grow and develop your own special tech power.

The "Design Development Creativity (MIS41020)" group Academic subscription starts immediately and runs for 6 months only. This opens unlimited DataCamp Learn Basic access and DataCamp Workspace Starter access for the duration of the access period.

I encourage you to register and explore what DataCamp has on offer. DataCamp provides learning tracks supporting career roles such as Data Analyst, Data Engineer, Data Scientist. It also provides technology centred tracks for acquiring competence with tools like Python, SQL, R, Google Sheets, Excel, ChatGPT, OpenAI and much much more. (Note this access does not include DataCamp Certifications).

Research Project

GENERAL GUIDELINES

0. Scope

The scope for this year's research project, the Game Design Evaluation, is outlined in the document linked here...
https://managingdesignanddevelopment.blogspot.com/2023/09/scope-for-term-paper-2023.html

1. Goal

To write up original methodological research in the style of an academic research article.

Study a product (see Scope above) using a small selection of product-design research methods chosen by you (but not surveys or questionnaires). You will apply a small number of empirical research methods to scrutinise an object or system that you have access to can interact with people who use it. Your aim is to better understand the use of the object. A secondary aim is to form an opinion on the efficacy of the chosen methods. You will generate data and findings applicable to the object (and methods).

Apply two or more chosen research methods (in addition to literature review): You will gather primary research data, applying chosen research methods empirically to technology objects, and their contexts and use situations. These activities are supported by literature review and secondary documentary data, for example scholarly articles etc. which support the main objective - conducting field research.

Tips on how to start this project...

Attempt to gather various kinds of data that you can analyse to make substantial inferences. Contrast your findings with those of other published studies (i.e. mentioned in your literature review).

• Phrase your investigation as a question - the "Working Title".  Initially, phrase a research question as the title of the paper (you can change it later).
• Identify an exemplary paper that you aspire to emulate or to compare your own paper with. 
• Write a statement of intent: This will probably evolve into the 'abstract'. Restate and expand on the research question in the abstract (you can change it later when you have analysed your findings).
• Consider Research Access: Do you have an interest in a particular product, or contacts in a company developing a product? Do you have access to a company that you would like to better understand or experience working on a project that would benefit from being studied like this?

A rubric for your proposal - ask yourself...

1. Wording of the Title stated as a question? 

2. Is/are the research target(s) identified? 

3. Is/are the research target(s) accessible? 

4. Are research methods stated? 

5. Are the research methods suited? 

6. Did you use the template? 

2. Deliverables: Term-paper plus video presentation

Term-paper: Paper May Not Exceed Ten Pages Including References. 

A 1-page Personal Reflection to be written and included as an appendix.

Appendices are not included in the page count limit.

Video presentation: The video presentation can give a concise overview of the subject matter and impact of your term-paper in a short video format (4-minute duration).

You are expected to create your own original narration and/or spoken audio content, similarly you should utilise as much of your own visual/graphical material as possible. You can of course utilise various elements sourced elsewhere (subject to licence) as background or linking pieces, e.g. diagrams, music etc. if needed as content or for artistic balance.

Grade deduction if the presentation/video has text-to-speech narration or uses 'canned animation.'

While not being graded separately from the term-paper, no presentation video results in losing half the available mark for the research project.

3. General pointer on writing...

The term paper is written in an academic style, presenting your background reading, method, research, analysis, theorising and critiquing aspects, for example of the history, situation, processes etc of a particular sourcing context. 

 
You must use the scientific conference template for the Hawaii International Conference on System Sciences (HICSS). Choose between either the LaTeX or Word template - copies of both are available on Google Drive, links below.

• Word template (https://drive.google.com/)
• LaTeX template (https://drive.google.com/)
• Papers should be submitted in Brightspace as a ".pdf" file.

Most important! Please ensure that any direct use of 3rd party material (particularly internal documentation) is presented within quotation marks or boxed or otherwise marked in some way and with the appropriate citation/identification.

4. Structure of a typical journal style research paper

 Please note, this is not a rigid structure. The scope of your project may require you to adapt the names of sections and sub-sections as needed.

Title
The title and abstract should both capture the essence of the study.

Abstract 

Introduction / Literature (positioning)
Give a brief introduction to the literature and positioning for the study.

Research Design / Methods / Context
Outline your research protocol, approach etc.

Data / Findings
Tell the story, provide the evidence, findings, account or narrative.

Analysis / Discussion
Analysis and discussion allow you to draw out the significance of what you have discovered. This is where you can apply/trial various analytical models or produce your own interpretation of the data, in order to better understand the evidence.

Conclusions
Conclusions summarise the findings concisely, often in a page. This is an overall synthesis distilling your analysis and its relevance to theory and the literature.

Bibliography/References
The bibliography/reference section is crucial to get right as it is the index to prior research and literature that you have referred to previously.

Appendices (if needed)
Use appendices to provide additional detail if necessary. Usually data samples, or intermediate representations, for example a sample of the data analysis process, coding frames, stages in the coding and summary or intermediate categories from data.

5. Grading

Grading will consider the following criteria:

1. The research project is clearly explained.
2. Critical positioning in literature.
3. Empirical work, data and evidence presented.
4. Overall quality of the document as a finished product.
5. Contributions are clear.

A brief explanation of letter grade descriptors is provided below.

Modular (letter) grades.

A+/A

• The report is suitable for submitting to conference, journal, or executive with little revision.
• There is a compelling logic to the report that reveals clear insight and understanding of the issues.
• Analytical techniques used are appropriate and correctly deployed.
• The analysis is convincing, complete and enables creative insight.
• The report is written in a clear, lucid, thoughtful and integrated manner-with complete grammatical accuracy and appropriate transitions.
• The report is complete and covers all important topics.
• Appropriate significance is attached to the information presented.
• Research gathered is summarised in some way, research and analytical methods described and discussed, evidence linked to argument and conclusions.

A-/B+

• The report may be suitable for submitting to conference, journal, or executive if sections are revised and improved.
• There is a clear logic to the report that reveals insight.
• Analytical techniques used are appropriate and correctly deployed.
• The analysis is convincing, complete and enables clear insight.
• The report is written in a clear, lucid, and thoughtful manner-with a high degree of grammatical accuracy.
• The report is complete and covers all important topics.
• Appropriate significance is attached to the information presented.

B/B-

• The report may be suitable as a discussion draft for further development or refinement.
• There is a clear logic to the report.
• Analytical techniques are deployed appropriately.
• The analysis is clear and the authors draw clear, but not comprehensive conclusions for their analyses.
• The report is written in a clear, lucid and thoughtful manner, with a good degree of grammatical accuracy.
• The report is substantially complete, but an important aspect of the topic is not addressed.
• The report may have used or presented some information in a way that was inappropriate. 

C

• The report may be suitable as a preliminary draft but needs substantial revision in a number of areas to develop further.
• The basic structure of the report is well organised but may need rebalancing.
• The content of the report may be partial, incomplete or unfinished with important aspects not addressed.
• The report used information that was substantially irrelevant, inappropriate or inappropriately deployed.
• The report’s analysis is incomplete and authors fail to draw relevant conclusions.
• The report may contain many errors in expression, grammar, spelling.

D/E

• The report may appear to be preliminary, speculative, and/or substantially incomplete.
• Whatever information provided is used inappropriately.
• The structure of the report may be inappropriate or need substantial reorganisation and/or rebalancing.
• There may be little analysis, evidence may not be founded, the findings may be inconclusive.
• The report appears to frequently use information that is substantially irrelevant, inappropriate or inappropriately deployed.
• The report may be poorly written, organised and presented.
• Frequent errors of grammatical expression.

---

Scope for the Term Paper

Game Design Evaluation

A design evaluation of an indie boardgame released at Spiel 22, Spiel 23, Spiel 24 e.g. see the Spiel website - SPIEL ESSEN - https://www.spiel-essen.de/en/  Each student to select a recent game published within the last 3 years. No mainstream games or videogames. Cross check the game details on BoardGameGeek BBG https://boardgamegeek.com/.

The term-paper will be a practical project based on your personal-plus-friends evaluation of an indie (independent/small company) board game as the 'whole product'. You will consider overall design, its playability, potential for improvement, its digital aspects or digital potential, and product variants such as expansions, spin-offs etc. 

Audience: Create a document that tells a story that people such as the designer, investor and publisher would benefit from learning about what you learnt by using and observing others using the product. It could be a useful input for a journalist to use in preparation for writing a product review.  Content and structure may include any/all of the following...

Pitch warm-up

---------

In a sentence “It’s like…”

Single player, multiplayer, puzzle, builder, collaborative, cooperative, competitive, open-world, sandbox etc?

Family/Adult, player age, number of players, time to play. Casual vs campaigning etc. 

Background research

----------

Inspiration? 

Genre?

Related titles in this genre? 

Published influences, related and similar titles (even, if you look for it, references in the game literature!!!!) 

Design structural elements

--------

Game design elements? Gamifications?

Outcomes? Win/lose? Display/sharing?

The application of emotion principles?

The application of the uncertainty principles?

Narrative backdrop constructed by the design

Narrative potential constructed by the players

Complexity, scope? Too much, too little?  

Developer perspective

------

Paper prototype or mock-ups?

Any playtest feedback?

Comment on ease to ‘onboard’ new players? Simple version, complex version. House rules.

Identified ideal player types, market segment? Appeals to who?

Market perspective

-------

Potential to adapt or expand, levels, extent, add-ons, expansions? Is it a platform.

Market size of equivalent or similar titles?

Route to market? (publisher, self-publish, Kickstarter, Gamefound, crowdfunded)

Packaging/presentation? (box, components, online, platforms) look and feel.

Countries/languages? Cultural fit. Suggest markets. 

Product Production: Value, costs, price, effort

--------

Price-point/Pricing? Cost to design/develop?

Cost to service/operate?

IP or licensing questions?

Potential to rebrand or repurpose the ‘engine’ to another genre?

Software Engineering at Google

 

(copied from the abseil website - a project involving some of the authors of the SWE Book)

In March, 2020, we published a book titled “Software Engineering at Google” curated by Titus Winters, Tom Manshreck and Hyrum Wright.

The Software Engineering at Google book (“SWE Book”) is not about programming, per se, but about the engineering practices utilized at Google to make their codebase sustainable and healthy. (These practices are paramount for common infrastructural code such as Abseil.)

We are happy to announce that we are providing a digital version of this book in HTML free of charge. Of course, we encourage you to get yourself a hard copy from O’Reilly if you wish.

Click on the link below to access a HTML copy of the book.

https://abseil.io/resources/swe-book