scratchpad是个好东西之May20：493 review-Testing

Usability inspection methods

discount usability: cheap, fast, easy to use

functionality at right time in right place in right form from user's view

Cognitive Walkthrough:

1. focused on: tasks, interface, learnability
2. may be: w/o real use, w prototype or mockup
3. procedure: 

1. define inputs (user, task, mockup/implementation, action seq)
2. convene analysis (facilitator, scribe(prob,assumptions), participates)
3. walk through action seq of tasks (why do, goal, feedback)
4. record critical info (credible success/failure story, assumptions, probs)
5. revise interface to fix prob

4. need: system representation, tasks, detailed action script
5. do: step through, 4 questions:

1. what right effect to achieve?
2. notice available correct action?
3. associate correct action w effect?
4. see progress after correct action?

6. + fast, cheap, put designers/developers in the shoes of user
7. - false positives, hard to measure severity of issues, focused on learning not skilled

Heuristic Evaluation:启发式

1. for each screen/page/state: 

1. inspect dialogue elements
2. compare w list of usability principles (Nielsen's)
3. consider other principles/results

2. Nielsen's:

1. visibility of status
2. match sys & real world
3. user control & freedom
4. consistency & standards
5. error prevention (eg. bday type in)
6. recognition rather than recall (eg. file path visible)
7. flexibility & efficiency of use (eg. shortcuts, tailor with macro)
8. aesthetic & minimalist design (eg. google main page)
9. help recognize, diagnose, recover from errors (error box)
10. help & doc

3. phases:

1. pre coordination (告知)
2. evaluation (free->specific, list of prob, assess severity)
3. severity rating (priority of prob, frequency+impact+persistence, discuss)
4. debriefing (discuss, results)

4. vs user testing:

1. + faster, no interpreting actions
2. - miss prob, false positives

5. + fast, cheap, find most severe issues, related to both learning & skilled use
6. - false positives, 3-5 skilled usability engineers, hard to separate issues from different users

Usability Testing

inspection, HE, analysis, experiment, talking, observing, comparing

1. formal model (certainty vs cost)
2. inspection method: fast, limited (eg. A-B test)
3. experiment: deep, narrow
4. survey: large number, limited why
5. talking: deep, not true
6. observing: deep, slow (best think aloud)
7. usage logs: useful, objective, no why 

formal experiments: 

1. hypothesis
2. independent variable (levels)
3. dependent variable 控制变量
4. speed, error rates, task completion, limited, missing subjective measures
5. A-B online experiments (to see which single var is most effective in increasing a response rate or other desired outcome. sufficient size/numbers)

A-B test: 

1. 一组两系统or两组一系统
2. split tasks (control for learning effect, A->B and B->A)

survey: large number, limited subjective data

1. goal
2. design instrument
3. pilot试点, sample
4. deploy部署
5. analyze

observation: 

look for critical incidents (error, long stall, unexpected path/event/use, distress)

Hawthorne effect 被观察者知道自己被观察时改变行为的倾向

-> think-aloud variant:

1. constructive interaction (2ppl w monitored conversation)
2. co-discovery learning (beginner & intermediate users)

-> think-aloud: let them talk, record

1. pre checklist
2. intro to script
3. paperwork (consent form同意书)
4. pre- questionnaire
5. tasks
6. logging instruments/recording device
7. prompts to user
8.  post-questionnaire
9. debrief script

structured observation:

1. process/workflow data
2. critical incidents
3. think aloud
4. get context, reasonable timeframe, small number

in-field observation: real environ, collaborative aspects, takes time, small number