2.4. Perform Rigorous Data Analysis

Begin with descriptive statistics. Calculate: ● Mean task completion time and standard deviation to understand average performance and variability. ● Success and error rates expressed as percentages, highlighting task difficulty. ● Click-path frequency to uncover common navigation patterns.

Visualize these metrics through bar charts, histograms, or box plots (if your course permits basic charting), labeling axes clearly and providing explanatory captions.

In this first part, we focus on quantitative analysis. This involves using statistical measures to understand how well users are performing tasks in your design.

1. Mean task completion time: This is the average time it takes for users to complete a task. Calculating the standard deviation helps us see how much variation there is in the times recorded; a small standard deviation means times are similar, while a larger one indicates a wider spread.
2. Success and error rates: These metrics help us quantify how often users succeed or fail at tasks. By expressing these as percentages, we can better gauge the difficulty of tasks. For instance, if 70% of users complete a task successfully, we know it’s generally user-friendly.
3. Click-path frequency: This metric analyzes the steps users take in navigation. It helps us understand which paths are most common and whether there are any unnecessary detours.
4. Visualization: Finally, these statistics are often presented using visual aids, such as bar charts or histograms, which make them easier to understand and share with others.

Think of a race: the mean completion time is like the average time taken by all runners to finish. If you see that one runner took a lot longer than everyone else (a high standard deviation), it raises questions about what might have gone wrong for them. For success rates, if one runner stumbled and fell (an error), it’s crucial to know how often that happens to improve the race conditions going forward. Visualizing these results, much like showing a medal tally in the Olympics, makes it easier for everyone to see who’s excelling and where improvements are needed.

Review all transcripts, observation notes, and open-ended survey responses. Conduct open coding by assigning initial labels to discrete ideas (e.g., “unclear icon,” “slow loading”). Progress to axial coding by grouping labels into broader categories (“navigation issues,” “system feedback”). Finally, use selective coding to identify core themes that explain the majority of user experiences.

Create an affinity diagram on a whiteboard or digital canvas: cluster similar codes spatially and draw connections to illustrate relationships between themes (e.g., navigation difficulties linked to information architecture inconsistencies).

The qualitative analysis provides insights into user experiences beyond what numbers can tell us. Here’s how to approach it:

1. Review Materials: Start by reading through transcripts, notes, and survey feedback. This helps you gather all user sentiments to find key issues.
2. Open Coding: Break down responses into basic ideas or codes. For instance, if several users mention that an icon is confusing, label that point as ‘unclear icon.’
3. Axial Coding: Group these codes into broader themes. If multiple comments relate to navigation, that can become a category like ‘navigation issues.’
4. Selective Coding: This step focuses on identifying the main themes that describe the overall user experience. It allows you to see overarching problems that many users face.
5. Affinity Diagram: Visual tools, such as an affinity diagram, help you organize and see connections between similar ideas, making it easy to identify systemic issues in your design.

Imagine you are editing a film. First, you watch all the footage and jot down exciting moments (open coding). Then, you categorize these moments into themes like ‘action sequences’ and ‘dramatic pauses’ (axial coding). Finally, you zero in on recurring elements like suspense and intensity across the film (selective coding). An affinity diagram would be like creating a storyboard where you group related scenes together, helping everyone understand how different parts of your film influence the overall story.

Develop a traceability matrix: a table that cross-references each original specification item (rows) against findings (columns) and indicates pass/fail status, severity rating, and recommended actions. For example: Design Met Severit Observation Recommendation

Checkout within 90 No High Average time 125 Simplify steps; add progress indicator Accessible color Yes Low Contrast ratios meet N/A This matrix ensures every requirement is accounted for and prioritized.

A traceability matrix serves as a critical tool for ensuring that all your design specifications are met during evaluation. Here’s how it works:

1. Design Specifications: Start by listing each requirement or goal from your design specification down the side of a table.
2. Findings: In the columns, indicate whether each requirement was met based on your data analysis, marking each one with 'Yes' or 'No.'
3. Severity Rating: Assign a level of severity to any issues found, which helps prioritize what to address first; a higher rating indicates a more critical problem.
4. Recommendations: Finally, outline specific actions to resolve any unmet requirements. For instance, if users struggled with a check-out process, recommend simplifying the steps and adding progress indicators. This matrix provides a clear overview of where your design stands and what needs attention.

Consider planning a vacation: you have your checklist of essential destinations and activities (design specifications). On your trip, you note which places you visited and any issues you faced (findings). If you didn’t go to a must-see museum, you might rate the importance of that and jot down a recommendation to include it on future trips. A traceability matrix is like this checklist, ensuring all key experiences are accounted for and where you need to improve your future plans.

--