Open method · v0.1 (phase 0)
How Stumblepoint measures
Every number is an estimate with an interval, never a verdict. The change between variants is the main product; absolute numbers are indicative.
1. Deterministic playback
An agent walks the prototype like a first-time user: it follows the words that best match the goal (information scent), fills required fields with test data, skips optional ones, and dismisses popups. The path it takes is saved as a locked flow. Measurement always replays that locked path at a fixed viewport, with reduced motion, so the same build gives the same numbers. Target: under 2 % variation between runs.
2. Effort: Keystroke-Level Model
| Operator | Typical | Range | Used for |
|---|---|---|---|
| K keystroke | 0.28 s | 0.20–0.40 | Each character typed |
| P point | Fitts' law | ±20 % | 0.23 + 0.166·log2(D/W + 1) from the previous pointer position to the target |
| B button | 0.10 s | 0.08–0.12 | Press and release (2 per click) |
| H homing | 0.40 s | 0.30–0.50 | Hand between mouse and keyboard |
| M mental | 1.35 s | 1.0–1.7 | Before each decision, each field, each optional field the user must decide to skip |
| S scroll | 1.0 s | 0.7–1.5 | Per screen scrolled to reach the target |
Clicks, keystrokes and choices are counted separately. A choice is a decision with more than one plausible option: a radio group, a select, an optional field, or several buttons that look like the next step.
3. Comprehension: the second axis
Cognitive walkthrough. For the main action on each step, four fixed questions are answered with a probability: Will the user try to achieve the right effect? Notice the action? Associate it with the effect? See progress afterwards? In phase 0 the answers come from transparent heuristics (a stand-in for Jev). Set STUMBLEPOINT_LLM=claude to have Claude answer them from the screenshot instead.
Information scent. How well the words on the chosen control match the words in the goal, based on a small English and Norwegian concept lexicon.
Typical questions. Each flow type (checkout, sign-up, booking) has a bank of questions users tend to ask, such as "What does shipping cost?" or "Can I change this later?". A question is asked at the first step where it becomes relevant, and counts as answered if anything the user has seen so far answers it. This bank is a synthetic fallback: in phase 1 it is replaced by questions real testers ask while thinking aloud.
4. Ranking
Variants are ranked on effort including comprehension: KLM time plus one extra mental operator for each expected "no" in the walkthrough, and half a mental operator for each unanswered question. Variants that do not reach the goal rank last. Every report also shows which variant the simple rule "fewest clicks" would pick; the model has to beat that rule to be worth anything.
5. Stumble points
A stumble point is a place where the user is likely to slow down or get confused: an unanswered question, a popup interruption, an optional field, a validation error, an account wall, an action below the fold, weak label scent, missing progress feedback, or several equally plausible next steps.
Known limits
- KLM assumes practised users without errors, so estimates are low for first-time users. Corrected by agent tests and the calibration loop in later phases.
- Prototypes have no real traffic or backend. Everything here is clearly marked as estimated.
- Gaze prediction, learned effort models and real analytics paths are later phases.