The activation answer that came out of the #202 dive, once the id-churn measurement fault (#473) was fixed and the numbers became readable.
The finding
Genuine installs, latest ping each, last 30 days, n=214:
|
installs |
with a real library (100+ files) |
share |
| finished onboarding |
114 (53.3%) |
83 |
73% |
| did not finish |
95 (44.4%) |
13 |
14% |
| unknown |
5 (2.3%) |
|
|
Finishing the wizard takes an install from 14% activated to 73% activated. Nothing else in the dataset separates activated from not-activated anywhere near as sharply. Library size, which is what #202 was originally opened about, turned out to be downstream of this.
44% never finish. That is the loss, and unlike the 99.5% figure that started #202, this one is real rather than an artefact.
Two distinct cohorts, do not merge them
The 95 who never finish are the bulk of the loss. 66% of them also report their subgen as unreachable (#479), which is consistent with stopping partway through rather than being a separate failure. For this group the question is where in the wizard they stop.
The 31 who finished and still show <100 files are 27% of finishers and a harder problem. The product reached them, they completed setup, and they still did not activate. A fix aimed at the first group will not touch these.
What is missing before designing anything
onboarding_step is recorded (0-11) and is the obvious next cut: it says where they stop, which is the difference between a wizard that asks too much too early and one that fails at a specific integration. I deliberately have not run it yet, because the step numbering needs checking against the current wizard before the distribution means anything. A step index that shifted when the wizard changed would produce a confident, wrong answer about which screen loses people.
Worth remembering from #202: every earlier conclusion drawn from this table was wrong because the denominator was wrong. The same caution applies to reading step numbers whose meaning may have drifted.
Caveat
Genuine requires positive evidence of a real install, so an install that arrived today on an older client is excluded until its second day. These figures undercount by construction. That was the deliberate trade against a number that was 80x too high.
Related: #202 (the dive), #473 (the measurement fault), #479 (subgen unreachable, overlapping population).
The activation answer that came out of the #202 dive, once the id-churn measurement fault (#473) was fixed and the numbers became readable.
The finding
Genuine installs, latest ping each, last 30 days, n=214:
Finishing the wizard takes an install from 14% activated to 73% activated. Nothing else in the dataset separates activated from not-activated anywhere near as sharply. Library size, which is what #202 was originally opened about, turned out to be downstream of this.
44% never finish. That is the loss, and unlike the 99.5% figure that started #202, this one is real rather than an artefact.
Two distinct cohorts, do not merge them
The 95 who never finish are the bulk of the loss. 66% of them also report their subgen as unreachable (#479), which is consistent with stopping partway through rather than being a separate failure. For this group the question is where in the wizard they stop.
The 31 who finished and still show <100 files are 27% of finishers and a harder problem. The product reached them, they completed setup, and they still did not activate. A fix aimed at the first group will not touch these.
What is missing before designing anything
onboarding_stepis recorded (0-11) and is the obvious next cut: it says where they stop, which is the difference between a wizard that asks too much too early and one that fails at a specific integration. I deliberately have not run it yet, because the step numbering needs checking against the current wizard before the distribution means anything. A step index that shifted when the wizard changed would produce a confident, wrong answer about which screen loses people.Worth remembering from #202: every earlier conclusion drawn from this table was wrong because the denominator was wrong. The same caution applies to reading step numbers whose meaning may have drifted.
Caveat
Genuine requires positive evidence of a real install, so an install that arrived today on an older client is excluded until its second day. These figures undercount by construction. That was the deliberate trade against a number that was 80x too high.
Related: #202 (the dive), #473 (the measurement fault), #479 (subgen unreachable, overlapping population).