Why testers drop out, and what to do about it

Twelve testers for fourteen days sounds easy until day nine, when two of them have disappeared. Knowing why helps you design against it.

Why they leave

• Nothing is at stake. A favour to a stranger has no cost to stop. Free swaps mostly fail here.
• Forgetting. The app sits unopened; the tester loses track of the commitment.
• Phones change. A reset, a new device or a cleaned-up account loses the opt-in.
• Storage. The app is uninstalled to make room. Uninstalling does not always mean opting out, but people often do both.
• Accidental opt-out. A tester taps “leave the test” while exploring the page, then opts in again. Google restarts their 14 days.
• The app cannot be found. The track is misconfigured, so the tester gives up. See the Google Groups guide.
• Fifteen apps is a lot. In an exchange, testing many apps every day gets tiring.

What to do about it

1. Recruit above twelve. A margin of three or more turns one departure into a non-event.
2. Check your setup before you invite anyone. Group on the track, release out of draft, every country covered.
3. Count opted-in testers, not invitations. Look at the track’s tester numbers on days 3, 7 and 12.
4. Remind people, with a reason. One short message that says what to test is better than a generic nudge.
5. Make leaving cost something. Credit, a deposit or a reputation. Anything that is easy to re-earn will not hold.
6. Have a replacement ready. A waiting list means a vacancy lasts hours, not days.

What TestersGuild does about it

It is the stake, in a form we can enforce: $15.00 of credit locked on joining. Leaving before the start is free for up to 3 times in a row; after the start leaving costs everything that is left. Each missed day of the daily check-in costs $1.00, and 3 misses remove the tester. Every app has 3 spare testers, and a reserve list takes over assignments when someone goes. Forfeited credit stays with us and never goes to other users.

None of this is magic. It is a cost to leaving, a margin and a queue. And it has not yet run with real strangers at scale, so we cannot tell you our dropout rate.