Part ① · the promised fixed-date read
Round 3's retention, now that it has actually aged
The R3 report shipped on Jul 10 with a dated promise: “re-read R3's real D7 on/after Jul 16 — this is the single number that says whether the winners actually retain.” That date has passed. R3 landed players Jul 3–10, so every cohort has now had its full 7 days. Here is the answer.
The headline: better ads bought better first days, and zero extra second weeks.
Round 3's blended D1 is 3.90% vs the benchmark's 2.33% — a real, significant improvement (z=2.50, p=0.012). Round 3's blended D7 is 1.14% vs the benchmark's 0.99% — no difference whatsoever (z=0.44, p=0.663). The R3 report guessed that a 3× D1 lift might carry through to a D7 of 1.5–3%. It did not. Everything the hook bought was gone within a week.
Which ads gave the best 7-day retention?
Every R3 ad, ranked by matured exact-day D7. Fractions are the real numerator/denominator — the dashboard rounds these to whole percents, which is exactly what makes them look more meaningful than they are. D7 = active on exactly day 7. D7+ = active on any day from 7 onward (same cohort, looser rule). Stayed = seen on ≥2 distinct days.
| Ad | Players | Picker | D7 exact | 95% CI | D7+ alive | Stayed ≥2d | D1 |
|---|
| r3-daily |
408 |
52.0% |
2.0%8/406 |
1.0–3.8 |
3.0%12/406 |
12.5%51/408 |
6.4%26/408 |
| r3-combat |
343 |
25.4% |
1.8%6/342 |
0.8–3.8 |
3.5%12/342 |
8.7%30/343 |
3.5%12/343 |
| r3-doors |
402 |
31.3% |
1.2%5/402 |
0.5–2.9 |
1.7%7/402 |
10.0%40/402 |
4.2%17/402 |
| r3-instant |
598 |
61.4% |
1.2%7/598 |
0.6–2.4 |
3.3%20/598 |
12.2%73/598 |
4.5%27/598 |
| r3-boss |
385 |
16.6% |
1.0%4/385 |
0.4–2.6 |
2.3%9/385 |
9.6%37/385 |
2.3%9/385 |
| r3-dev |
485 |
46.6% |
1.0%5/485 |
0.4–2.4 |
2.5%12/485 |
10.5%51/485 |
3.1%15/485 |
| r3-honest |
392 |
41.6% |
1.0%4/392 |
0.4–2.6 |
1.0%4/392 |
9.7%38/392 |
5.6%22/392 |
| r3-cards |
377 |
16.7% |
0.8%3/377 |
0.3–2.3 |
1.6%6/377 |
5.0%19/377 |
1.6%6/377 |
| r3-time |
407 |
50.9% |
0.7%3/407 |
0.3–2.1 |
2.5%10/407 |
11.1%45/407 |
3.9%16/407 |
| r3-codex |
412 |
37.4% |
0.7%3/412 |
0.2–2.1 |
3.9%16/412 |
11.9%49/412 |
3.4%14/412 |
| climax_curiositymatured benchmark | 1,114 | 24.1% | 1.0%11/1114 | 0.6–1.8 | 4.6%51/1114 | 11.0%123/1114 | 2.3%26/1114 |
| R3 pooled | 4,209 | — | 1.14%48/4,206 | 0.9–1.5 | 2.57%108/4,206 | 10.29%433/4,209 | 3.90% |
Read the fractions, not the percents
The whole D7 column is built on 3 to 8 people per ad. One player either side moves an ad several rank positions. A χ² homogeneity test across all ten ads returns χ²=5.41, df=9, p=0.799 — that is as close to "these ten ads are the same ad" as data gets. Even best-vs-worst (daily 8/406 vs codex 3/412) is z=1.54, p=0.123 — not significant before you even account for the 45 pairwise comparisons on the table.
The proof that it's noise: the ranking won't hold still
Same players, same week, three reasonable definitions of “came back.” If the D7 column carried signal, an ad's rank would barely move between them. Instead:
| Ad | Rank · D7 exact | Rank · D7+ alive | Rank · stayed ≥2d | Swing |
|---|
| r3-daily | 1 | 4 | 1 | ±3 |
| r3-combat | 2 | 2 | 9 | ±7 |
| r3-doors | 3 | 8 | 6 | ±5 |
| r3-instant | 4 | 3 | 2 | ±2 |
| r3-boss | 5 | 7 | 8 | ±3 |
| r3-dev | 6 | 5 | 5 | ±1 |
| r3-honest | 7 | 10 | 7 | ±3 |
| r3-cards | 8 | 9 | 10 | ±2 |
| r3-time | 9 | 6 | 4 | ±5 |
| r3-codex | 10 | 1 | 3 | ±9 |
codex is dead last on exact-day D7 and first on D7+. combat is 2nd on D7 and 9th on stayed. time is 9th on D7 and 4th on stayed. A metric that reshuffles its own leaderboard when you nudge the definition by one day is not measuring the ad.
The one retention read this budget can actually support
“Stayed ≥2 days” uses every player as its denominator instead of the handful who returned on one specific calendar day — so it has ~10× the events behind it, and it separates: χ²=18.51, p=0.0296. Best vs worst is daily vs cards: z=3.66, p=0.0002 — real, and it survives the multiple-comparison threshold.
So: which R3 ads retained best?
daily and instant — and they are the only honest answer available. They are 1st and 2nd on the one powered metric (stayed ≥2 days: 12.5% / 12.2%), and top-4 on both D7 definitions, which no other ad manages. cards is the only clear loser (5.0%, less than half the leaders).
The caveat that matters more than the ranking: the gap between the best and worst ad's retention is worth ~7pp of a ≥2-day return, and nothing at all at day 7. Both R3 winners we carried forward (daily, instant) are confirmed as the right carry — but they were the right carry for CTR and completion reasons, and retention did not add information.
