Measuring an app without tracking people

You can learn almost everything you need to improve an app without knowing a single thing about who is using it. That surprises people who have spent time around growth dashboards, where the default is to follow each user through every screen. But the questions that actually change what gets built are simpler than that, and they can be answered with counts.

The three questions worth asking

For a small, focused app, three questions cover most decisions.

Activation: do people get to the point? Of the people who open the app for the first time, how many complete the one thing it exists for? A first scan, a first logged visit, a first finished set. If that number is low, the problem is almost always in the first two minutes, and you can see where the drop happens without knowing who dropped.

Retention: do they come back? How many installs are still being opened a week later, and a month later? This can be measured as an aggregate curve rather than a list of individual people.

Feature use: what is actually used? Which screens and features get opened, and which never do? A feature that nobody touches is a candidate for removal, and a feature that everyone uses deserves more care.

None of these need a name, an email address, a location, or an advertising identifier.

How to answer them anonymously

The technique is dull, which is the point.

• Count events, not people. Record that a "first scan completed" happened, not who completed it. Events carry the app version and maybe the language, and nothing that identifies a person.
• Look only at aggregates. Read totals and rates across everyone. If a number is only meaningful when you can see one person's path, it is the wrong number.
• No cross-app identifiers. Never attach an advertising ID or link events to activity in other apps. That is the line between measurement and tracking.
• Keep the content out. A fitness app can count finished workouts without ever sending the video. A notes app can count saves without sending a word of the note.
• Offer a switch. Put an analytics opt-out in the app's settings, where anyone can turn it off, and respect it immediately.

Measurement asks how the app is doing; tracking asks what a person is doing, and only the first one is our business.

What we deliberately don't learn

Measuring this way means giving things up, and it is worth being honest about them.

We don't know who our most active users are. We can't email someone who stopped opening the app to ask why. We can't build a profile that says a certain kind of person tends to churn in week three. We can't tell whether the same person uses the app on two devices. And some of our apps, like Stampr, ship with no analytics SDK at all, so for those we learn only from reviews and messages people choose to send.

All of that is fine. The questions those answers would serve are mostly about retention for its own sake or about advertising, and neither is how we decide what to build. We wrote about why a single-purpose tool should not be optimized for time spent in the case against subscriptions for small apps.

What good measurement looks like in practice

It looks like a handful of numbers, reviewed every few weeks. Did activation go up after the new first-run tour? Did anyone use the export button this month? Did a crash spike after the last release? When a number moves, you change something, ship it, and look again.

It also looks like humility. Anonymous counts tell you what happened, not why. For the why, the best tool is still reading reviews and support email, and occasionally asking a real person to try the app while you watch.

The more we build this way, the more convinced we are that a good app can be measured carefully without ever watching anyone.