I was treated to Dave McClure's "Startup Metrics" talk during Seedcamp in London last month. If you have not seen Dave do this talk, do yourself a favor and click on this link and spend a few minutes with the slides. Or even better, go see Dave give it live.
The ideas are simple, but so few actually apply them rigorously. In a nutshell, the methodology is: build, test, measure, iterate, test, measure, iterate, test, measure ……….
Which leads me to the point of this post. Measurement is not a simple thing. What do you measure and how do you measure it?
One of our firm's favorite measurements is the cohort analysis. From Wikipedia:
A cohort study or panel study is a form of longitudinal study used in medicine, social science and ecology. It is one type of study design and should be compared with a cross-sectional study.
A cohort
is a group of people who share a common characteristic or experience
within a defined period (e.g., are born, leave school, lose their job,
are exposed to a drug or a vaccine, etc.). Thus a group of people who
were born on a day or in a particular period, say 1948, form a birth
cohort. The comparison group may be the general population from which
the cohort is drawn, or it may be another cohort of persons thought to
have had little or no exposure to the substance under investigation,
but otherwise similar. Alternatively, subgroups within the cohort may
be compared with each other.
Like most things, it is easier to show one than explain one. And thanks to Robert J Moore, we have a few really interesting cohort analyses on Twitter to look at. He shared them in a guest post on Techcrunch yesterday.
This chart shows how new Twitter users behave over time.
And this chart shows how Twitter usage grows over time for new Twitter users who stick with the service.
I think both charts are interesting and you would not necessarily notice this behavior by looking at all users together because you need to isolate a certain group and observe them over time to see what is happening.
I'd encourage everyone doing a web startup to adopt the startup metrics methodology and within that methodology, make sure you are looking at cohorts of users, not just all of your users in the aggregate.
