Cohort Metrics For Startups Revealed – Part I: Plain English

Meet Silver Surfer aka 500 SuperMentor Dan Martell (@danmartell).  Dan is the Co-Founder of Flowtown, a social marketing application for businesses.  Self diagnosed as having intense ADHD, you can either find him hosting dinners, speaking about marketing or jumping off cliffs around the world.

Authors note: This post was written in collaboration with Assaf Arkin, Flowtown’s lead engineer and the creator of Vanity – a ruby framework for experiment driven development (EDD).

Every time I speak at a conference and present how we do weekly cohorts for our product metrics, everyone wants an explanation. They kind of get it, but don’t fully understand it. The following 2 part series is going to try and define what a cohort is and then how to understand aged groups. If you have any subsequent questions – please ask in the comments.

Cohorts in plain English

Our Direction

A cohort is a group of people who share a common characteristic or experience within a defined period. For example, all the people who signed up for our service on a given week, or all the people who signed up from Facebook, Twitter or other channels, or all the people who signed up for our basic, premium or supreme plans.

I’m going to start with a simpler examples, and then work to describe age cohorts, which seems to confuse more people. Let’s start by dividing our customers based on the channel from which they came through.

Grouping activity into cohorts

Salutation Nation - 137

Say we run ads through several channels. A conversion analysis may tell us something like this:
[table id=1 /]

We don’t care how many impressions we got, as you can seen they’re not correlated with number of signups. It’s a vanity metric. We don’t care how many signups we got either, we can always get more by spending more on a given campaign. What’s more interesting is, which channel should we double down on to get the best customers for the least amount of money.

And you would think the answer is A. At $1.11/signup, channel A is %13 cheaper than channel B and 20% cheaper than channel C. Clearly we should be doubling down on channel A and forgetting about the under performing channels B and C. But is that the truth?

Aligning the channel

Rush

Our business is based on subscriptions, so let’s have a look at the life time value of each channel:
[table id=2 /]

As you can see, customers that came through channel C remained customers for an average of 8 months, 45% longer than customers who came through channel A. More customers that came through channel C choose the $10 and $20/month subscription plan, whereas customers coming through channel A opted for the $5 and $10/month plans.

As a result the customer life-time value of channel C is more than double that of channel A. It may be 20% more expensive to acquire customers through channel C, but that $4 cost translates into $51 more revenue.

What we’ve done here is segment out customer base into cohorts, each cohort representing a group of customers we acquired through a given channel, and then looked at their behavior over time. That’s a typical cohort analysis.

If you have any questions, please don’t hesitate to leave a question in the comments.

UPDATE: Part II continues here.

Never miss a beat

  • http://twitter.com/KieranO Kieran O’Neill

    Enjoyed this basic introduction, but would like it to carry on to more advanced topics on cohort analysis. Perhaps in the next installment? :)

  • http://twitter.com/esturistico esturisti.co

    We love this topic as it has been difficult for us to track over time how our cohorts behaves. Keep them coming! 

  • http://about.me/bradleyjoyce bradleyjoyce

    Thanks Dan, appreciate the post! Not a lot of people out there talking about the nitty gritty details of these types of metrics.

    • http://twitter.com/danmartell Dan Martell

      Exactly.  It’s why I felt motivated to discuss how their done and use them as reference when I’m talking to entrepreneurs.

      Thanks for the comment.

    • http://twitter.com/danmartell Dan Martell

      Exactly.  It’s why I felt motivated to discuss how their done and use them as reference when I’m talking to entrepreneurs.

      Thanks for the comment.

  • http://www.GetDateIdeas.com Will

    Bad-ass post! Thanks for writing this, Dan! Still so much to learn…looking forward to the next few posts in this series!

    • http://twitter.com/danmartell Dan Martell

      Thx Will! ;)

    • http://twitter.com/danmartell Dan Martell

      Thx Will! ;)

  • Darragh

    Excellent. I would love to hear of simple techniques for analysing cohorts and advisable time-frames etc through which they should be examined. 

    • http://twitter.com/danmartell Dan Martell

      Great question.  I actually cover that a bit more in Part II re: Aged Groups.

      I’ll be in the comments answering questions there.

      Let me know.

    • http://twitter.com/danmartell Dan Martell

      Great question.  I actually cover that a bit more in Part II re: Aged Groups.

      I’ll be in the comments answering questions there.

      Let me know.

  • http://changespeakingout.blogspot.com/ Carl J. Mistlebauer

    Obviously, then another analysis could be attempted to find out why customers from a particular channel paid for a higher plan vs another, and why a particular channel netted long term customers.  By looking at retention on a weekly basis you isolate the factor of improvements increasing usefulness but I am not sure, without knowing what the different pricing per plan exacting if cohorts by channel is ideal.  Maybe running the same analysis by plan price might give one more insight…

  • http://twitter.com/alexkehayias Alex Kehayias

    The life time value table isn’t showing up [table id=2 /]

  • http://twitter.com/geoffw8 Geoff Wright

    Arghghhghggh! Really wish I could see those tables :/

  • http://twitter.com/jayamster Jason Amster

    Yeah, the tables aren’t’ showing up.

  • Scott de Rozic

    Double down with alex – can you re-add the tables back in?

    [table id=1 /]

  • Andrew

    Where are the tables?