Product Mgmt For NewBies #6 – Think Metrics

Have you ever measured how much time do you spend having coffee / tea breaks or having random water cooler discussions / sutta breaks with colleagues ?  Have you measured how much do you spend checking / responding to email ?

I had taken a stab at measuring these back in 2008 and I had realized I was spending just way too much on them. A few days of measurement helped me improve my productivity and time spent in office tremendously. I was trying to measure and improve my productivity as a software developer, that was also when I took a series of steps to improve my productivity using little tips and tricks (published quite a few blogs posts too on the topic).

There is a famous saying, “you cant improve what you don’t measure”.

When building a product/ feature, the idea is to move some metric. What is that metric ?

Your users ability to accomplish a task ?

Your ability to make money from them ?

All objectives can be defined in the form of a metric and can be measured.

And yes, keep in mind there are LOTs of metrics you CAN measure. Does not mean you need to measure/ focus on them all.

Define the objectives & focus on the key metrics.(Top 1 or 2 metrics only)

Its extremely easy and usual to get lost in a barrage of metrics. Page views, bounce rate, exit rate, conversion rate, click through rate and a thousand other metrics. But you need to very clearly define the objectives of a product and just focus on the top 1 or top 2 metrics at any given stage of the product / company.

  • For example, at product launch, the key metric to measure might be the # of users doing the first step or the first few steps after installing the app  – essentially answering the question whether the users are able to see value in the app to invest the time for those first few steps ? Is your app’s on-boarding experience appropriate ?
  • Next phase could be to measure how many users are taking the key step of accomplishing the task that the product was designed to do. For example, for Practo, it would be whether users are calling the doctor / scheduling an appointment with the doctor they were looking for ?
  • The following phase might be whether users are coming back after the first use ?
  • And so forth.

After a point you might be measure a combination of the a few metrics ( the top 2 metrics I referred to.). But restrict yourself to a max of 2. Do not go beyond that. They are a distraction. ( There are enough distractions in a PM’s life anyway)

Feel free to measure and go through a 100 metrics for sanity purposes. To ensure that nothing is completely broken / has a bug. The idea is to focus your time on moving only the top 1 or 2 metrics.  

Feel free to setup alerts for several sanity metrics.

Just so that you don’t end up wasting time going over all the sanity metrics you can setup alerts so that you get an email when there is something abnormal. ( like # of users in a particular segment of users, or the usage rate of a particular important feature etc.)

Here is how you can do this on Google Analytics. Having these alerts in place will give you a bit of peace of mind that someone is checking them every day / week so you can focus on the other stuff.

The top 1 or 2 metrics will also ensure alignment and help you take trade offs.

Your boss / peers / engineers / designers  and all other stakeholders. You might be chasing a different metric and your boss might be looking at a completely different one. Align yourselves on metrics and not features.

When the discussion is all about that top metric, its much easier to take those trade offs while designing  / architecting a feature / system.

When planning/ prioritizing a feature – try to predict how much change do you expect to see in the top metric.

This will help you prioritize. A feature that’s cool and the latest in-thing but will not move the key metric does not deserve attention at that given stage. This will also help you improve your product judgement about what product change brings about how much change in the key metric. Key to being a successful PM. Its calculated bets you take after all.

Master Product Analytics tools 

You wont believe how effective you can be in your decision making if you knew how to make use of Google Analytics / Localytics / Mix Panel or whichever Analytics tools you use.

There are tons of video resources available on the internet to help you learn. Spend your weekends on them. These tools are going to be your bread and butter. You HAVE to be master. You have NO choice.

Caution: Don’t be too obsessed by Quantitative Analysis, Qualitative study is equally important.

Many folks believe that quantitative data and Analytics tools like Google Anlaytics etc have all the ‘insights’ they need. I strongly disagree. There is no better alternative to actually seeing your products being used by real users. (Whether you see them face to face, or you see recordings is your choice).  I have said this many times, metrics/ analytics are symptoms of a problem / behavior, the root cause is usually found when observing / talking to your users. So please do talk to your users as often as you can ( 1-2 times a week is a good number).

Useful Follow up Resources:

Would love to hear your thoughts  / feedback on the topic.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s