User behavior in App with Flurry Analytics

This blog post is a short summary of my experience with the Flurry analytics SDK. Adding the SDK to our application has been a good experience (so far) and has drastically improved our ability to make data driven decisions.

Why ?

We have a connected app and are able to gauge the performance of the app (we thought) based on the server based transactions but we found that there were certain questions that we just could not answer with our logs on the server.

• Why isn't there a direct correlation between the number of downloads and the number of transactions ? Do our users even launch the app after they download it ?
• We spent a lot of time adding a feature, do the users understand it ? do they even navigate to the screen ?
• many more ...

What ?

We started looking for ways we could answer these and other questions. The stress was on figuring our how users interacted with our application:

• Do they use the features we put into the application?
• How often do they use them ?
• What screens do they visit in the app and how often ? (should we really be spending the time and effort in updating help section inside our app?)
• Once they visit the screen do they perform the action we want them to perform or get side tracked by something else ?
• ...

We already had transaction based data on the server already about our users but we were looking to finding out what percent of our users actually reached the point where they could perform a transaction.

How?

The engineers in us already knew the answer : we had to put in some sort of logging mechanism in our application that logged the user activity. Nothing ground breaking, we have been doing that in web applications already. Things we considered:

• Should we build our own ? This was interesting from engineering point of view and we could just extend what we were already doing in our web app. We decided to look for third party libraries as we just did not have the bandwidth and time to invest in building and testing something for all the platforms. Reinventing the wheel is a sin that modern dev teams should stay from.
• Impact on app performance. We decided very early on that what ever library we choose should have minimal impact on the app performance. User activity was important but not at the cost of our apps speed.
• Dashboard: we did not want to invest time in building a dashboard that displayed the metrics for the same reason we started looking for a third party library.
• Cost: It almost always comes down to cost. All the stakeholders loved the idea of knowing the user behavior but no one wanted to put down a number on the spend. We started looking for the cheapest option we could find.

Flurry

We looked at a few packages (not a lot) and quickly decided on using Flurry analytics SDK. Here is a breakdown of the reasons:

• Free : Talk about in-expensive.
• Reported activity offline and did not impact app performance.
• Required very little development effort, the documentation was simple and to the point and we just added the library as a black box into our app.
• Self service system, we did not have to speak to account managers or do a webex to understand how the system works. (I hate these calls)
• Widely used in the industry. That took care of the end of life questions.
• Support for the mobile platforms we cared about.
• The analytics dashboard was just awesome.

Notes:

• Adding the SDK impacted our test cycle more than development. We had to add a relatively large number of test cases in our functional and performance tests.
• We decided against using the pageviews in the SDK. We created our own custom events for page views.
• Sessions on Android were handled differently from the default behavior. We made the implementation decision based on our application needs.
• As with any good SaaS platform, the SDK website allows us to download all our data through APIS and we plan to do this periodically. We found that the data for the past few days often changes due to the nature of the SDK so we plan to download data before the last 7 days.

All in all, it was pretty easy getting up and running and we were able to add the SDK and track some of the core events within one iteration along with solving other worldly problems.

Privacy
Tracking anything about the user has taken a beating in the recent months especially when mobile applications are involved. Though I agree with most of the arguments made for privacy, some of the concerns that have surfaced are way over the top. I will leave this topic for another blog post.

Privacy was one of the first hurdles we had to cross internally before implementation could begin. We set our goal: We wanted to know what "users" did when they opened our application, not a particular "user". The privacy concerns played a important role in our implementation decisions (rightly so) and we made a conscious effort not to include any PII in the data we logged. We are also not using the data we collect in making any real time decisions on what to display to a particular user (we cannot, it wasn't built for this.).

Summary

Adding the analytics sdk in our application(s) did not solve all our issues but it definitely provided us more data to base our product decisions on.

Moving forward:

It is awesome that Flurry has recently added HTML5 to its list of supported platforms. It makes my life easier if i can use the same engine for mobile web and apps.

I also love the user segmentation and funnel features they have recently added to their dashboard.