Monday April 27, 2009

Effects of Moving to a First Party Cookie Picking up where I left off yesterday (okay, three months ago, but who's counting)...

We took the plunge and made the switch from the 2o7.net cookie to a first party sun.com measurement cookie.  Other than the self-serving benefits of improved data, it seemed like an improvement to have a browser image request go to metrics.sun.com rather than the mysterious 2o7.net.

The outcome was an improvement in several data quality metrics.

The correction factor for monthly unique users improved from 78% to 83% (see last entry for what this means). 
Percent of users who delete their measurement cookie at least once per month decreased from 20% to 14%.
Percent of users who block the measurement cookie dropped from about 5% to less than 1%.

As part of this analysis we also backed out what percent of our users use more than one computer in a month to visit our site: 30%. Of course, this was unaffected by the cookie change. (It would have been remarkable if it had been. Its consistency was a bit of a control in our calculations.)

Accurately determining the cookie deletion rate and multiple computer use is an inexact science based on comparison of IP addresses, user O/S and browser version profiles for a given visitor ID.  So, I don't bet my life on the accuracy of these breakdowns.  Only the correction factor can be accurately measured; the constituent mechanisms behind it are slipperier to divine.

We've watched these numbers over nine months now since we made the change, and although they do vary a little from month to month, we have not seen a definite trend either up or down in any of them.

My assessment is that it was worthwhile to make the switch from third to first party cookies, although I would characterize the improvement in quality of data as modest. The observed difference between deletion rates for first and third party cookies is not dramatic. Add to that the fact that much of the error is driven by multiple computer use, and the bottom line ends up being that the change is favorable, but not eye popping.

Nevertheless, I'm much more confident now having evaluated these parameters for myself on our own site.  I no longer have to rely on published data from other sources which don't transfer to our situation, and I am in a better position to defend the level of accuracy in our data.

( Apr 27 2009, 08:15:54 PM MDT ) Permalink Comments [1]