Sometime over the summer, there was a discussion of analytics in the AVC comments and Shana said something like "I would love to do a serious data analysis on AVC's analytics." So I reached out to Shana and told her that I would give her access to my Google Analytics and Disqus Analytics accounts and she could go crazy on the numbers. But I told her that she had to produce a post out of all of that work. She agreed and this guest post is the result of those efforts.
——————————————————————————————-
One of the things I find really hard to wrap my head around about AVC.com is that for all intents and purposes it runs like any other media site. To me, this is my bar where I hang out, but in reality, this site functions much like many other media sites such as the Atlantic, or Refinery29. There is content, there are analytics, and there are ways of pushing out content, there are some ads and tools to push them out, there are some tools to make the community more social, but not much else. If AVC.com had a business model (which it doesn’t, the advertising money goes to charity), it would be one similar to many content sites out there: Increase Users; Increase pageviews; Sell ads. What makes this site unusual is that there is a large community of users, primarily driven by technology built by the team at Disqus.
It also leads to some interesting questions about this site in comparison to other media sites. Most content sites are still trying to figure out the role of comments. Do they ignore them? Do they not have them? Do they feature some content? Do they write about the comments? Do they reward commenting behavior? Does having a community make a difference to the business model of content sites?
On this site, it does. Not only does it make a difference, comments here are highly correlated with unique pageviews of repeat users, uniques in general (not just for repeat users), time on site by repeat users, and time on site by everyone.
One half of unique pageviews over the past 9 months have been generated by repeat users.
I wanted to see if unique pageviews of returning users was correlated with the number of comments. I used a correlation coefficient (Spearman’s Rank Correlation Coefficient) which is a measure of correlation variables that behave monotonically, or in other words, the variables move up and down together. The correlation coefficient for unique pageviews generated by repeat visitors is 0.7973 to comments. This is a high correlation coefficient and suggests the two are linked.
To give a comparison point to explain this correlation coefficient, SEOMOZ reports for Good SEO Experiments a correlation coefficient(a linear measure of correlation) of 0.3 is considered quite good, even though 0.3 usually implies fairly low correlation. Having a community on your site is therefore way more likely to be a factor that would generate significant traffic than SEO efforts, if we compare statistical significance.
I also looked at the rate of change for the percent of returning users versus the percent of new users. They line up quite nicely. They have a correlation coefficient of .9387. However, the rate of change for new users as well as repeat users is quite small. Granted, this is a niche audience, so I’m not totally surprised. Still, it is nice to know that total user activity is very much driven by regular user activity.
However, average time on site for all users to comments is less correlative (though still significantly so), with a correlation coefficient of 0.6733. Similarly, there is correlation coefficient of 0.6848 for average time on site for returning users versus comments. I suspect the reason is that some people like emailing back replies, some people like to go the site to write replies, and some people like using Engagio to write replies. Unfortunately there is no way to directly measure which people on this site are using email, Engagio, or the site itself to reply to comments.
The correlation of all unique visitors is also highly correlative to comments. (correlation coefficient =0.8413).
This data leads me to believe that people are in fact coming to the site not just for the posts, but for the community surrounding the posts. People are more curious about the chatter and the interactions that come out of the posts than the post itself. Building out community means over time you will build out a growing site.
If you are a web publisher/media company and you are looking at this post, having a strong commenting platform (like Disqus) is going to be essential to your long-term success as a media outlet. Communities can be bigger drivers of traffic than Search Engine Optimization. Having a strong moderation/community management team in place is more essential than having SEO staff in the long term, since there is a higher correlation to factors that matter to growth and ad sales (pageviews, uniques, time one site) to having community. The reason is that people are not just on your media sites to read: They are there to interact with other readers about what they have read. Teaching your writers and your community to stick to your site to discuss articles in depth ends up causing long-term growth.
(some notes:)
1)My friend Daniel Choi, a PHD Candidate in Molecular Biology/Computational Biology at Princeton, helped me understand rho based correlations. Thank you Daniel.
2)For the sake of discussion, Disqus and Google Analytics are two different reporting tools. GA also samples when you are looking at daily data for 9 months for a site of this size. Please therefore take this post with a grain of statistical salt.
3) William Mougayar was kind of enough to give me some data about Fred to see if Fred’s presence in the comments matters. It didn’t make it into the post for a variety of reasons. Thank you William, anyway.
4) Thank you Fred for cleaning up some of the language about correlations during the editing process
5)IRL I’m a web analyst who is job hunting for my next gig while handling some side projects. If you like this post, feel free to get in touch)
