So much for the methodology of the book; it is more rewarding to discuss some of the observations of the field work. First of all, it is noticeable how the current use of analytics has matured since 2013. Petre almost exclusively talks about tactics involving Chartbeat’s reach metrics - pageviews, visitors - while 2022 shows lots of workflows based on reading time, conversions, engagement or value.
What is really absent from the book is the fact that the relationship between content strategy and KPIs is more layered and certainly goes way beyond measuring just the reach of news articles.
I think it’s here that Petre does the greatest disservice to journalism.
In any good newsroom there will be articles that are meant to generate loyalty, conversions, engagement or brand awareness. In any of these cases reach is important, but for news brands for whom subscribers are an important part of their business model (everywhere outside the US), the analysis of data isn't one-dimensional. Sports articles behave differently than breaking news, long reads are a special breed, and various formats address different user needs. There is no one size fits all metric or tactic. While Petre writes that editors seem to identify editorial analytics with click bait strategies, she spends little time correcting that feeling.
Granted, data strategies have become much more sophisticated in the last few years. The ‘data destroys quality journalism’ narrative was probably a more widespread feeling in 2013 than today. But even then the researcher should have realised that you were looking at a skill set that was only 5 years old. Chartbeat didn't freeze or monopolise the analytics world, it was just a portion of phase one. At the end of the book Petre acknowledged that Chartbeat threw out a couple of these assumptions years later.
A point she does make well is how Chartbeat started to make metrics acceptable in the newsroom setting. Petre observes that Chartbeat has gone to great lengths to show editors that the numbers actually often reinforce the gut feeling that they already had. Or, if it is a tough truth that the tool is selling, they have been able to make the numbers look more magical and positive, maybe even a bit undefined ("meaning uncertainty" as Petre puts it) so everybody still has some wriggle room to create a narrative about success. A great example is the fact that if stories outperform even the top threshold on a given day, the visual representation of that success (here, a dial) breaks and won’t display the real number. While you could argue this is a real ‘off the charts’ moment and it is in some ways brilliant, reading this in 2022 from the smartocto HQ, it’s hard not to see the limitations and flaws.
We know that the user interface plays a huge role in people accepting a more data inspired culture, and in 2022 we’re beyond extremes: newsrooms need properly visualised, honest and transparent data because editors are more ready to accept that they can learn from data. Honesty and trust are two sides of the same coin. That’s what we do at smartocto - and we know that’s what works. We try to be straight shooters, but that is probably our Dutch-Serbian nature.