By Louise Lief
They are extremely addictive. Repeat users describe an adrenaline rush, followed by rapid mood swings that fluctuate between exhilaration, and anxiety, self-doubt and despair. They also report high stress levels.
In our brave new big data world, web metrics, the statistics that measure page views, unique visitors, bounce rates, engagement time, tweets, Facebook “Likes,” and a host of other things, have become proxies for an organization’s effectiveness.
But after reading several critiques of current analytics tools, including reports by the Investigative Reporting Workshop at American University where I am based, the Reuters Institute for the Study of Journalism, and a fascinating study by Columbia University’s Tow Center on Digital Journalism on the emotional and behavioral impact metrics can have on employees and organizations, I wanted to take a deeper dive.
So I organized a workshop on the topic for the D.C. Science Writers’ Association, where I served on the board. I asked Brian Boyer, National Public Radio’s Senior Supervising Visuals Editor, to be our guide.
A word about Brian (that’s him in the photo, cavorting with a journalism award.) He’s an intense guy. An award-winning designer and computer scientist, he has thought a great deal about the strengths and weaknesses of mass market analytics tools and their applications to organizations like NPR, which aspires to be a national model for high quality journalism “strengthening the cultural, civic, and social fabric of our democracy.” He heads a combined news applications and multimedia team of photographers, coders, and designers, and favors t-shirts (more on that later.)
Brian began our session not with an overview of the pros and cons of various analytics tools, but with the question, “Why?” He asked us to introduce ourselves by saying who we were, what we did, and why we did it (that last one can get a bit complicated.) Our only tools were post-its and sharpies.
Brian’s focus on “why” stems from author Simon Sinek’s 2009 book Start with Why. Sinek’s main thesis is that if people don’t identify in some deep and fundamental way with why your organization exists, and why you do what you do, they won’t gravitate to you or stick around. The way you communicate to the outside world, says Sinek, should personify your why. You should seek to inspire.
The problem with the mass-market metrics, says Brian, is “a lot of tools we have to measure impact have nothing to do with why we do it. We end up being blinded by the tools we have.”
Google Analytics and Facebook Insights, the free analytics tools many use, give lots of information but are designed to maximize the impact of online ads. That’s their why. Is it yours?
Their measures do not necessarily tell us what people think about our mission, or how we might meaningfully engage with people who care or might potentially become concerned or involved in issues we think are important. Most tools now tell us about “reach” or audience intent – which could be the function of sensational headlines that do not necessarily inform or engage. Comments tend to skew negative. One analytics “blind spot,” writes Paul Bradshaw, a media professor at Birmingham City University, is that people may read a story even if they never visit the page where it is published.
But, Brian says, “if we ask, ‘what is our mission and how will we know if we have accomplished it?’” then we can start building and using more appropriate tools to measure it.
Our group was an eclectic bunch. Some were journalists, others were communicators from government agencies, scientific associations and universities. Their interests ranged from agriculture, to chemistry, medical research, wildlife conservation, and space exploration.
Our collective list of hoped for achievements for our various organizations revolved around four general goals:
-- Get increased support for funding.
-- Get basic knowledge to the public.
-- Empower people to act.
-- Speak to people’s values.
Then we divided into affinity groups. Each group was asked to list specific outcomes they would like to see. Here is a brief list of what we came up with:
--Connecting people and creating robust networks.
--Equipping people with knowledge to create greater goods.
--Inspiring people and explaining complexity
--Make science a cultural movement like music and art.
Next, Brian asked us to list concrete actions to measure those desired outcomes. He asked us to think – happy coincidence! -- like scientists, to form hypotheses we could test.
The list we made here included asking readers a question before they read a story and then again afterwards to see if opinions changed, or measuring how many people signed up for a newsletter or used an online tool after reading a story. If people were given an opportunity to take a next step, such as signing a petition, how many would do so? And so on.
Brian’s why at NPR is to make people care. He and his team continuously experiment with new ways to do that.
The thing about experiments is that the results often surprise us. That happened with NPR’s “Planet Money Makes a T-Shirt”, a global supply chain story that tracked the making of a t-shirt from cotton seed to finished product. NPR launched a kickstarter campaign to fund the project. If you donated you got one of the t-shirts, which sports a martini-drinking squirrel. They hoped for $50,000. They raised $590,000. Hundreds of thousands of people visited the site on its first day and lingered, and donors shared pictures of themselves on social media wearing the t-shirt.
The gripping podcast series Serial is another example. The first of its genre to ever to win a Peabody Award, it has been called an “audio game changer” and has been downloaded more than 80 million times. The producers were as surprised as everyone else. They just wanted to tell what they thought was a compelling story about flaws in the criminal justice system in a new way. But Serial’s great storytelling happened to coincide with technological advances in smartphones and cars that made its astonishingly rapid dissemination possible.
Can these successes be replicated? Was it the format, the subject matter, all of the above, or luck? What will happen as technologies continue to evolve, and to what extent will current metrics help answer these questions?
The key takeaway for me from Brian’s workshop was that when facing these uncertainties, your why will help you navigate. Metrics aren’t going away, but asking why helps put them -- imperfect measures for understanding the messy, contradictory creatures we are -- in perspective.
It gives you a measure of freedom. Otherwise, you may feel as Gawker Media founder and CEO Nick Denton did, when he told the Tow Center’s Caitlin Petre, “We – the freest journalists on the planet – were slaves to the Facebook algorithm.”