What Metrics Matter Most? TV Industry Grapples With Ratings Shortcomings in Fast-Growing Content Landscape

This is the final installment in a five-part series examining the transformation of television as the industry prepares to celebrate the Primetime Emmy Awards on Sept. 19. 

The rapid migration of TV viewers to streaming services and on-demand platforms has put added pressure on the need for transparent measurement standards at a time when Nielsen, the standard-bearer for TV ratings for decades, is under fire from the industry watchdog group Media Ratings Council.

Earlier this month, the MRC suspended its backing of Nielsen’s national ratings service, amid charges the service has not been counting TV viewers correctly during the coronavirus pandemic. No rival has emerged to take Nielsen’s place, though NBCUniversal is working on its own measurement system. NBCU and others are motivated to take extraordinary steps to address the measurement shortcomings in the expanding TV universe because of the fear that ad-supported networks are leaving money on the table with marketers.

In the meantime, the industry is searching for a better yardstick for the SVOD landscape, which has become amplified during the pandemic with four new prominent new streaming platforms — HBO Max, Peacock, Discovery Plus and Paramount Plus — joining Netflix, Hulu, Amazon, Disney Plus and Apple TV Plus as major contenders for audience share. And all of these services have different metrics for their success, adding to the confusion across the industry about the performance benchmarks that matter most.

“As the streaming space gets more crowded and major players are seeking to carve out their own success, the most important metrics to consider are audience engagement and retention,” said Dietrich von Behren, chief business officer at Reelgood, a streaming navigation service that allows users to easily browse through streaming options from a single hub.

Nielsen earlier this year launched the Gauge, a service that measures the share of total audience viewing and how it breaks down among broadcast, cable and streaming, as a big-picture view of the larger TV ecosystem.

“Are people watching? How much are they churning? And what shall define how these questions are answered is the popularity of the original content each streaming service can deliver to their subscribers,” von Behren says.

---

Television in Transition

• • • Monday: Season — Why the construct of the fall season endures despite TV’s vast expansion.
    • Tuesday: Development — How sweeping changes in production protocols are affecting the creative process.
    • Wednesday: Casting  — What the shift to video auditions means for actors and talent discovery.
    • Thursday: Marketing — Why the business of ballyhoo has moved from attention-getting stunts to algorithms.
    • Friday: Metrics — What matters most these days as the on-demand world challenges audience measurement norms.

---

Netflix places its emphasis on revenue and market share — both total revenue and revenue per subscriber, an individual with knowledge of the heavyweight streamer’s thinking told Variety. As such, subscriber numbers are important to the company, both in absolutes and in terms of reach. Netflix is known for carefully watching how many users start a new series, how many users watch episodes in their entirety and how many users dive in to binge-watch more than one episode in a sitting.

Other platforms look at everything from how quickly users come to the show to the amount of social-media chatter generated by a given title. To analysts, in the SVOD arena, however, the rubber meets the road in quarterly earnings with subscribers.

“Subscriber acquisition and retention are ultimately the only metrics that should matter to streamers,” says Alejandro Rojas, director of applied analytics at Parrot Analytics, “but streamers need to know how to get these numbers, and global streaming-video companies need to understand which types of stories are most appealing to their audiences.”

According to Rojas, the clearest leading indicator of subscriber growth is demand. “Demand for licensed content is the key to retaining these subscribers — streamers have no idea how content on other platforms are performing, so demand data democratizes access to this information and allows streamers to make smarter decisions on licensing new content to their platforms,” he says.

That’s where Parrot’s “Demand Measurement” service comes in. It tracks the way modern digital consumers interact with content — from researching shows on IMDB or Wikipedia, watching a trailer on YouTube, reacting to the show on social media and watching episodes of the actual series. “We are providing an objective view of what audiences want, and how much they want it,” Rojas says.

Rojas points to the massive global demand for Disney Plus’ Star Wars- and Marvel-branded content as one of the biggest success stories of the youthful streaming era. “‘The Mandalorian,’ ‘WandaVision,’ ‘The Falcon and the Winter Soldier’ and ‘Loki’ have all become the most in-demand TV series in the world across all platforms with weeks, and sometimes days, of debuting,” he says. “It was easy to expect these shows would do well, but the extent of their rapid demand growth and overnight cultural dominance was still surprising.”

Yan Liu, CEO of TVision Insights, cites co-viewing, attention, as well as the viewer’s share of time spent on each application and the percentage of households with the application installed as another important metric to watch. “Those numbers indicate the popularity of a specific application, and can indicate both opportunity for growth, and the likelihood that subscribers will continue to pay for the service,” Liu explains.

“Disney Plus was able to attract new subscribers, who were older, with the launch of ‘Mulan’ (in 2020),” Liu notes. “The Disney Plus subscriber base is twice as likely to include homes with children as the typical viewing household, but the new subscribers from the ‘Mulan’ launch looked nothing like the Disney Plus core audience. ‘Mulan’ subscribers were on average older (median age of 47 vs. 36) and were far less likely to have children in the home (31% vs. 47%) than the typical Disney Plus subscriber,” Liu states.

Brian Fuhrer, senior VP of product strategy at Nielsen Global Media, attests that the old pay-TV adage that “usage equals retention” is still critical and will be of increasing value as the streaming landscape continues to fragment.

“Obviously, the streamers themselves know a great deal about how many times shows are requested, but understanding the characteristics of the people that are viewing the content is going to be important for content development and valuation,” he says.

Knowing what core users look like from an age, gender, race or ethnicity perspective can identify key opportunities and competitive weaknesses to guide program creation and acquisition from other sources.

“From a sheer metrics perspective, the basic questions ‘how many, how often, and how long’ still apply to audience analysis,” he continues. “Again, while platforms have great insight into their own users, understanding their competitors and what non-users of their services look like is another key component to informing strategy.”

Ratings concerns are of major importance to creative talent and talent representatives, who have traditionally secured bigger paydays and bonuses based on TV shows hitting certain performance benchmarks. In the streaming arena, it’s hard if not impossible for talent to be armed with the information to negotiate a better deal in success.

This is why, Fuhrer explains, Nielsen publicly shares the minutes viewed metric for top SVOD programs. “It allows you to put the content of very dissimilar durations (e.g. ‘Grey’s Anatomy’ with 376 episodes vs. ‘Outer Banks’ with 20) on a level and comparable playing field.”

Nielsen’s battle with the Media Ratings Council — the New York-based organization comprised of leaders of media companies, marketers and advertising agencies — has only made the landscape of audience measurement more complicated. A Nielsen spokesperson said that, while the MRC’s move was disappointing, the suspension “will not impact the usability of our data.”

The Nielsen representative emphasized that as far as TV networks and other clients are concerned, it’s business as usual in terms of setting advertising transactions based on Nielsen data.

“Nielsen remains the currency of choice for media companies, advertisers and agencies,” the company said in a statement to Variety, calling itself committed to the audit process and working with the MRC on resolving this suspension. “We will also take the opportunity to focus on innovating our core products and continue to deliver data that clients can rely on, ultimately creating a better media future for the entire industry.”