As audience measurement systems grow to include vast amounts of data from Smart TVs, set-top boxes, and HbbTV devices, a key question emerges: Can we trust the data?

Traditional panel-based systems earned trust over the years by being transparent and built on solid statistical methods. But today, we’re dealing with data from tens of millions of devices, coming from diverse sources. To keep that same level of trust, we need new ways to make sure the data is accurate, reflects real audiences, and isn’t open to misuse.

Key approaches to ensuring trustworthiness

Here’s how the industry is addressing this important issue:

1. Calibration against panels

Even as big data takes a larger role, panels haven’t disappeared – they’re being used to calibrate and validate device-level data.

For example:

  • In the U.S., Nielsen still relies on its panel data to model demographics like age and gender, since this kind of information typically isn’t available from devices themselves.
  • This hybrid approach ensures that while millions of devices can tell you what was watched and when, panels still help answer the all-important question: who was watching.

2. Third-party validation & auditing

To maintain credibility, many measurement systems and data providers undergo independent audits or validation by third-party organizations. Some notable examples include:

  • The Media Rating Council (MRC) in the U.S., which audits and accredits measurement methodologies. Nielsen, for instance, lost its MRC accreditation for national TV ratings in 2021 but regained it after making improvements to transparency and accuracy.
  • AGF Videoforschung in Germany and AGTT in Austria have introduced HbbTV-based measurement alongside traditional panels and often rely on external technical audits to ensure data consistency.
  • BARB in the UK is actively exploring streaming measurement integration and works with independent partners to ensure its hybrid systems meet their rigorous standards.

These audits typically evaluate:

  • Sampling methods and representativeness
  • Data collection integrity
  • Device filtering and de-duplication
  • Privacy compliance
  • Transparency in modeling and reporting

3. Anomaly detection & filtering

Device data can be noisy or even misleading – for example, a Smart TV left on all day with no one watching, or a set-top box reporting phantom views due to firmware quirks. Traditional panels usually avoid these problems, as users are required to log in/out or interact during the viewing session.

To address these challenges, many systems now use:

  • Machine learning algorithms to detect and filter out anomalies
  • Cross-device matching to minimize duplication
  • Event validation, such as cross-referencing broadcast schedules or using Automatic Content Recognition (ACR) to fingerprint content, ensuring accuracy

4. Transparent methodology reporting

As measurement methods grow more complex, trust is built through transparency. Leading measurement providers now publish methodology papers, technical specifications, and change logs to keep broadcasters, agencies, and advertisers well-informed.

For example, Austria’s Teletest 2.0 project shared details about including HbbTV data, the scale of the data (1.1 million devices), and how it would be integrated alongside the existing panel.

This transparency helps demystify the process, boosting stakeholders’ confidence in using the data.

Why verification matters, maybe more than ever

With real financial decisions (ranging from ad spend to content commissioning) relying on viewership data, trust is non-negotiable. As data volumes increase and new collection methods emerge, so does the potential for error or manipulation. Independent audits, hybrid models, and transparent processes are crucial to ensure that the data behind billion-euro decisions remains reliable.

Relevant material

Downloads & further reading

Additional materials regarding this article

More information about the Icareus Audience Measurement

Icareus Audience Measurement

Read more about Icareus Audience Measurement

Related posts

Audience measurement has traditionally relied on small, representative panels such as those maintained by Nielsen, Kantar Media, AGF Videoforschung and BARB. While these have offered consistent benchmarks for decades, the explosion of viewing options across digital, time-shifted, and connected platforms has outgrown the limitations of small-sample data.

The industry is shifting toward a more comprehensive approach that blends traditional panels with actual data from millions of connected devices – like Smart TVs, set-top boxes, and HbbTV platforms. The result is a more detailed view of what people are watching, when, and on which device – but not without some challenges.

Why traditional panels are losing ground

Traditional panels have been useful, offering things like solid demographic insights (age, gender, household makeup) and a long history of trust. They’ve given broadcasters a standard ‘currency’ to sell ad space and provided advertisers with a verified audience.

Panel-based measurement systems are statistically solid, but their small scale can be a problem. Many shows – especially niche or regional ones – might not get recorded at all if none of the panel households were watching. Nielsen, for instance, found that in just one quarter, over 8,000 programs showed ‘zero ratings’. Not because no one watched, but because the panel missed them.

In a media landscape that values speed, panel systems fall short, taking days to report viewership data, especially for time-shifted content. To add to that, traditional people meters often miss second-by-second behavior and may only track when a person starts or stops watching, or give average-minute data.

Traditional panels have a difficult time accurately measuring mobile and OTT streaming, capturing out-of-home viewing (such as in public transit or coffee shops), or accounting for shared account usage (such as a whole family watching under one Netflix profile). These blind spots are reflected in the data.

While panels still play a core role – especially in modeling who’s watching – they’re now being supplemented (or even competed with) by device-level data and hybrid methods that introduce more speed, scale, and precision.

It’s not the end of panels: it’s the evolution into one piece of a much broader measurement ecosystem.

How big data enhances the picture

To close the gaps in traditional measurement, audience measurement providers have started integrating “big data” from connected devices. These sources include:

  • Return-path data (RPD) from set-top boxes
  • Automatic content recognition (ACR) from smart TVs
  • HbbTV platforms, which enable hybrid data collection from both broadcast and broadband environments

By tapping into these data streams, providers can build a more complete, granular, and real-time view of viewer behavior.

For example, Nielsen now uses data from over 30 million devices in the U.S. to supplement their panel, leading to more accurate readings—especially for smaller or time-shifted audiences.

The same trend is seen on a number of markets where the old TV audience measurement firms have started measuring both linear and FAST channel streaming consumption, as well as VOD viewing by broadcasters. This has prompted them to introduce total reporting on all viewed video on all platforms, or Total Video. It used to be that TV meant broadcast or cable. Now it’s video people view “all the time”: on TVs, phones, tablets, laptops, Live, on-demand, or time-shifted, On broadcast, YouTube, Netflix, TikTok, Disney+, etc., in-house, out-of-home, or on the move.

Traditional measurement systems couldn’t capture this full picture. Advertisers, content creators, and broadcasters need comparable, consolidated, and transparent data to understand audience behavior in this fragmented landscape.

Real life examples are Nielsen One, Kantar CrossMedia or Médiamétrie.

Real-world examples show that audience measurement is changing

In Turkey, the broadcaster ATV adopted an HbbTV-based audience measurement system. It allowed them to collect anonymized viewing data from over 2 million connected Smart TVs through interactive apps, enhanced guides, and catch-up services. This gave the channel deeper insight into how viewers consumed content, without relying solely on estimates.

Austria also offers a compelling case. In September 2024, AGTT (Austria’s TV measurement body) launched Teletest 2.0, which combines its traditional panel data with information from over 1.1 million HbbTV-connected devices. This hybrid model enhances reliability, reduces volatility in small target groups, and better represents niche channels. Time-shifted viewing data is now available as soon as the next day. It’s a strong step toward scalable and stable audience measurement that matches today’s complex viewing environment.

Persidera, Italy’s leading independent digital terrestrial network operator, has also expanded its service offerings to include advanced audience measurement solutions for broadcasters. The accurate and real-time audience measurement allows broadcasters to gain immediate insights into viewer behavior without the delays associated with traditional panel-based metrics. The data is gathered from over half a million daily active devices.

User consent on smart TVs: a critical gatekeeper

A significant element when collecting data from Smart TVs is user consent. Due to privacy concerns and regulations like GDPR and other similar frameworks, manufacturers and content providers must obtain explicit permission from users before collecting and sharing their viewing data. Consent is usually obtained through on-screen prompts during TV setup or app installations. For HbbTV platforms, a consent banner is displayed to the users as soon as they enter a channel.

Although individual opt-in rates differ based on a variety of factors, overall trends indicate that American consumers exhibit higher rates of consent for Smart TV data collection than their European peers. The distinction is primarily governed by local privacy legislation and data privacy attitudes in society. Additionally, the openness and character of the consent mechanism employed by different Smart TV makers significantly affect user opt-in rates.

Some brands have faced criticism for less transparent consent processes, burying ACR (Automatic Content Recognition) opt-ins deep in long terms and conditions so that it becomes hard for users to make an informed decision.

It’s important to mention that these rates have a direct impact on usable data volume for measurement systems. Even when millions of devices are deployed, only a subset of them actually report because of consent constraints.

Challenges in scaling real-time audience measurement

As connected device data becomes central to audience measurement, it brings significant challenges. Measuring millions of devices means collecting, processing, and storing vast volumes of second-by-second data. A single Smart TV might send tuning data every few seconds – multiply that by 10+ million devices, and you’re dealing with petabytes of data each month. This demands highly scalable cloud infrastructure, real-time data processing pipelines (such as Apache Kafka, Spark, etc.), and robust data governance.

Device and platform fragmentation makes it difficult to ensure consistent data collection across all these devices. APIs differ, firmware updates break things, and some platforms (like Apple TV or Netflix) may not allow third-party measurement at all.

For high quality data, identity resolution and deduplication is needed. One person might start watching on a Smart TV in the living room, then switch to their phone in bed. How do you know it’s the same person? And how do you avoid counting them twice?

In many cases device-level data doesn’t reveal who is watching. Panels are still worth it in such cases. On horizontal markets this may be more of an issue than on vertical platforms where you most likely have access to the subscriber data.

All technical challenges can surely be overcome, but building a robust, scalable, privacy-compliant measurement system is expensive. Smaller broadcasters or markets may not be able to afford these capabilities unless they partner with vendors or benefit from national-level solutions. In most cases it is still cheaper than building and running an audience panel with custom hardware.

The bottom line then becomes the quality and trustworthiness of audience data. How can we ensure that it’s valid, correct and reliable? This is where national panels may still hold strong value: to serve as a true currency for advertising, audience data needs to be validated.

The shift is happening, and it will benefit everyone

The shift to real-audience measurement fueled by connected devices is not just a technical change – it’s a cultural one. Advertisers and broadcasters can now gain insight into actual viewership at a level of granularity that is beyond estimates based on small panels. But to realize the full potential of this change, they must balance scale of data with ethical issues around consent and privacy.

As Smart TV and HbbTV adoption continues to grow, the marriage of panel-based and device-based measurement will likely be the industry standard, providing the best of both accuracy and reach.

Helsinki, 15.08,2024:  Icareus is happy to announce its new version of Audience Measurement solution at IBC2024. It will be showcased at the IBC2024 exhibition held in September in Amsterdam and is immediately  available as cloud service to our customers.

Connectivity is transforming audience measurement by enabling real-time data collection and more accurate insights into viewer behaviour. Icareus Audience Measurement solution offers TV operators, broadcasters, and other service providers tools to gather detailed information on how audiences are engaging with their TV channels. This data is collected in real-time directly from Smart TVs and set-top boxes. The solution supports multiple standards, including interactive TV environments like MHP and HbbTV, and is compatible with all major TV brands such as Samsung, LG, and Sony.

Traditional audience panels should be complemented, or even replaced, with direct, real-time data collection systems like Icareus Audience measurement, because they often rely on a limited number of devices, and thus can miss key insights. Icareus Audience Measurement collects data from all connected devices in the TV network, offering real-time insights that are far more comprehensive and reliable, even for niche channels on e.g. satellite platforms.

This real-time access to viewing data allows operators and broadcasters to make informed decisions without the delay of waiting several days for panel-based metrics. Additionally, Icareus can fine-tune panel sizes as needed to meet different cost and operational requirements, providing flexibility and efficiency.

All the data is presented in flexible, customizable Icareus Audience Dashboards that allows data analysists to customize their experience by modifying and building their own KPIs and dashboards. For full transparency and data management, all collected data can be exported to third-party data management platforms or enriched with additional third-party data, enhancing the usability, depth and value of the system. Our Analytics APIs help 3rd parties to exploit the platform to its full extend with their existing and in-house solutions.

Icareus Audience Measurement can be used in multiple use-cases to improve broadcasters and operators’ operations, viewer engagement and revenues:

  • Improving Viewer Experience by understanding viewer preferences
  • Advertising Revenue Optimization via improved viewer data for more precise ads targeting
  • Competitive Benchmarking to understand market share and channel performance
  • Content Development, Programming and Scheduling decisions by providing understanding which shows, genres and time slots attract the most viewers
  • Audience Engagement and Retention by analyzing the demographics, geographic and behaviours of audiences

Read more about the solution: https://icareus.com/products/audience-measurement/

Or if you are coming to IBC2024, please book a meeting with us: https://icareus.com/events-ibc-2024/

Icareus Ltd is established 2001 and has a headquarters in Helsinki, Finland. Combining online and broadcast technologies have been the core of our activities over two decades, true pioneers some say. We’ve been making the change from linear TV to today’s multi-screen experience. Icareus solutions are enjoyed by millions of consumers via our broadcaster, operator, OTT and OVP customers in over 60 countries.

Icareus TV & Video Cloud and Icareus Playout, a trusted broadcast platform for EPG, Data carousels and OTA support solutions for HbbTV, OTT, TV Everywhere, Addressable TV advertising and Audience Measurement.

For more Information:                                                                                                  

Ms. Jessica Glad
Marketing
Icareus Ltd

Email: [email protected]
Telephone: +358 (0)9 2289 0801

Follow Icareus on:

LinkedIn: https://www.linkedin.com/company/icareus-ltd-

TV Pixel, loved by advertisers, needed by broadcasters, crucial for content owners, controlled by regulators (via e.g. GDPR). Read how the TV Pixel can be used to identify the TV, or viewer in the very best case.

Continue reading
Audience Measurement from 2 million+ Turkish TVs and HbbTV services for four channels with the effort of one

Helsinki, 23.5.2018, Icareus has delivered a turnkey HbbTV solution to ATV, the leading commercial broadcaster in Turkey. ATV wanted to offer more to their viewers and to understand better how their channels are watched, so they turned to Icareus to design and deliver a complete SmartTV service based on HbbTV (www.hbbtv.org). Icareus build a turnkey HbbTV solution based on its existing products that enabled ATV (www.atv.com.tr) and its sister channels A Haber, A Spor and children’s’ channel Minika to offer valuable services to their viewers via SmartTVs.

Continue reading