iSpot is the respected leader in the industry because of our measurement grade data. It starts with our Editorial Team, who updates and maintains our TV Ad Catalog, but that is only the beginning. You know that part of the story, but the best dataset for TV ads needs the best data science to generate reliable and repeatable measurement. With over 80 years of combined experience in data science and analytics, we asked why iSpot data science is more than raw ACR data and applied algorithms. Say, “hello,” to the iSpot data science team.
Starting with raw ACR (Automated Content Recognition) data, Leah Fredman, Ph.D., Data Scientist with iSpot began, “just because there’s the word ‘automated’ in the title, which makes [ACR] seem like it’s a simple thing, it’s pretty much the opposite. You have to get the data to a point where it’s going to be automated, and still make sure what you’re getting is what you expect.”
“It goes even deeper,” added Michael Bardaro, Ph.D., VP of Data Science. “You have several challenges. You must identify the [TV] program and if we are dealing with any time shift problems. Broadcast networks are a collection of affiliates in all different time zones. They can play different media or content depending on the time of day. These issues are just the tip of the iceberg. Take the Big Bang Theory, for example. The theme song is always the same. The media that is on during the theme song is always the same. So which episode was it? There’s an unbelievable amount of work that goes into taking raw ACR data and turning that into something useful that we can share with our clients.”
Andy Berner, Ph.D., Senior Data Scientist with iSpot, joined in. “Part of the value we add is the ad catalog that we can use as a source of truth. We can look at the ACR data and say, ’this TV appeared to be tuned-in to this network.’ We can use that context to make informed decisions. On the flip side, you have some cases where you have no ad detections with specific creatives. We can still make an informed decision. Here is an ad pod. All the ads preceding and all the ads after this ad were there, so I can infer that an ad was aired on-screen because there’s a 30-second gap in the sequence of where I’d expect to see it.”
Berner continued, “not all ads detect the same, not all networks detect the same, and there is a lot of variants that ideally you’d like to find a way to pull out of the system.”
“We, as a data science team, can provide results that are explainable, precise, and accurate. You can repeat what we did.”
Sophia Carayannopoulos, Associate Data Science Engineer
For iSpot, the detection of ads isn’t enough. iSpot must consider the location of a TV and who is watching. “You have a collection of households, but you also have TVs in hotel and business lobbies, sports bars, dorms, waiting rooms, even prisons,” Bardaro stated. “Do you react to an ad the same way when you’re sitting in an airport, and CNN is on every TV around you?”
“You don’t know anything about the exposed population in a more public setting as opposed to a household,” added Berner.
Nicole Lawless-DesJardins, Ph.D., Senior Data Scientist, goes deeper. “We’re not just interested in knowing what played, but how people react to it. We tie that back to business outcomes, either online or offline, such as in a store. That work of linking our data to our other data assets is where a lot of the cleanup comes in. We’re filtering down to, ‘this is a household that’s watching TV in a normative way.’ Then we can determine how they reacted to those ads.”
“That is a point that should not be taken lightly,” said Darby Greenwell, Ph.D., VP of Analytics and Data Science. “A lot of other data science teams that I’ve seen skip these things and focus on an algorithm. They think to themselves, ‘I know this machine learning thing that will do this for me.’ Any good data scientist knows that is not the right approach. If you put garbage into a very smart algorithm, you end up with very smart garbage.”
Fredman added, “It’s not like we clean the data one time and go, ‘now we’ll just let this model roll.’ iSpot data science makes sure every time we improve a model, we consider the best method to produce the most useful and trustful results.”
Sophia Carayannopoulos, Associate Data Science Engineer and the newest person on the data science team, provided this insight. “We, as a data science team, can provide results that are explainable, precise, and accurate. You can repeat what we did. The customer who is looking at these results can then understand what to do with the information.”
Bardaro adds, “That’s true, anybody can grab data and put something together that indicates you should buy this or that. This is an industry where 20-percent, 30-percent errors are common. If you’re spending $600 million a year on TV ads, and you’re talking about a 30% error rate, that’s a significant amount of money. These errors are difficult to find because they’re systematic and inherent to the data. iSpot data science is a team of experts who hunt these errors and biases down. Customer doesn’t have a 30%, ‘surprise,’ in terms of what they are buying, or what they are measuring and trying to understand.”
“There are technology changes all the time. ACR 15 years ago is probably unrecognizable to ACR today…”
Nicole Lawless-DesJardins, Senior Data Scientist
“Other providers have data science, but some things make iSpot unique,” Berner said. “I think one thing that is cool about our team is rather than having people from a relatively narrow set of fields, we’ve got a great deal of intellectual diversity, in terms of backgrounds and skillsets.”
Andy Ewing, Ph.D., Senior Data Scientist, added on the diversity of the team, “I, as the economist, would say it’s our comparative advantage. The social sciences that we have represented, the physical sciences, the non-sciences, the computer science. We can always count on each other to try and puzzle these things out in the most meaningful way.”
Is “mission accomplished” every achieved? On that question, Lawless-DesJardins started, “There are technology changes all the time. ACR 15 years ago is probably unrecognizable to ACR today – or recognizable, but very different.”
Carayannopoulos had this to say in closing. “I don’t think anyone is okay with complacency or mediocrity. If we were to get to a 99%, everything’s perfect state, we’re still going to ask ‘well, how did we miss that 1%.’ We will never be done.”
If your head is swimming from all the different issues that must be considered to squeeze trustworthy results out of measurement grade data, we certainly understand. iSpot has provided you just a peek into the complexities of data science that go far beyond neural nets, formulas, and algorithms. It is why we believe iSpot has assembled the best TV ad data science team on the planet.
If you’d like to learn even more about iSpot’s data science team and processes, please email marketing@ispot.tv.