Kochava: Measuring lift with the marketing research panel of the future

Kochava How do you measure ad campaigns? Real-time attribution has become the watchword. But linking outcomes of campaigns as evidence of a job well done may not be providing an accurate picture. For savvy advertisers there’s a lot more that can be done with the available data. Grant Simmons, VP of Client Analytics, Kochava, and VP of Kochava Foundry proposes a shift in thinking.

Effective marketing is challenging to determine, and answering the question: “Was my marketing truly impactful?” may be like asking a rhetorical one. In measuring lift, one of the many pitfalls that nab marketers is equating real-time attribution with campaign outcomes as evidence of impact. But in doing so, they are sacrificing accuracy for the immediacy of having an answer.

To truly measure lift requires copious data, resources, time, and money; however, there are other more traditional, accurate, and efficient ways to measure the impact of ad campaigns. Although not measured in real time, the testing is much more sound.

Recently we shifted some thinking from row-level install analysis to measuring campaigns through modelling methods we believe are statistically defensible. We don’t pretend that we’ve built absolutely perfect models because “all models are wrong, some models are useful”, but I believe that the framework is sound and provides remarkable insight.

Ultimately, the measurement helps answer the questions that matter—”What was the value of running this campaign? Of serving this ad? Of touching the same prospect multiple times? Was serving ads to prior installers worth it?”—along with other slices observed through an incremental lens. The results are then anonymized due to the modeling and statistical approach, keeping with where the privacy winds are blowing, yet still providing meaningful measurement in an increasingly non-deterministic world.

And to that end, how do you figure out a moment using row-level data? Regardless, today’s marketer will soon be met with a smaller dataset to derive measurement value from – and how Kochava intends to make that pivot will prove critical.

The challenges marketers face

One thing we can all agree on—2020 has been a year of unparalleled living. Within the digital advertising ecosystem alone marketers are being challenged with respect to how they target and measure their campaigns. Measuring true incrementality is already challenging by nature. Historically, marketers measure an exposed and holdout (control) group; however, ensuring there is no bias is one hurdle. Then, there is the loss of revenue by not advertising to the holdout group.

More recently, changes with how device data is collected on iOS 14 have added another significant hurdle to measuring lift. Row-level measurement will not be not possible with Apple’s SKAdNetwork in 2021 when Apple begins enforcing its AppTrackingTransparency (ATT) framework. At that time, marketers will lose real-time postbacks for iOS users because the IDFA will no longer be available unless a user consents. Add to that the impending deprecation of the third-party cookie, and marketers have two more hurdles to overcome.

A new way of measuring lift with traditional methods

Traditionally, a market research panel (a.k.a. marketing panel) is a group of individuals who had shared much information about themselves with a brand’s marketing team. Kochava has reinvigorated the marketing panel by eliminating the time and cost it takes to assemble one with the Kochava Collective, our privacy-first, data marketplace. It’s an effective way to create an unbiased, comparable holdout group.

The Collective holds more than 8B unique devices globally. These devices include device IDs (that have been consented to), geo, points of interest, interest and behaviors, and a number of other attributes gleaned from consented devices.

To ensure there is no bias in the holdout groups created, devices are scored based on several characteristics, such as the length of time it has been in the data set, recency of app usage, geo, and transactional app engagement, among other attributes. By creating a holdout group with the Collective, marketers don’t have to sacrifice segmenting a portion of their audience and can still maximize their reach.

>>Use Case: QSR chain proves incremental lift from connected TV campaign

A video on demand (VOD) platform recently accessed the Collective to prove the efficacy of a QSR chain’s ads displayed on their platform.

In this case, the service provider had already created an exposed and holdout group. Their primary source for targeting were IP addresses and had 300K segmented in the holdout group. They needed a way to verify there was no bias against the exposed group.

The Kochava data science team mapped the holdout group to the Collective’s app graph to create a household identity graph and used it to map the available devices to the household IP addresses the client provided. With the device scores available, 66.1% of the devices from the exposed and holdout groups were mapped.

The scores showed that the two were nearly identical, confirming that the group receiving ads mirrored those in the control group. The team then used the evidence from the incrementality exercise to show that the campaigns displayed on their platform had resulted in statistically significant lift.

The Future

Measure lift and maximize your reach by using data from the Collective to create a new panel or validate an existing panel. Once we have the panel in hand, we onboard your audience data and create an app graph to ensure the groups are comparable (ie, limit bias).

Measuring lift is typically an expensive endeavor in terms of time and money, but we’re discovering that a data set like the Collective provides a means to measure the only question that matters: “How much do I want to spend on an ad?”

KochavaGrant Simmons, VP of Client Analytics, Kochava, and VP of Kochava Foundry