INCREMENTALITY PLATFORMS REVIEWS
An independent and unbiased review of all incrementality platforms
Incrementality platforms help marketers measure return over investment. There are a few methods to do incrementality tests. Every marketer experienced the horror realizing that their own advertising activities is cannibalizing organic results. Incrementality platforms help marketers turn waste to profit
Why Incrementality ?
Everyone dreams that their product will get seen by millions of people. Everyone wants to be Flappy Bird and Among Us, but those are one in a million. Literally. There are more than a million apps out there and app developers end up spending millions of dollars on advertising to get people to download their games.
Most advertising budget is wasted.
In June 2019, Uber Technologies filed a lawsuit against Bidmotion, Taptica, YouAppi, AdAction, and Phunware. The claim was that these ad networks (and another 100 “unnamed” ad networks) were conducting fraud, showed negligence, and acted in unfair competition against them.
Uber won this case, but the details were sealed.
The lawsuit showed that Uber found out about these fraudulent networks by accident – turning off 10% of their budget, the analytics and marketing teams noticed that new installs and new registration numbers were not going down in correlation with the budget cuts.
Uber started digging around and ended up cutting away 80% (!!!!!!) of their budget – realizing it had no impact over their new installs and new registration numbers!
Just like most developers/advertisers – Uber was equipped with state of the art attribution technology, analytics tools, and was using the ad networks ranked “top” by most of the attribution companies.
Part of the problem was in the incentives!
It used to be common to incentivize user acquisition managers by setting goals for volumes of paid installs, or increasing return over advertising spend (ROAS)
But return over advertising spend may come unjust, if the advertising activities are cannibalizing organic installs.
It could be very confusing – but a higher ROAS % might lead to a lower ROI!!!
“I used to spend over $100,000 per month on Google until I had to cut my budget for a week, waiting for a new app version to be released. I wanted to die when I saw that the number of daily installs didn’t flinch during that week” (Joel M, Growth Marketer)
What is Incrementality ?
Incrementality measures the true effectiveness of Advertising, regardless of tracking.
The goal of paid advertising is to create incremental revenues. Whether if it’s to establish a stronger brand equity, or to push people to download an app and use it.
Incrementality testing requires Advertising to create various scenarios to isolate conversions data. Tests use changes in the marketing activities to compare how a change in activity influenced campaigns performance over time.
The goal for marketing in any organization is to drive growth by driving customers and prospects through the marketing funnel. Awareness > Interest > Desire > Action
Efficiency in Advertising is reached when budgets produces results that would not have happened if it was not for the Advertising activities. (from INCRMNTAL.com)
Incrementality testing became more important recently, as mobile operating systems and browsers started restricting access to user identifiers like Cookies and IDFA.
Google restricted cookies from the chrome browser, and Apple killed the IDFA making attribution and tracking almost impossible.
Incrementality doesn’t need device identifiers – as incrementality testing is a statistical approach using machine learning and data science to understand causality behind marketing performance results.
There are several ways to do incrementality checks: •Testing by switching off advertising all together and restarting it (“Blackout”) •Using machine learning and data science (“statistical”) •Conducting user surveys / asking the users what made them convert (“surveys”) •Showing some users random ads and comparing the performance to users that saw the campaign ads (“control group”) •
Incrementality Platforms Review – Overview
| Core Feature | Who does it best ? | Runner up |
|---|---|---|
| On-going incrementality testing capabilities | INCRMNTAL – was the only platform that offers on-going testing. All other platforms rely on the advertiser spending money on ghost ads or stopping campaigns in a country to figure out the incremental value | Facebook Lift – facebook can offer a lift test as part of the standard campaign platform, providing ongoing tests. Only downside is the media cost over random ads served. |
| Ease of integration / onboarding | Appsflyer – offer a very simple SDK for developers to integrate. Once the SDK is integrated, it has access to almost all the data they need for incrementality | INCRMNTAL – while there is no SDK – if a marketer uses attribution solutions or can provide data access through BigQuery or Amazon S3 – incrmntal built plug-ins to do the integration through their platform |
| Pricing / pricing model | INCRMNTAL – had a completely free package, and also offered 30 days free for all packages. The pricing was flat and not based on how much traffic or how much spend was tracked. | Facebook Lift – charge advertisers for the clicks or impressions served. Even if the ads were random to the control group. |
| Compliant with privacy regulations and restrictions (post IDFA, cookies) | INCRMNTAL – didn’t even want the ids / cookies. I could onboard aggregated data. I saw that they had an option of a DPA only in cases where developers can’t remove the ids!!! | Measured.com – has a lot of material about not needing cookies. Note measured doesn’t work well for mobile – only for PC advertising. |
Choosing the Right Incrementality Platforms
might have different needs. It’s important to select the right platform, as choosing the wrong. Incrementality platform could lead to a lot of frustration and waste. My recommendation is to select an incrementality platform that offers
Easy integration, testing for the incrementality of campaigns, Incrementality of different channels. A good incrementality platform will allow you to test campaigns without needing to stop them.
With the changes in privacy restrictions – it’s really important to choose a Platform that is compliant with the latest privacy restrictions. Pricing is also an important factor to selecting an incrementality testing Platform. Flat pricing is my preference. Most platforms focus on showing the advertiser what is incremental But it’s really important to select a platform that also shows you how to eliminate cannibalization and waste.
Detailed incrementality platform reviews
Check out the snapshot rating for each incrementality platform I reviewed. Each review has a link to a more in-depth analysis. I’m a big incrementality geek. Each reviews dives deep into the nuts and bolts of every aspect of the platforms.
INCRMNTAL (www.incrmntal.com)
Summary:
INCRMNTAL’s incrementality measurement platform offers continuous measurement in a privacy first world. The platform offers advertisers with the ability to test campaigns, channels and activities in an on-going way, without forcing the marketer to stop any campaigns or activities.
I’m a big fan of their clean interface and “tickets” approach, allowing me to understand all of the marketing activities I did and how they influence my overall results. Testing incrementality just requires me to drag a ticket to the “measurement” column and within minutes, I can see exactly what is the incremental value of anything I want.
Pros:
- Integration took me 2 minutes. I use Appsflyer for attribution, and incrmntal already had a partner module on Appsflyer. I just needed to give incrmntal access to the cohort api and boom – everything worked seamlessly.
- On-going testing – there was no need for me to stop any campaign to get the incremental value for it.
- Accurate results – I tried incrmntal for over 3 months and managed to increase my ROI by 2x!
- Flat pricing – my personal favorite. I took the business package that gave me unlimited testing (I tested over 50 activities for 3 of my apps).
- Extra: I really like the way the results are explained. It was really clear for me, but also to my engineering team who wanted to understand the data science incrmntal was applying.
Incrementality Platform Reviews rating for INCRMNTAL:
- Ease of Integration: ★★★★☆
- Pricing: ★★★★★
- Useful insights: ★★★★★
- Unbiased results: ★★★★★
- Compliancy with privacy requirements: ★★★★★
- Overall Score: ★★★★★
Measured (www.measured.com)
Summary:
Measured.com mostly focuses on desktop advertising analysis offering advertisers with an attribution suite and incrementality experiments. The platform ingests, unifies and stores data for attribution, offering customization, monitoring and data management.
The incrementality testing component offers advertisers with a systematic set of experiments where advertisers will need to stop certain campaigns or channels one after another in order to map results and understanding the incremental contribution of the campaigns to the marketing mix.
Pros:
- Strong for affiliate marketing – Integrated with most affiliate platforms
- Offers multi touch attribution (web advertising only!)
Cons:
- Forces the marketer to stop advertising to test incrementality
- Pricing is a % of media spend + software license fees
- Audience testing approach is not compliant with privacy requirements (2021)
- Very difficult to understand the results in the reporting dashboard
Incrementality Platform Reviews rating for Measured:
- Ease of Integration: ★☆☆☆☆
- Pricing: ★★☆☆☆
- Useful insights: ★☆☆☆☆
- Unbiased results: ★★★★☆
- Compliancy with privacy requirements: ☆☆☆☆☆
- Overall Score: ★★☆☆☆
MetricWorks (www.metric.works)
Summary:
metric.works offers top down incrementality measurement, LTV prediction and marketing automation. That’s a very big promise.
Requiring three to twelve months of data, metricworks takes three weeks to provide insights as a report to marketers.
The approach forces marketers to stop campaigns in a country or region, in order to evaluate the incremental impact of advertising and come up with a recommendation for a new media mix in a different country.
Note: while the website talks about technology – I did not get to see any platform or dashboard while testing metric.works, so I’m not clear if they have any technology or if it’s just a service.
Pros & Cons:
- Forces the marketer to stop advertising to test incrementality
- Pricing is based on a % of media spend (about 1-2% of media spend!!!!)
- Integration takes 2-3 weeks, and needed me to pass data from the past 12 months!
- Since the pricing is based on media spend, I couldn’t trust the suggestions
- I never saw a dashboard
- They wanted upfront pricing….based on media spend….to measure incrementality….?!?!?!?!?!
Incrementality Platform Reviews rating for MetricWorks:
- Ease of Integration: ★☆☆☆☆
- Pricing: ★☆☆☆☆
- Useful insights: ★☆☆☆☆
- Unbiased results: ★☆☆☆☆
- Compliancy with privacy requirements: ★★★☆☆
- Overall Score: ★★☆☆☆
remerge (www.remerge.io)
Summary:
remerge is a little different than the other platforms I reviewed, because remerge is a demand side platform (DSP) that sells media to advertisers, but also offers a unique pricing model – cost per incremental conversion.
The incrementality measurement method used by remerge requires the ability to identify users on exchanges and ssps and as of April 2021, this option only works on android phones, because the android OS still allows access to a device identifier (GAID).
Remerge only offers cost per incrementality on media purchased through their DSP.
Pros & Cons:
- Cost per incremental conversion is a very lucrative offer
- Remerge offers a very scientific method to pricing incremental conversions
- The incremental campaigns are only retargeting campaigns
- The platform only works for android apps
- There is no way to audit the results and the media costs
- Integration is simple if you are already using an attribution technology
Incrementality Platform Reviews rating for Remerge:
- Ease of Integration: ★★★★☆
- Pricing: ★★★☆☆
- Useful insights: ★☆☆☆☆
- Unbiased results: ★☆☆☆☆
- Compliancy with privacy requirements: ☆☆☆☆☆
- Overall Score: ★★☆☆☆
Facebook Conversion Lift (www.facebook.com)
Summary:
Facebook conversion lift offers an incremental analysis over the campaign activities on facebook. The product is similar to that of remerge.io (link to review) but works using an audience split.
There is no way to audit the test or validate the results.
With the current privacy restrictions facebook has yet to make a clear statement if the product will be eliminated or if it will continue offering conversion lift tests only for android devices.
The lift tests are only showing results for the campaign activities on facebook.
Unfortunately the results from the facebook conversion test are not actionable. The test can only show from facebook’s point of view if the results of the campaigns on facebook are incremental or not, but there’s no underlying actionable insight.
Pros & Cons:
- There is no way to audit the results and the media costs
- If you already use facebook for advertising, you can use the feature
- There is a minimum spend of $5,000 for each test
- The results are not actionable
- There is a clear bias in the results, as facebook’s interest is for you as an advertiser to spend more
Incrementality Platform Reviews rating for Facebook Conversion Lift:
- Ease of Integration: ★★★★★
- Pricing: ★★☆☆☆
- Useful insights: ★☆☆☆☆
- Unbiased results: ★☆☆☆☆
- Compliancy with privacy requirements: ☆☆☆☆☆
- Overall Score: ★★★☆☆
Appsflyer (www.appsflyer.com)
Summary:
Appsflyer is known as a mobile attribution solution offering attribution by ids (IDFA, GAID) and fingerprinting. Appsflyer introduced an incrementality feature in October 2021, strangely enough, the product offers incrementality measurement only for retargeting.
The product offers marketers the creation of a split audience group using device ids and synching the audience group to various partner DSPs to create a test where one group sees ads and the other group sees no ads to compare the impact of ads over performance.
Since the product requires device identifiers to create an audience split and use those for precise targeting – the product is relatively redundant now that the operating systems created privacy restrictions giving no access to the device identifiers.
Pros & Cons:
- If you are already an appsflyer customer, there is no additional integration required
- The approach uses device identifiers , a method that became deprecated due to privacy restrictions
- Advertisers need to pay media owners over the media inventories even if serving no ads to the control group
- Appsflyer pricing model is based on the volume measured
- An integration of the appsflyer sdk is required
Incrementality Platform Reviews rating for Appsflyer Incrementality:
- Ease of Integration: ★★★★★
- Pricing: ★★★☆☆
- Useful insights: ★☆☆☆☆
- Unbiased results: ★☆☆☆☆
- Compliancy with privacy requirements: ☆☆☆☆☆
- Overall Score: ★★☆☆☆
Kochava Foundry (www.kochava.com)
Summary:
Kochava Foundry is a premium service offered by Kochava, lead by Grant Simmons, Kochava’s VP of Kochava Foundry.
The foundry team offers expert consultation services for advanced measurement, marketing strategy, anti-fraud, setting the right app subscription model, and conducting brand sentiment surveys.
While the kochava foundty is not an incrementality platform, I wanted to include this in our review, as grant simmons is one of the top experts in our industry and his knowledge and understanding in measurement is not to be ignored.
Pros & Cons:
- While not a platform per se – consultation lead by Grant Simmons who is regarded as one of the top experts in measurement
- Available as a premium service to kochava’s customers
- Requires advertiser to run a series of blackout tests in specific times per day and days per week
- Since this is a service – results are curated and explained to the advertiser alongside a suggested plan to execute both the tests and make strategic changes to the media mix
- As a service – the tests are ad hoc and spot tests. There is no on-going testing of incrementality
- Pricing is based on a service charge
Incrementality Platform Reviews rating for Kochava Foundry:
- Ease of Integration: ★☆☆☆☆
- Pricing: ★★☆☆☆
- Useful insights: ★★★★★
- Unbiased results: ★★★★☆
- Compliancy with privacy requirements: ★★★★☆
- Overall Score: ★★★★☆
LeadsRx (www.leadsrx.com)
Summary:
LeadsRx is a multi-touch attribution platform, offering advertisers with a method to combine several attribution methods in order to understand marketing performance through the view of up to 7 different attribution models.
The platform works based on cookie data and customer 1st party data to create an identity graph.
Using user personas attribution, leadsrx shows a simulated customer journey to offer precise attribution.
Pros & Cons:
- Leadsrx offers a multi-touch attribution model , but only works using cookies
- The platform does not work well for mobile advertisers
Incrementality Platform Reviews rating for LeadsRx:
- Ease of Integration: ★☆☆☆☆
- Pricing: ★☆☆☆☆
- Useful insights: ★★☆☆☆
- Unbiased results: ★☆☆☆☆
- Compliancy with privacy requirements: ☆☆☆☆☆
- Overall Score: ★☆☆☆☆