Of late, the significance of using testing and optimization to improve conversions has increased manifold, however, the use of recently developed A/B testing tools has been mostly monochromatic ^[1].

The reason why I say this is since testing and optimization is primarily being used by a large segment of online advertisers/marketers and majority of the traffic on these Test pages, comes from paid sources. On the other hand, an equally large segment – comprising of publishers, bloggers and webmasters have not reaped any benefits yet. We’ll try to explore an important possible use of A/B Testing for publishers.

What can Publishers do with A/B Testing?

A/B Testing and Multi-Armed Bandit solutions can be used by publishers to increase their ad revenues by finding the best possible ad location, size and color theme, among other factors.

Many people argue that a red button instead of green may increase button clicks, but cannot increase sales, at the end of the day. Keeping my views on the matter aside, the point is who can argue when more clicks directly mean more revenue? That’s the benefit of A/B Testing for Publishers. So if your page layout A is getting more clicks on ads than page layout B, it directly converts into more revenue for the publisher, end of story.

Current Scenario

In the last few months, I have spoken to a decent number of publishers and almost none of them have used A/B testing to improve ad revenues.

Limitations of Current A/B Testing Softwares

Widely used A/B Testing software are not made for Publishers. Here’s why:

• Can’t Track Ad Clicks to measure results: Since most of the Ad Networks include the creatives in the form of an iframe, A/B Testing softwares are unable to track ad clicks. Not to forget it is also a program policy violation for many ad networks to measure ad clicks directly using any analytics or testing software.
• No Support for creating variations automatically: A/B testing ads can be slightly more complicated as the number of variations rise exponentially. Typically, you would want to show 3 Ad units on a page and you have 6-7 Key spots, 2-3 important ad sizes for each locations and 5-6 color themes for each spot. Number of variations would easily reach 100s and there is no reason why creating variations should not be automated. Of course, creating a very large number of variations is only advisable for large publishers.
• Manually Editing Code does not help either: Even if you manually create the new ad code (which you want to compare the control with), say in the Adsense‘s Ad Management panel, you can’t just edit the HTML code of the variation, inside popular A/B testing tools (most of them are client-side). This will not work. This is because A/B testing tools (again, the client side ones) push the updates/changes (which you made) on the page, after the page load. The Adsense JavaScript will not work, and not to forget – messing with the Adsense JS code is against program policies.

Towards MAB and Continuous Optimization

Technically, An ideal product for Publishers will not be A/B testing, what they should use is other Multi-arm bandit solutions, simply because optimization should be seen as a continuous process.

Anybody who have changed layouts for a website which servers ads would notice the CTR going up, right after the design/layout change. This happens primarily because your layout develops banner blindness over time. Continuous MAB Testing is the perfect solution. Creating a leading indicator of change in the user behavior towards Ad units isn’t a difficult task which can help keep Ad units performing well all the time. In a non-technical language, Once you start the experiment on your website, Continuous Optimization and MAB will continue serving majority of your impressions to the best performing variation and as soon as the system notices a (statistically significant) drop in the CTR, the system is capable of finding the best possible variation it self. It is also technically feasible to use machine learning to find the best spot for your advertisements (outside the defined variations) and changing it, as the user behaviour changes.

Case Studies: Will it Really Help?

Ad size: Check this video from the Adsense Team. In a live example which they demonstrate, the replacement of 120×600 Skyscraper with a 160×600 Wide Skyscraper and replacement of a 468×60 Banner by a 300×250 Medium Rectangle, caused a 109% Increase in Adsense Revenues for that website.

Text vs Image Ads: Google recommends that you enable both – image and text ads for your website. Stats speak otherwise. Keith Lock from Webmaster.net improved his earnings by a whopping 475% after testing between image vs text and sticking with text ads.

^{[1] A/B Testing is still being mainly used by internet marketers and advertisers. However, off late, Mobile App Developers have also started using A/B testing. However, no data is available on how well it has been adopted.}

^{Disclaimer: We, at Ad Pushup, are developing an ad revenue optimization tool which will use MAB solutions to help publishers improve ad revenues. We’re calling it an A/B testing tool, simply since since majority of our prospects do not understand MAB at the moment.}

---