Header bidding: Past, Present and Future Challenges
With the introduction and wider adoption of header bidding the market has been flooded by increased volumes of ad impressions – both on the sell and buy-side. While it has been addressed by increasing server capacity and adding new ways of throttling out non-relevant traffic, the latest casualty of header bidding is the second-price auction. In a matter of months the industry has pivoted from second-price auctions to first-price auctions. The second-price auction is not to be blamed, but the lack of transparency in how SSPs were determining the clearing price within such auctions has led to demand for an added level of clarification, and as a result, a switch to a first-price based approach.
This is great news for all involved and highlights how programmatic has evolved to deliver better bid processes and significantly expanded access to demand. It also highlights the growing role that powerful analytics and algorithmic intelligence play in providing tangible solutions that replace complex workarounds.
In order to better understand why SSPs are taking this step and what this means for programmatic, let’s look at how header bidding evolved, which problems it solved, and what challenges it created.
From waterfall to header bidding
The SSP’s primary role at creation was to aggregate demand for a publisher by integrating with relevant demand partners (DSPs). At the start, only one SSP was needed to accomplish this goal. In this environment, bidders lacked concrete insights into what a fair price might be. To solve this second-price auctions were adopted as an industry standard as they gave bidders an incentive and peace of mind to bid based on the value they attached to the impressions (https://en.wikipedia.org/wiki/Vickrey_auction). The publisher created a set floor price and the middle ground / ideal price was then established between the first and second price bids.
But, with time, the number of SSPs and DSPs grew significantly. Publishers then needed multiple SSPs, all of which had to be integrated, in order to access total demand available to the publisher in the open market. The solution to this was the introduction of “waterfalls”, a cascading series of SSP calls that solved the problem of multiple SSPs but led to a host of secondary issues.
Eliminating waterfalls and their added complexity and related issues was one of the leading reasons for creating header bidding solutions. The waterfalls created latency while failing to give publishers the very thing they’d initially been implemented to do: deliver the best single impression value.
In FIGURE 1, the waterfall model, we see that publishers miss the opportunity to sell an impression at the highest possible price and must accept unnecessary latency as they ask the SSP for bid responses and make decisions one SSP at a time. In this case, this is also only a subset of what’s going on, as additional impressions have already been reserved before the waterfall was initiated by the ad server in order to deliver non-programmatic campaigns.
The split between programmatic and non-programmatic that results marks a significant missed opportunity, as in many cases programmatic can outperform non-programmatic demand for a specific impression.
In FIGURE 2, the header bidding model, traditional header bidding is shown illustrating how it simplifies the process. This also highlights how header bidding runs before the ad server is called, then rapidly identifies a programmatic price which is fed into the ad server to make a decision.
The value of second price and the rise of the first price auction
Header bidding’s success is based on its ability to enable a publisher to have winning bids from multiple SSPs compared in parallel and to decide on the overall best price across all of them. But for every single SSP (and all integrated DSPs) it means that they suddenly have to handle an increased amount of requests without an actual promise to monetize the impression by rendering the ad. Apart from impressions being dedicated for guaranteed delivery that, within traditional publisher ad server, comes before non-guaranteed or price priority (and header bidding demand as well), there is also a competition between SSPs in choosing the best clearing price.
Even with different floor prices, soft floor prices and different methods applied in order to pick clearing price, the value of bids passed from the SSP to header bidding for the final decision isn’t necessarily the value the Advertiser was willing to pay. This potentially means less revenue for the publisher and a decrease in an advertiser’s ability to win the impressions they want.
How might this play out? Let’s look at two scenarios:
An Advertiser using a header bidding connected SSP bids $10. Within the second-price auction in SSP1 the second-price bid is $5. The winning bid that gets passed from SSP1 as the final bid to the header bidder for the Advertiser is $5 dollars. However, if the winning bid that comes from another Advertiser using SSP2 is $8, that bid beats the $5 bid passed from SSP1. This despite the first Advertiser’s willingness to pay $10 in order to secure those high-value impressions. In this instance, even if the first Advertiser bid $15, they would still lose the auction due to the second-price dynamic operating between the SSP/header bidding.
In the second scenario, the header bidding solution issues bid requests in parallel through all connected SSPs. However, DSPs no longer exclusively connect to one SSP. As a result the DSPs will often bid for the same impression through multiple SSPs. This means that the advertiser having the highest bid, will win through the SSP having the highest clearing price on the specific impression in the header bidding decision. In other words – the SSP that somehow yields the highest clearing price on the same bid, from the same DSP, will win and earn their fee.
These challenges resulted in a fresh wave of workarounds with SSPs competing to clear the winning price as close to the first-price as possible in order to more effectively represent the winning bid in the header bidding decision.
Some of the most common work arounds include:
- Modified clearing prices, where the price is chosen in-between 1st and 2nd highest prices,
- Bid price reductions, where the winning price is a pre-defined fraction of highest bid price,
- Intelligent optimizations of floor or soft floor prices to be set as close to the highest bid as possible before the bid requests are sent out to DSPs.
These workarounds are well intentioned but problematic as they break the ground rules of the second-price auction, tamper with transparency, and undermine trust. The result creates as many issues as it solves for both buyers and sellers.
In Adform, we believe that this kind of header bidding gamesmanship, without clear rules by all bidding parties is bad for the industry and for both demand and supply partners. Lack of transparency within second-price auctions and the related setup now magnify the very issues they were implemented to solve. Within a header bidding-driven ecosystem, the only option is to phase out second-price auctions and to adopt first-price auctions.
With first-price auctions, the SSP takes the highest bid, passes it on to the header bidding solution, and then all bids are evaluated fairly and transparently on equal footing. No funky algorithms or custom logic are applied and bid prices remain fully transparent and traceable.
Adform SSP will be transitioning to first-price auctions for all impressions originating via header bidders starting in October with a complete transitioned planned by the end of 2017.
Will the industry repeat the same mistakes?
While the transition to first-price auctions will solve SSP challenges in clearing a competitive price within current header bidding solutions, we believe that there will be more movement towards first-price auctions in the future with the goal of bringing more transparency to programmatic trading. It does not necessarily mean that second-price auctions will disappear. However, to remain in use they will need to be 100% transparent and this is unlikely to happen within SSPs. For DSPs, bidding in a first-price auction is less of a challenge as they have adopted intelligent bidding algorithms designed to find the price point at which they can win the impressions they need. Thus DSPs are instead more likely to continue focusing on supply path optimizations.
Looking forward, the majority of communication between ad tech solutions will happen server-side. This unlocks a wide range of new possibilities and significant efficiency improvements compared to information exchanged client side. Server-side integrations allow the provision of more than one bid from SSPs to header or exchange bidders, where they, while being integrated into the ad server, can work collaboratively for better decision making and competitive pricing which reduces the tech tax while being fairer for both publishers and advertisers.