You’re familiar with the general concept of virtual advertising, but can’t sleep until you know EXACTLY how the technology works? You’re not alone. So for our fellow lovers of AI, machine learning and all things techy, we’ve put together this handy guide to explain just how a software-based, broadcaster-friendly virtual advertising solution works.

When selecting a virtual advertising platform, the main components to consider are:

  1. The tools and technologies required for each phase of an event;
  2. The architecture;
  3. The real-time technologies.

So, let’s dig in…


Virtual advertising platform: three phases of tools and technologies

First, it’s important to understand the scope of the virtual advertising platform. Each phase requires its own set of tools and technologies to be successful.

Pre-event

For the preparation phase, your virtual advertising system should include an event management system. This will allow you to create an event schedule, upload the artwork, create zones where the artwork will be placed and sequence the play-out of the artwork throughout the event. It should be easy to access (web based and cloud hosted), easy to use, as well as have the ability to implement both strong controls (to restrict access) and business logic (to ensure appropriate combinations of sponsors/brands).

Live event

This is where the magic happens – replacing physical space and objects with virtual assets into a live television broadcast. The virtual advertising platform needs to be highly accurate to replace only the required pixels, very fast so as not to significantly delay the broadcast output, incredibly reliable to ensure the expected virtual minutes are delivered and easy to integrate within a complex live production. *see real-time technologies below for more details on how this is accomplished.

Post-event

After the event, your virtual advertising platform should provide sophisticated and intuitive reporting and analytics to all stakeholders (the club/agency/league and their brands/sponsors). The detailed implementation report should be supported with highlights videos and rich data, and be presented in a user-friendly web-based UI that respects league rights on video distribution.


Virtual advertising architecture: modular and distributed

Second, consider the sports event for which you’d like to implement a virtual advertising system. For each event, the size of the production (number of cameras, broadcasters, production integrations, etc.) and the size and distribution of the audience will vary greatly. One size doesn’t fit all, as was made apparent during the recent pandemic. With on-site operations drastically limited, the need for flexibility has never been higher. Therefore, your virtual advertising solution should be designed in a way that efficiently and economically accommodates these varying needs.

A modular and distributed architecture will allow for maximum flexibility during deployment. That means, video inputs, production integrations and virtual outputs should all operate independently so that each component can efficiently scale. This type of architecture is well-suited for a wide variety of deployment models, particularly so for remote (REMI) production, where live content is captured from a remote location and managed from a central control room.


Virtual advertising real-time technologies

Finally, let’s define the real-time broadcast quality technologies that enable a virtual advertising solution to work its magic. During a live sports production, the quality thresholds are extremely high and the latency needs are very low. This is due to the fact that the single most valuable asset of any rights holder, the world feed, is being virtually modified. Most importantly, the end result must look so natural and visually realistic that the audience would never guess they are seeing a virtual ad placement.


Keying

The keying sub-system is responsible for managing occlusions in the area where a virtual asset is to be drawn. That means, identifying foreground objects (players, balls, etc.) as separate items from the background surface (an LED board or grass area). Failures in the keying mechanism can cause foreground objects to be drawn over, or background areas to not be drawn over, both of which result in a poor viewing experience and break the illusion of augmented reality.

There are different technological approaches to keying, each with their own strengths and weaknesses, such as:

Chroma keying

An older and simpler technology, chroma keying is lighter in terms of deployment hardware, but is also more susceptible to changing conditions and has limitations related to color clash;

Infra-red light keying

A very robust technology in both indoor and outdoor settings, but infra-red keying is more complex and costly to deploy with specialized hardware required at the venue

Computer vision based keying

A newer technology, which can offer the best combination of deployment simplicity and general purpose performance. However, the performance is heavily reliant on the quality of the AI implementation, which can vary greatly between vendors.


Tracking

To provide a reliable target for virtual ad replacement, keying must be complemented with tracking. Tracking is the technique used to keep the virtual objects locked into their correct position in the scene. During sports production, the cameras pan from side to side, zoom in/out and are adjusted (racked) based on lighting conditions at the venue. These dynamics change the perspective of the scene as captured by the camera and in turn the virtual objects being placed. Accurate tracking ensures those virtual objects do not wobble and appear stable, thus providing a seamless viewing experience.


Rendering

The final stage of the real-time virtual advertising process is to insert the virtual ads into the live feed. Sophisticated rendering is required to accomplish this in such a way that the advertising looks authentic while providing multiple safety mechanisms (such as automated or manually invoked bypass and failover) to ensure the virtual insertion “does no harm”. It’s easy to provide crisp graphics that look pixel sharp, but would appear unnatural and misaligned to the actual weather/lighting conditions such as sun and shade, rain, or snow. The rendering process must also support all of the broadcast signal flows such as HD, UHD, HDR and whatever the future holds.


Hopefully this guide has provided you with the technical details you were craving to finally rest easily.

Still have questions? We could talk about this ALL DAY. Give us a call and we’ll gladly indulge in more super-techy chit chat about virtual advertising solutions.

Steve Plunkett

Chief Product Officer | Supponor