Channel 4 Television are using advanced optimization technology based on genetic algorithms
“The whole area cries out for advanced software technologies. But it is not an area that you can just jump into.”
That is how Stephen Ottner, Systems Manager at Channel 4 summarises the information technology perspective on the management of commercial airtime. With the ability to encapsulate existing constraints and accommodate new constraints, it is possible for knowledge-based systems to keep pace with a fast-changing market and with the developing nature of a TV company. It is a business wrapped up in a mystique and with a language all of its own – and one where skills and experience are highly valued. For Channel 4, sequencing of commercial breaks (a process that used to be undertaken manually) provided a logical starting point for the use of AI technology: “It was a bounded project, involving a limited number of users and limited risk, but with significant potential gains,” comments Stephen Ottner.
From January 1993, Channel 4 has been a statutory corporation responsible for its own financial future, dependent on advertising revenue for 96% of its revenue – and responsible for selling its own airtime (a responsibility that had previously been given to the regional ITV companies).
During the day, Channel 4’s strength is the housewife market whilst in the evenings Channel 4’s strength lies in its varied targeting ability. In comparison with ITV, Channel 4 audiences contain a greater proportion of younger, lighter, up-market, male viewers (audience research has also identified Channel 4’s ability to target cluster groups defined by names such as “Progressive Priscillas” and “Free-thinking Franks”). Unlike the ITV companies, Channel 4 also promotes the fact that it is able to give advertisers access to a UK network of all the ITV regions – booked from a single point. At the same time, regional advertisers can choose to buy Channel 4’s six so-called “macro-regions” (which correspond with the existing ITV regions) separately. The combination of two to five regions is known as a “Super-macro” and provides advertisers considerable flexibility in placing advertising copy.
Sequencing of commercials within each break has presented Channel 4 with something of a headache – ensuring that spots on different regions go out simultaneously whilst satisfying as many of the advertisers’ requests for special positions as possible: “Advertisers are buying audience, not airtime,” comments Stephen Ottner. “It is a complex trading commodity.” Although, in theory, a five minute break could contain 180 spots across all regions, in practice the maximum scheduled break is four minutes with 50 spots. Selling of this volatile medium is only closed off half a day in advance of transmission time, at which point the breaks need to be sequenced.
It was estimated that around 20% of advertising would be sold regionally. In fact, the figure is nearer 40%, half of which is made up of Super-macros. Advertisers may also specify to have commercials placed first in the break, last in the break or “Top & Tail” in a break -making break sequencing a challenge if optimal use of airtime is to be achieved. Definition of a knowledge-based system to solve the problem requires observation of a number of prioritised “rules”: Top of the list is the need for no overlaps or gaps, with Top and Tail or First and Last network spots also receiving high priority. Lower down the list are First and Last Super-macro spots and non-reporting Super-macros sequenced to play at the same time.
Stephen Ottner chose to use “XpertRule”, a respected knowledge-based system generator from Manchester-based XpertRule Software Limited: “Conventional programming could not cope with the need for additions and changes and simply putting a graphical front-end on the mainframe sales system would still leave the option for manual manipulation.”
A downloaded flat file from the mainframe-based airtime sales system (“SAMS”) passes an unordered list of spots in the break to the XpertRule Break Sequencer which uses “Genetic Algorithm” optimization to ensure that all constraints are complied with before the ordered information is passed back to SAMS.
Software Development Director at XpertRule Software, Haider Attar comments: “Optimization problems can be solved by iterative trial and error. But, as the number of possible combinations grows it becomes impractical to try all combinations to arrive at a solution in a reasonable time. Rule of thumb can be used to narrow down the options but, in most cases, good rules are not available or are difficult to capture.” He adds: “Numerical optimization techniques are currently available in most advanced spreadsheets, but these tend to be incapable of optimizing problems involving sequencing or scheduling and they are “exploitation” rather than “exploration” techniques. They also fail to cope with outcomes that involve subjective assessment by an expert.” The solution used by XpertRule involved the use of genetic algorithm techniques which allows the exploration of large search spaces for optimal or near optimal solutions. Random creation and regeneration of “Chromosomes” (solutions) and the mixing of “Genes” (parameters) allows a process of natural selection to create Chromosomes that have the greatest “fitness to survive” – a solution. Trading off time taken to reach the solution and the quality of that solution allows a “good” solution to be reached within a short time, as opposed to an optimal solution that might require infinite time.
Stephen Ottner comments on the fact that the initial specification for the Break Sequencer was contained within half a dozen sheets of A4 paper: “XpertRule Software have asked us what we needed, rather than tying us down to an all-embracing detailed specification. It is an approach that has allowed scope for change and discovery. Development has been an iterative process. In spite of the apparent mystique surrounding TV airtime, they quickly understood the business requirements and produced a workable solution without having come down here – they were good at getting to the heart of the matter. The target of one month for completion of the first phase of the project was actually bettered by XpertRule Software. One of the factors that attracted Channel 4 to XpertRule Software was the ability of XpertRule to run on PCs and under Microsoft Windows – two technologies that formed part of Channel 4’s IT strategy. XpertRule Software has also been able to incorporate PC graphics to simulate the graphical representation that users sketched when explaining how a break was built up – listing the regions along a horizontal axis and with time running vertically down the screen.
Stephen Ottner believes the fact that it has been possible to extract and return the existing mainframe data into and from the knowledge-based system has helped to build up confidence in using a system designed to take over much of the drudgery involved in break sequencing: “The team is now totally sold on the idea. Two or three people used to be wholly committed to sequencing work, whereas now it only requires one or two. It is an important motivation since the work is now manageable with the number of people available – they are not working just above the waterline.” Stephen Ottner stresses that none of the progress with XpertRule does away with the expertise of the staff. Indeed, one of the most recent enhancements to the system has been the addition of a facility that permits manual override: “There are many soft constraints that cannot be built into the system: Being sensitive to the positioning of products within a break; dealing with agencies and coping with necessarily detailed arrangements – simply making sure that there are no cock-ups. Looking at breaks is still important.”
Stephen Ottner acknowledges that there was initially some scepticism about the ability of a relatively small software house from Manchester to get to grips with the needs of a major TV company based in the heart of the West End. At present Channel 4 is planning to involve XpertRule Software in a much larger* project involving its advanced software technologies for a part of airtime management of much greater strategic importance than break sequencing. Implementation of the first project has been an important testbed for Channel 4 and has required an understanding of the role of human expertise as much as that of a knowledge-based system. Stephen Ottner summarises the benefits that are universally accepted: “Instead of simply managing breaks, we are able to find better options for the use of airtime. As far as we know, no other TV company does anything like this.”