Of cricket and climate

An unfolding saga in the game of cricket in recent years has been the question of whether technology should be used to aid umpires in the case of close or potentially controversial decisions. Like many things in life, the question of whether to do this has turned out to be more complex than it may at first have appeared to be. There have been situations in which the on field umpire has asked for a replay, and the replay has been unclear but has none the less been used to overrule an on field umpire who probably saw more. There have been situations in which the players have appealed to a video replay that didn’t show anything, when they likely knew themselves what had happened and the situation would previously have been resolved with a gentlemanly code of conduct. There have been situations when the television company did not manage to produce the appropriate replay in time, and the umpire then made a decision that was revealed to be incorrect five minutes later. Many decisions depend on whether the batsman hit the ball, and a mixture of sound and picture is used to make these decisions, and determining which items of the bat, ball, ground, clothing, safety equipment etc came in contact with each other is often no clearer using the technology than not.

Fans of other sports are no doubt nodding at this point, as similar issues have come up in most sports that have attempted similar things. Cricket has one further issue, not unique to it but relatively central to it, which is the question of how technology should be used in the interpretation of the leg before wicket rule (LBW).

One of the principal ways of getting a batsman out is for the bowler to bowl at the batsman, for the batsman to miss the ball, and for the ball to then strike the wooden stumps behind him. It would be possible for a batsman to avoid getting out this way by his simply standing in front of the stumps at all times. In order to avoid this, the rules of cricket allow for a batsman to be out if he is standing directly in front of the stumps, the ball hits his leg, and the ball would have gone on to hit the stumps. (The rule is actually more complex than this, but the complexities are not relevant to the point I am making). The umpire stands in a good position from which to judge whether the ball will hit the stumps, and traditionally the umpire’s judgment has been used to decide whether the ball would have hit the stumps.

Umpires inevitably make mistakes, and there have been many accusations of umpire bias over the years. For a long time people have been watching replays in slow motion on television in order to second guess umpires, but these have never been conclusive. Occasionally an umpire will make an obviously wrong decision, but most of the time there is as much of an element of doubt watching at home as there is for the umpire. Or perhaps more: the umpire is in a better viewing position than the TV cameras.

In recent years, however, things have changed. The “Hawk-eye” system was initially used by television companies, and there was then pressure for it to be used in assisting umpires as well. Basically, this system looks at a number of video replays, and from them constructs a three dimensional model of the ball, the pitch, the bat, etc. From the this model, the path of the ball is extrapolated going forward. Television viewers see a computer graphic image of the ball hitting (or not) a computer graphic image of the stumps, and are told whether the ball would have hit the stumps and whether the batsman was or was not out.

Every since this system has been in place as part of television coverage, there has been pressure for it to be used in umpiring decisions. When people have asked me about this, I have stated my position with unexpected vehemence, particularly given that I am generally in favour of using video replays as part of the adjudication process. For I am, at present, unequivocally opposed to the use of Hawk-Eye and similar decisions in umpiring decisions.

My reason for this is as follows. Hawk-Eye is a system featuring a lot of complex computer code. The code is proprietary, so what follows is largely reasonably well informed speculation. Although we do know the laws of physics with respect to motion of cricket balls, air resistance, the effect of gravity, bounce when the ball hits the pitch, linear and angular momentum, etc etc etc, the complexity of a even a relatively simple system such as a cricket ball moving in a cricket game is such that it is difficult to impossible to develop a useful model directly from the physics. In addition there are margins of error when triangulating the motion of the ball from video imagery. What this means is that Hawk-Eye’s models are not really physical models per se. What they have likely done is a more simple matter of trial and error followed by extrapolation. The ball has been measured going through the air, perhaps half way down the pitch. Various ways of further extrapolating the position of the ball have been tried, and they have been compared with the actual motion of the ball further down the pitch. Trial and error has continued until prediction and reality have become close, and the resulting (statistically derived) algorithm has been used to predict the motion of the ball in cases where the ball has not actually traveled the full distance (because, perhaps the batsman’s leg was in the way).

This is all actually fine, except that there are circumstances in which the system can break down. Weather conditions that were never encountered in the development phase might be one. Balls made by different manufacturers might behave differently. Pitches in different places may be made of different kinds of grass. In cricket, local conditions matter a lot, and pitches in different countries are known to have different characteristics and favour different types of bowlers and bowling. (The greatest players will succeed in any conditions, but some players will only be effective in certain kinds of conditions).

These sorts of empirical models are not always terribly robust: the further you are from looking it the actual physics the more likely that your software will do a poor job when you depart from the exact conditions in which you did your calibration. A model developed in English conditions by Englishmen may not work as well when used in India. It may contain biases, either accidentally or deliberately.

However, there is a dangerous tendency of people to fall in love with the technology: in cases where the technology disagrees with reality, then it is reality that is at fault. “Science” has given you a definitive answer, which is obviously right. However, if you dive into the science, you will discover it has been developed using a methodology involving a lot of ad hoc steps and statistical approximations and errors.

It may well be that Hawk-Eye, as it exists, is better than the unaided umpire’s eye. However, arbitrary choices have gone into its design. Human biases have gone into its design. Different, equally skilled programmers may have made different choices. Hawk-Eye is not definitive.

In tennis, Hawk-Eye is used in umpiring decisions. Players have the right to challenge line calls. When they do so, a simulation of the ball hitting the line on the court is shown, and the computer rules the ball in or out.

And yet, when this decision is made, all the umpire sees is the simulation. A simulation of the ball intersects with a simulation of the line. There is no attempt to superimpose the simulated ball on the real ball, or the simulated line on the real line. It is all technology. Players have from time to time complained that the computer’s decision is wrong. And yet, on television there is no way of telling that.

Given that Hawk-Eye is not definitive, another nasty possibility rears its head. Like baseball, cricket is a game greatly suited to gambling. There are a huge number of statistics that followers of the game are interested in, and it is possible to bet on the outcomes of most of them. Cricket has had a significant number of corruption scandals in recent decades. Most famously, a captain of the South African national team took bribes from Indian bookmakers to lose matches. (He later died in a mysterious plane crash). Many other players took money from bookmakers in return for various favours, more of which probably influenced specific match statistics rather than actual results of matches. The manager of the Pakistan team may or may not have been murdered by gangsters for reasons related to gambling during the last world cup.

Imagine then a situation where statisticians and programmers are running a system of computer code that nobody understands (and which, in fact, they are extremely secretive about the workings of, on the basis that it is “proprietary intellectual property” and a valuable trade secret) which has the ability to overrule umpires decisions. Nobody outside the firm they work for (and indeed few people inside the firm) knows exactly who the people are who run this stuff – certainly not the official administrators of the game. The potential for corruption is obvious.

Far fetched? Well, the best way of running such corruption would be to not be obvious about it. Decisions should still appear approximately right. You do not change the result of matches, but apply a systematic bias of 5% in the direction that improves your profitability. As it happens, I am not a bookmaker, but I am a financial analyst. Give me a 5% systematic bias in the financial markets that I can control, and I will shortly be a very rich man.

You don’t believe that bookmakers could figure out who was in charge of this code, who was vulnerable to bribery, and get to them? Well, bookmakers have got to a lot of people in this game. Players, officials, and coaches. And as far as understanding technologists and statisticians, bookmakers are way ahead of just about everyone. Bookmakers these days employ huge numbers of programmers and statisticians. If you think about their business for even a moment, it is obvious why. In fact, it is likely that statisticians and programmers in both these places know one another already, as they were likely recruited from the same pools of people – mathematically adept obsessives who love sport.

Which is why I have been opposed to the use of Hawk-Eye or similar systems to assist umpires in cricket. The potential for corruption is too great.

Except, when people have pressed me further, I have qualified this a little, and I have actually said that Hawk-Eye and similar systems should not be used unless the code and the data are open. If people at home are able to go through the code line by line, see what it does, and then run it themselves, and duplicate it themselves, using the same video footage being used by the actual system assisting the umpires, then any corruption will be visible instantly, and the game is safe.

As it happens, I suspect that the administrators of cricket and the television networks broadcasting it consider the footage (ie the data) to be too valuable to make it available to everyone. I suspect also that the developers of Hawk-Eye consider their intellectual property to be too valuable to make the code available for technically minded cricket fans to run at home. If so, then fine. LBW decisions should be made by the on field umpires. They may be corruptible, but at least they are the people responsible for their actions, rather than some unidentifiable programmer. Hawk-Eye is a wonderful tool for observation and commentary on the game, but involving it in the administration of the game raises issues that its developers may not want to deal with.

This story, of course, is very similar to the stories of many other fields of endeavour, particularly scientific endeavour. A huge amount of modern life, and a huge amount of modern science, involves computer simulations of statistical systems, either in the foreground or the background. The level of complexity and the level of obscurity of these systems is such that (intentionally or unintentionally) such systems are very vulnerable to becoming sloppy, biased, or corrupt.

This, of course, is why, if it deals with anything important, such code must be open. And data must be available. Outside eyes must be able to investigate such biases, and the possibility of the presence of outside eyes must be there in order to discourage corruption and simply to discourage sloppiness and detect genuine errors. And the data must be available, so that other people with different biases can construct competing models, as, always, little is ever settled and improvement comes through competition.

It is when results are robust to such biases that interesting things are discovered.