### Why simulate ?

The purpose of a simulation of football matches is to provide probabilities for each team, participating in a scheme of scheduled matches, to reach certain feats. The feats can be anything from just winning a single match to winning a group in a tournament or win the whole tournament. Also, because each scheduled match gets a result assigned to it during the simulation, dependent facts can be quantified, such as the resulting FIFA points (and thus the resulting ranking) of all participating teams after the scheduled matches are 'played'.

### How does a simulation work ?

A simulation consists of scheduled matches where both teams in a match are known or where one or both teams can be derived given earlier simulated match results. A quarter final where the number 1 of group A plays against the number 2 of group B can be perfectly simulated as long as the final standing for groups A and B, resulting from earlier simulated group matches, can be computed first. Further more, both elo and FIFA update the ratings of a team directly after a match is played and the updated rating is input for the next match. This updating behaviour needs to be implemented in the simulation model to be able to calculate the correct ratings after the scheduled matches are 'played'.

Now, of course you can assign one match result to each scheduled match once and determine the things you are interested in, but that will not give you the probabilities you are looking for. For that you need Monte Carlo simulations or a high number of random samples from a probability distribution for your main random variable: in this problem the match result of each scheduled match. And the larger that number of samples (N), the more reliable the resulting probabilities will be.

So, technically, to perform simulations this process is followed:

in one simulation the results of all scheduled matches (included in the simulation) are determined and all things you are interested in, are registered after that single simulation. That can for instance be the final ranking of teams in a group or the composition of knock-out matches in a tournament or the resulting position of all teams in the FIFA ranking. Then you perform that single simulation 10.000 times and vary in each simulation the match result of each scheduled match. After each single simulation you still register those same things you are interested in. After the 10.000 simulations are performed, you can just count in the resulting registration let's say the number of times a team ends at first place in a group or as a tournament winner or at spot 4 in the FIFA-ranking. And you get a probability for that event by just dividing that counted number by 10.000.

### Predicting the match result

To be able to vary the match result in a meaningful way in each simulation we are looking for a good, describing probability distribution for the match result, given both opponents in the scheduled match and more specific their mutual strengths. A probability distribution is the mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment. It describes the probability of something happening as a function of an independent variable. The independent variable must be computable with a non-random formula.

For this problem we could do with one computable, independent measure that expresses the win expectancy of a team in a match against an opponent, based on a difference in strengths of both teams and possibly some other factors that can deliver a meaningful contribution to the match result. You can think of for instance the actual 'form' of a team, the line-up, their system of play and the qualities of players the coach selected to execute his/her ideas. But these factors are all difficult to quantify easily, because you will need to gather and maintain a lot of detailed extra information about each match in your dataset. We need simple but meaningful factors or no other factors.

### Elo win expectancy

Elo (http://www.eloratings.net) has defined such a win expectancy and it can be easily computed because it is only dependent on the difference of strengths of both teams and one other factor which has a proven meaningful contribution to the match result and is very easy to determine: the Home Field Advantage (HFA). When a team plays the match on home ground its win expectancy rises significantly, of course dependent on the difference in strengths of both teams. When a home team is much weaker than the visiting team the HFA will contribute very little to the win expectancy of the home team (less than 2 percent points). However, when both teams are about equal in strength the HFA will contribute up to 14 percent point to the win expectancy.

The win expectancy (We) according to elo is calculated as follows:

where dr equals the difference in elo ratings of both teams in the match plus 100 points if the match is not played at a neutral venue. If a match is played at a neutral venue the 100 HFA-points are ignored.

But the elo win expectancy is still not the probability distribution from which we can sample, it is the independent variable. Besides, to be able to properly calculate the final group standing from some scheduled group matches we need information about the exact number of goals scored by each team in each match, because goal difference and scored goals over all group matches are also important tie breakers (when match points are equal) to determine the final group standing. So we need a probability distribution that describes the number of goals scored by a team as dependent on the elo win expectancy in a match.

### ClubElo

Lars Schiefler from ClubElo (http://clubelo.com) estimated such a probability distribution function. He estimated his probability distribution function on a very large dataset of football match results. Only little drawback was that it contained only matches with clubs involved, played in both domestic club competitions and in international club competitions like the European Cups.

Until now I nonetheless used his function in my simulations because it was available and tested and others used this function as well, so I was able to compare my simulation results with them. But over the years I came to the conclusion that the characteristics of NT-football might be a little bit different from club football. To research this I decided to estimate my own probability distribution function based on a very large dataset of NT football match results.

### Probability to score goals

Some theory first. The probability to score a certain number of goals by a team in a match is described by the Poisson distribution. This distribution has one parameter that is equal to the expected value of the distribution. The assumption is that the expected number of goals is depending on the elo win expectancy.

The Poisson distribution:

where:

For example:

in case the average expected number of goals is 0,5

etc.

in case the average expected number of goals is 1,9

etc.

The average expected number of goals G is what we are trying to determine. So we need to determine G as a function of the independent variable 'elo win expectancy':

This function is estimated based on almost 40.000 NT-matches in my match database using the least squares method which minimizes the vertical distance between function value and data point for each elo win expectancy. I apply polynomial regression so that the function is modelled as a 4th degree polynomial in independent variable We.

### Data research and results

In the graphs below you see the data points I derived from my database. One graph for matches on non-neutral or home ground; one graph for matches on neutral ground.

On the X-axis of each graph the elo win expectancy is plotted ascending from 0 to 1, on the Y-axis the average number of goals scored is plotted. In the 'home ground' graph you see two lines: red is for the average number of goals scored by the home team in a match with the given win expectancy, the blue line depicts the average number of goals scored by the away team in a match with the given win expectancy (for the home team). In the 'neutral ground' graph only one line is plotted, there is no home advantage in these matches.

The estimated functions for **non-neutral or home ground** matches are:

for the average expected number of goals for the home team (Gh):

for the average expected number of goals for the away team (Ga):

The estimated functions for matches on **neutral **ground are:

for the average expected number of goals (G):

The win expectancy We for the 'away' team in a match on neutral ground is equal toIn the graph below these equations are compared with ClubElo's probability distributions, for non-neutral matches only as ClubElo has no separate functions for neutral matches.

### Win, draw, lose probability

About me:

*Software engineer, happily unmarried and non-religious. You won't find me on Twitter or other so called social media. Dutchman, joined the blog in March 2018.*

Ed- this is probably a dumb question, but for the non-neutral matches, the We for the away team refers to the away teams We, correct? (That is, 1 minus the home teams We)?

ReplyDeleteAlex, no that's not correct. I should have emphasized this more in the paragraph Data research and results.

ReplyDeleteTo be absolutely clear: for non-neutral matches the We is always the win expectancy for the home playing team, the team that plays at a venue in their home country (and you understand that's not always the same as the first mentioned team in a match).

Only for neutral matches you calculate for each team in a match their own win expectancy, of course not including the HFA factor in this case.

Great- thanks so much

Deletehi. great work. thanks

ReplyDeleteI've just now saw this - amazing work!

ReplyDeleteNext stage would be to calculate your own ELO ratings, that maximize predictabity given your dataset :)

ReplyDeleteQuestion - how are 30-minute extra time and penalty shootouts considered?

ReplyDeleteAssume you have a one-off play-off match in the simulation. The result of the match is drawn from the probability distribution functions: the number of goals scored by the home team and the number of goals scored by the away team is determined.

ReplyDeleteIf those numbers are the same then the end result of the match is a draw. So this draw is reached after extra time (if that's played in the match) or after regular time (if no extra time is played in the match). So 'extra time' does not really occur in the simulation.

However, this match as a one-off play-off needs a winner. In that case a penalty shoot-out is also simulated. And that's done rather simple and completely dependent on the elo win expectancy for the home team in the match: if that's higher than or equal to 0.5 then the home team is the winner of the PSO, otherwise the away team is the PSO-winner.

So then the, in terms of elo-rating combined with home advantage, strongest team always wins the PSO.

One could argue that the result of a PSO should be drawn randomly from a uniform distribution, but analysis of all PSO-matches in my match database showed that in some 60% of the cases the team with the higher elo win expectancy won the PSO. That's why I decided to let the PSO result be dependent on the win expectancy, although I realize that now each simulated PSO always results in the 'elo strongest' team winning it.

Hi, Firstly thank you so much Ed for maintaining this wonderful website. I've been a big fan for a long time and hope you can keep it up long into the future!

ReplyDeleteRegarding your above reply about extra time and penalties which I've only discovered now (and apologies if you've addressed it in a more recent post):

1. Have you since made any improvements to your model so that it doesn't assume the stronger team always wins the PSO? It seems to me that this will overestimate the strongest teams' chances of progressing in a knockout match by perhaps over 5 percentage points in the case of two evenly matched teams, which feels pretty significant?

2. It seems to me a less significant inaccuracy is not accounting for the fact we expect more goals in a match that could go to extra time than a normal match, which could in turn affect the win/draw/loss probability. Have you considered anything to account for this?

Thanks :)

Just had another thought: for KO matches where a draw (after a PSO if needed) is never the outcome, is the elo win expectancy already a very good (if not exact) indicator of the probability of winning the tie, without the need to transform the number via simulating each team's goals scored etc.? Have you considered this when simulating knockout stages for the purpose of calculating chances of each team winning the tournament etc, when, as well as a draw not being possible, goal difference and goals scored can't determine which team progresses?

ReplyDeleteHi Ed (that's a bit confusing :D ),

ReplyDeleteThanks for your kind words.

I have somewhere in 2022 (after the comment above that triggered me also) adapted the PSO-handling in my simulation model. Since then it determines the PSO-winner based on a random draw from a uniform distribution between 0 and 1, which is compared to the exact chance that the higher elo-rated team wins the PSO. This probability I've determined (again) from my match database and was 56.9% at the time. So if the random drawn number is less or equal to 0.569 then the team with the higher elo rating (at the moment the match was played) wins the PSO, otherwise the other team.

Regarding the extra expected goals in extra time: I've determined the basic models about the correlation between the elo win expectancy and the scored goals per team using the match results from all matches in that database. For me, as is the common ground rule, the result after extra time (in case a match goes to extra time) is 'the result of the match' and in my analysis I've handled those results completely the same way as a match result 'after 90 minutes'. And nowadays, with refs taking more and better action against time wasting and also the, sometimes extensive, VAR-checks, the 90 minutes are frequently extended with 5 to 10 minutes 'extra time' already. I don't account for those implicit extra minutes either.

As you say, I expect it has no (at least no significant) effect on the simulation results.

Your third point about using the elo win expectancy directly as the measure for determining the outcome of KO-matches would imply to implement a completely new path in my simulation model, just for this type of match. The currently implemented method already gives the result of such a match after extra time (which could be a win, a draw or a loss for the home team), using exactly the same model as when determining the result for regular matches 'after 90 minutes'. So, in case it's a KO-match AND the final match result is a draw, I only have to do one (small) extra step: determine the PSO-winner as described above.

As a software engineer I always try to make generic code that's useable for all situations (in this case for all kinds of match types).

Thank you very much Ed (sorry that my parents couldn't pick any other name :D), that all makes sense.

ReplyDeleteRegarding extra goals in extra time, do you know how many of the 40000 matches in your database went to extra time? Presumably a low enough number that their effect on inflating expected goals for a normal match is very minimal?

On the last point, I completely understand it's not worth implementing a completely new path in your simulation model, especially now the PSO issue discussed above is fixed. My question now would simply be: for any given elo win expectancy x, how close is the win probability (in a knockout tie) your model spits out to x? Presumably you would ideally want them to be the same? Or maybe if your model comes out with a different probability to x, then we can take this to suggest elo win expectancy is not actually a true measure of win expectancy (for football at least)? Either way, I'd be very interested to see a graph of elo win probability Vs your model's outputed KO win probability if you have it?

There are 652 matches that went to extra time in a total of 46861 matches in my database at the moment. This is 1.4% of all matches.

DeleteThinking some more about your suggestion: the elo win expectancy can't be used as a direct measure to determine the outcome of a KO-match, because the match result 'draw' isn't incorporated into it. Just take a look at the first graph in my post above in the paragraph about Win, draw, lose probability. If the blue line was a straight line from (0, 0) to (1, 1) then we could use this measure for that purpose. But you see that f.i. a We of 0.6 results in a win probability of something just below 0.5.

You see in the second graph that for almost all win expectancies there is a probability to draw, growing to a substantial 27% for a win expectancy that's near 0.5.

Second thought: you wanted a predictor for the KO win probability. If you assume that roughly half the draw probability for a given win expectancy can be added to the direct win probability of a KO-match (as PSO winner in a KO-match) then the graph of the win probability of a KO match versus the win expectancy of the match could well be something of a straight line from (0, 0) to (1, 1).

Delete