12 Aug When ‘best fit’ is not the same as ‘perfect fit’
Some time ago I wrote a piece about the problems of benchmarks in the financial world. Having just issued the half yearly profile reports, where we talk about CDC performance against a benchmark, it seems an appropriate time to revisit the issue.
The Financial Conduct Authority (FCA) requires Wealth Managers to provide investors with a means to be able to make an assessment of investment performance. This is widely interpreted as providing a ‘benchmark’ to allow investors to make a reasoned judgment on whether their manager(s) is doing a good job. Whilst the FCA guidance is explicit that a benchmark has to be provided, it falls short of saying what type of benchmark. Not surprisingly this is interpreted differently across the industry.
The problem for the finance industry (not to mention investors) is that the perfect benchmark simply does not exist. An effective benchmark needs to fulfil two functions, reliability and validity. The best way to think of reliability is to consider its repeatability. A good example of this is the humble classroom ruler, which as long as it is calibrated correctly, will give reliable measurements time after time. Most benchmarks carry a high degree of reliability.
Validity is different. It is concerned with the extent to which a benchmark measures what it is supposed to measure. It is the latter point where most benchmarks fall down and cause financial professionals to have sleepless nights. Back in 2012, when we launched our discretionary service, we used the APCIMs series of benchmarks. We did so on the basis that it was a recognised industry benchmark, it was regularly published and it changed very little, meaning it met the reliability test. It also quite closely married up with the weightings in our model portfolios, so whilst we could not claim 100% validity, it was the nearest ‘best fit’.
Between 2012 and 2019, APCIMs went through a number of changes which meant the benchmark became detached from CDC portfolio weightings and therefore validity became compromised. Herein lies the problem with benchmarks. If the asset allocation committee responsible for compiling a benchmark wishes to say, reduce equity and add to bonds, they simply do it – it is solely a ‘book-keeping’ exercise. Investment managers do not enjoy that same level of flexibility as they might have capital gains tax to consider for example. Moreover changing the composition of assets in our portfolios on the sole premise of remaining close to a benchmark strikes us as intellectually bankrupt and highly reactive.
In 2019 we changed our chosen benchmarks to ARC (which is short for Asset Risk Consultants), who gather data from around 50 discretionary managers grouped into 4 risk rated portfolios. The good thing about using ARC is that the performance over the long run should generally more closely reflect how the whole industry is doing. Sadly, as mentioned earlier, the world of benchmarking is far from perfect and during times of crisis, such as we have witnessed, the dispersion of views across the industry participants increases. In other words, a Balanced portfolio can be interpreted very differently across ARC members and this can lead to wide variance from the benchmark in the short term.
The bottom line is that whilst most benchmarks are reliable, few carry high degrees of validity and as such should be used as a guideline only, rather than a true determinant of performance.
Dr Andrew Mann
Investment Director