‘Replication crisis’: Why academic research struggles in the real world
I read a lot – and widely. In part out of sheer curiosity, sometimes to delve into a new subject and then again to find interesting information I might use in articles and social media. Today I present one of the latter, an article written by Robin Wigglesworth for the Markets Insight column in the Financial Times newspaper late last year. I cut it out and kept it, choosing to use it this month, times of financial stress and media hype.
Over the last decade scientists have found that the results of research papers have been increasingly difficult to reproduce – the replication crisis – and thereby validate. Wigglesworth questions whether this is not just the case in medicine and psychology but also in financial markets. I’m sure you too have come across endless articles in so-called leading financial publications that reference papers by ‘Fellow & Friend’ on the efficient market hypothesis and price/earnings – or whatever.
Here I quote from the FT article: ‘This is the incendiary argument of Campbell Harvey, professor of finance at Duke University. He reckons that at least half of the 400 supposedly market-beating strategies identified in top financial journals over the years are bogus. Worse, he worries that many fellow academics are in denial about this.’ This from the man whose 1986 PhD thesis was about how yield curves predict recessions.
He believes that one of the culprits might be the academic system itself which incentivises young staff to publish something sensational in a prestigious journal in the hope of landing tenure. A more nuanced reason is what’s known a ‘p-hacking’ where ‘p’ is the probability of something happening by pure chance as opposed to being statistically significant. ‘P-hacking’ is when researchers overtly or subconsciously twist the data to find a superficially compelling but ultimately spurious relationship between variables’.
So: next time a financial salesman comes up to you with a proposal based on academic research that has been back-tested – Caveat Emptor!
Tags: correlation, probability, Research papers
The views and opinions expressed on the STA’s blog do not necessarily represent those of the Society of Technical Analysts (the “STA”), or of any officer, director or member of the STA. The STA makes no representations as to the accuracy, completeness, or reliability of any information on the blog or found by following any link on blog, and none of the STA, STA Administrative Services or any current or past executive board members are liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. None of the information on the STA’s blog constitutes investment advice.
- Looking forwards and backwards: At this time of year December 2, 2022
- Long term weather forecast for power trading: Webinar from Refinitiv 17 November 2022 November 23, 2022
- ‘Are you delusional?’: The title of Victoria Shepherd’s recent magazine article November 14, 2022
- Larry the lad looking good: Despite trading since 1962 November 11, 2022
- The Day of the (Living) Dead: Ghosts and ghouls at this time of year November 1, 2022
- A fireside chat with author Jack D Schwager: He of Market Wizards fame on
- Market Wizards – and their lesser known cohorts: Jack Schwager hits the book launch circuit on
- Colours, clashes and clichés: How and why use colourful charts on
- Do you know Aroon? I do, but not very well on
- Must read classics: Richard W Schabacker – the real bible of technical analysis on