‘Replication crisis’: Why academic research struggles in the real world
I read a lot – and widely. In part out of sheer curiosity, sometimes to delve into a new subject and then again to find interesting information I might use in articles and social media. Today I present one of the latter, an article written by Robin Wigglesworth for the Markets Insight column in the Financial Times newspaper late last year. I cut it out and kept it, choosing to use it this month, times of financial stress and media hype.
Over the last decade scientists have found that the results of research papers have been increasingly difficult to reproduce – the replication crisis – and thereby validate. Wigglesworth questions whether this is not just the case in medicine and psychology but also in financial markets. I’m sure you too have come across endless articles in so-called leading financial publications that reference papers by ‘Fellow & Friend’ on the efficient market hypothesis and price/earnings – or whatever.
Here I quote from the FT article: ‘This is the incendiary argument of Campbell Harvey, professor of finance at Duke University. He reckons that at least half of the 400 supposedly market-beating strategies identified in top financial journals over the years are bogus. Worse, he worries that many fellow academics are in denial about this.’ This from the man whose 1986 PhD thesis was about how yield curves predict recessions.
He believes that one of the culprits might be the academic system itself which incentivises young staff to publish something sensational in a prestigious journal in the hope of landing tenure. A more nuanced reason is what’s known a ‘p-hacking’ where ‘p’ is the probability of something happening by pure chance as opposed to being statistically significant. ‘P-hacking’ is when researchers overtly or subconsciously twist the data to find a superficially compelling but ultimately spurious relationship between variables’.
So: next time a financial salesman comes up to you with a proposal based on academic research that has been back-tested – Caveat Emptor!
Tags: correlation, probability, Research papers
The views and opinions expressed on the STA’s blog do not necessarily represent those of the Society of Technical Analysts (the “STA”), or of any officer, director or member of the STA. The STA makes no representations as to the accuracy, completeness, or reliability of any information on the blog or found by following any link on blog, and none of the STA, STA Administrative Services or any current or past executive board members are liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. None of the information on the STA’s blog constitutes investment advice.
- Quick video interview with Elizabeth Miller: A panellist at the STA’s April conference. February 27, 2024
- D’mitri Richardson breezes in from Chicago: Propelled by the cycles of W D Gann February 14, 2024
- STA Podcast with Stephen Hoad: Ahead of the Technicals to Trading Systems Conference February 5, 2024
- Podcast with Alan Dunne MSTA: Ahead of the STA’s Technicals to Trading Systems Conference January 31, 2024
- How NOT to write about the markets: As a technical analyst, economist or media pundit January 22, 2024