‘Replication crisis’: Why academic research struggles in the real world
I read a lot – and widely. In part out of sheer curiosity, sometimes to delve into a new subject and then again to find interesting information I might use in articles and social media. Today I present one of the latter, an article written by Robin Wigglesworth for the Markets Insight column in the Financial Times newspaper late last year. I cut it out and kept it, choosing to use it this month, times of financial stress and media hype.
Over the last decade scientists have found that the results of research papers have been increasingly difficult to reproduce – the replication crisis – and thereby validate. Wigglesworth questions whether this is not just the case in medicine and psychology but also in financial markets. I’m sure you too have come across endless articles in so-called leading financial publications that reference papers by ‘Fellow & Friend’ on the efficient market hypothesis and price/earnings – or whatever.
Here I quote from the FT article: ‘This is the incendiary argument of Campbell Harvey, professor of finance at Duke University. He reckons that at least half of the 400 supposedly market-beating strategies identified in top financial journals over the years are bogus. Worse, he worries that many fellow academics are in denial about this.’ This from the man whose 1986 PhD thesis was about how yield curves predict recessions.
He believes that one of the culprits might be the academic system itself which incentivises young staff to publish something sensational in a prestigious journal in the hope of landing tenure. A more nuanced reason is what’s known a ‘p-hacking’ where ‘p’ is the probability of something happening by pure chance as opposed to being statistically significant. ‘P-hacking’ is when researchers overtly or subconsciously twist the data to find a superficially compelling but ultimately spurious relationship between variables’.
So: next time a financial salesman comes up to you with a proposal based on academic research that has been back-tested – Caveat Emptor!
Tags: correlation, probability, Research papers
The views and opinions expressed on the STA’s blog do not necessarily represent those of the Society of Technical Analysts (the “STA”), or of any officer, director or member of the STA. The STA makes no representations as to the accuracy, completeness, or reliability of any information on the blog or found by following any link on blog, and none of the STA, STA Administrative Services or any current or past executive board members are liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. None of the information on the STA’s blog constitutes investment advice.
- IFTA South East Asia Webinar Collaboration: ‘Second Semester from Cycle Perspective’ June 23, 2022
- June is party time in London: The STA gets in on the act June 16, 2022
- Candles at night time: But wait and see what they look like first thing in the morning June 1, 2022
- ‘Replication crisis’: Why academic research struggles in the real world May 18, 2022
- ‘Talking about Risk’ – ‘’one of those topics nobody wants to talk about.’’ A webinar by Perry J. Kaufman for STA members May 11, 2022
- A fireside chat with author Jack D Schwager: He of Market Wizards fame on
- Market Wizards – and their lesser known cohorts: Jack Schwager hits the book launch circuit on
- Colours, clashes and clichés: How and why use colourful charts on
- Do you know Aroon? I do, but not very well on
- Must read classics: Richard W Schabacker – the real bible of technical analysis on