Alex Pollock at the American Enterprise Institute writes about the role of models in economic science (or what "would be" a science "if it weren’t for the people") and financial decision-making. He argues that the widespread use of models tends to lead to their obsolecence:
Perversely, the more everyone believes the model, and the more everyone uses the same model, the more likely it is to induce changes in the market that make it cease to work.
In this cycle, the market and the regulators became enamored of the statistical treatments of risk, whereas the most important issue is always the human sources of risk. These human sources include short memories and the inclination to convince ourselves that we are experiencing "innovation" and "creativity," when all that is happening is a lowering of credit standards by new names.
As I understand his argument, there are a couple reasons for this. Some models– ones that deal with very specific pieces of the future– only work if they're obscure: if everyone "knows" that the price of magnesium is definitely going to rise, and everyone buys magnesium futures, the future price of magnesium changes. Models reinforce the belief that "this time it's different," and help people unlearn old, hard-won lessons. (As Pollock puts it elsewhere, one of the differences between science and finance is that scientists don't forget previous errors– astronomers haven't gone back to geocentrism, and old ideas tend to die with old scientists– while generational change in finance tends to wipe away wisdom, leaving only hubris and a belief in one's own youthful invincibility.) Models also tend to obscure the continued, lurking presence of uncertainty:
Because uncertainty is fundamental, sometimes disastrous mistakes will continue to be made by entrepreneurs, bankers, borrowers, central bankers, government agencies, politicians, and by the interaction of all of the above.
[Economics Frank] Knight wrote: "If the law of change is known, no [economic] profits can arise." Likewise: "If the law of change is known, no financial crises can arise." But in economics and finance, the law of change is never known. So change reflecting uncertainty goes on, bringing booms and busts periodically, and Adam Smith’s "progress of opulence" on the trend.
Have economists have tried to measure the impact of the popularity of models on markets? The Knight quote comes from his 1921 book Risk, Uncertainty, and Profit, and I have to assume that economists have tried to measure (a model, if you will) how widespread use of, say, a statistical model affects markets and either increases or decreases the reliability of that model. It seems to me that this would be one of those things that people would have tried to study, but I don't know enough about the field to know.
The full quote from Knight's book is:
It cannot, then, be change, which is the cause of profit, since if the law of change is known, as in fact is largely the case, no profit can arise. The connection between change and profit is uncertain and always indirect. Change may cause a situation out of which profit will be made, if it brings about ignorance of the future. Without change of some sort there would, it is true, be no profits, for if everything moved along in a an absolutely uniform way, the future would be completely foreknown in the present and competition would certainly adjust things to the ideal state where all prices would equal costs. It is this fact that change is a necessary condition of our being ignorant of the future (though ignorance need not follow from the fact of change and only to a limited extent does so) that has given rise to the error that change is the cause of profit…. It is not dynamic change, nor any change, as such, which causes profit, but the divergence of actual conditions from those which have been expected on the basis of which business arrangements have been made. (38-39)