A few weeks ago I came across this article in Slate about how physicians don't do a good job of estimating how long terminally ill people have to live:
Doctors prefer not to prognosticate for three reasons: We don't like to be wrong; we don't want to take away hope for survival or good quality of life in the time that remains; and we just aren't adequately trained to do it. And our reluctance to make such guesses means that when we do try to predict the future, we're pretty lousy at it….
Since doctors typically avoid making predictions, these tools are infrequently dusted off and put to use. Our collective reluctance to offer patients a prognosis makes us less accurate in the rare instances we actually do it.
In his seminal book Death Foretold: Prophecy and Prognosis in Medical Care, Nicholas Christakis, a medical doctor and sociologist, argues that medical science has given the processes of diagnosis and treatment disproportionate emphasis in the educational curricula of doctors…. [A]voiding prognosis is a professional norm for doctors at all levels of training. In our research, teaching, and communication, we focus almost exclusively on the ever-expanding sciences of diagnosis and treatment, leaving prognosis almost entirely to the side.
Making predictions about the lives of… cancer patients can be particularly tricky. William Dahut, clinical director of the Center for Cancer Research at the National Cancer Institute, blames "a general lack of understanding of the specific biology of the cancer as well as a general lack of understanding of the biology of the individual." Doctors and scientists often refer to an individual's biology as "host factors," making allowances for the fact that patients are indeed different—in immunity, resilience, and attitude. The difficulty in accounting for such differences is another reason that predictive accuracy is so low….
Christakis argues that studying and delivering prognoses to patients is part of the ethical obligation of doctors to their patients. "Furthermore," he writes, "physicians should legitimate discussions regarding prognosis not only with their patients but with each other." As such, doctors would recast the professional norm to include open and frank discussion of prognosis in medical care.
In so doing, we need to strive for honesty and avoid "hanging crepe," the idea of delivering a poor prognosis simply to combat our tendency to be overly optimistic and to keep our hands clean: If the patient dies, I predicted it and therefore appear accurate; if the patient outlives my prediction, everyone is pleasantly surprised and thus I'm not held accountable.
We know thanks to Philip Tetlock how expert political judgment works. It would be interesting to look at a variety of different professions or disciplines that are under pressure to make predictions or forecasts, and see if there are interesting differences in the ways physicians, meteorologists, financial researchers, intelligence analysts, and others handle those demands.