The trick is whether the cases of disagreement were themselves predictable!
Given a fixed number of oncologists and deploying Watson only to support those oncologists, yes, you're correct that its only useful if it outperforms them. But I think of it more like Watson is a single hive-mind team of like a thousand med students near the end of residency: they get most things right but there are a few places where more experienced doctors will be better, though the scale with the hive mind is far higher.
You have one of two reactions to that. 1) Hire fewer senior oncologists and have them focus on the more difficult cases and leave the hive-mind to deal with thousands of routine cases, or 2) ignore the hive mind until its literally better than a typical senior oncologist.
The medial profession seems to repeat this cycle of "only full doctors can do anything because even seemingly routine cases might be hiding something more serious" to "maybe some routine things can be done by people with less training and full doctors should focus on the more difficult cases". See nurse practitioners, dental assistants, and, in my mind at least where we are going with things like Watson.
I think part of the problem for Watson is that it needs someone to gather the data, which is usually a doctor. So if you're pairing each patient with an oncologist for intake anyway, it's not clear that "examination plus enter all data into Watson" is a benefit over "examination plus make a decision".
I guess the ideal outcome for Watson (if it doesn't outpace expert oncologists) would be something like "experience nurse practitioner does an exam, and enters data into Watson" or maybe even "special oncology-trained NP does an exam with Watson".
The other part I don't know is what oncology accuracy rates look like. If the reason to not majorly expand screening is cost and availability, Watson could be huge. If it's false positives from our existing rate, there's a lot less value.
Wider screening isn't great. Essentially, people with medical problems self-select pretty alright already, and wide early-screening initiatives for most cancers introduces as much or more false positives (that persist through screening!) than actual cases of cancer it catches - and moreover, it's not even clear that the early screening is effective - the false negative rate is high enough that the overall incidence of advanced cancer is unchanged even with early screening. (Bleyer and Welch, NEJM, 2012)
Basically, we need better screening, not more screening at our current levels, and it's not clear whether watson can provide that.
Precisely - since mammography is the standard "counterintuitive Bayes rule" primer, I remembered those numbers don't support wider screening.
I'm not discounting the possibility of a useful role for Watson in wider screening, but it's not clear to me where it would be. If it happens after any kind of extensive examination, doctor-hours are being committed regardless and there's little gain. If it happens at a population-level screen like mammograms and colonoscopies, "almost as good as oncologists" isn't enough to add any value.
Given a fixed number of oncologists and deploying Watson only to support those oncologists, yes, you're correct that its only useful if it outperforms them. But I think of it more like Watson is a single hive-mind team of like a thousand med students near the end of residency: they get most things right but there are a few places where more experienced doctors will be better, though the scale with the hive mind is far higher.
You have one of two reactions to that. 1) Hire fewer senior oncologists and have them focus on the more difficult cases and leave the hive-mind to deal with thousands of routine cases, or 2) ignore the hive mind until its literally better than a typical senior oncologist.
The medial profession seems to repeat this cycle of "only full doctors can do anything because even seemingly routine cases might be hiding something more serious" to "maybe some routine things can be done by people with less training and full doctors should focus on the more difficult cases". See nurse practitioners, dental assistants, and, in my mind at least where we are going with things like Watson.