That is precisely the beauty of something like an Apple Watch. Everybody can be screened without wasting resources and the device can alert if it is really something that needs a specialist. Of course, we are on the early stages of something like that and the devices are not really capable of diagnosing on their own, thus you end up with a spike in consultations. But given some time it should be way more accurate and helpful.

It's entirely possible that a test which was net positive when applied "with probable cause", can even become a net negative if applied without. Because testing the whole population will produce many more false positives.

I doubt the tests they are running here will actually go that far. But pervasive testing does need different trade-offs to targeted testing. At least we may have more complete data from which to understand this.

> Because testing the whole population will produce many more false positives.

That is the issue that should get resolved with time. The watch will receive a lot of feedback from medical professionals to adjust its reporting because now we'll know so much more about what symptoms are really worth paying attention. But of course, there will be an adjustment period like what we are seeing now. This has always been true of new technologies introduced in medicine.

> That is the issue that should get resolved with time.

Bear in mind that this hasn't happened for the other major overdiagnosis issue (cancer screening), and the best solution found so far there is to screen less. This really isn't the sort of thing where natural technological progress is enough to be comfortable saying things like "it'll get resolved with time". There are already counterexamples to what you're saying, it's not a matter of an "adjustment period".

If I understand right, one of the problems in the case of cancer is that we don't have great data about the people without symptoms who wider screening would catch. Once we catch them, we treat them, and many recover. But to say how many of those would have been fine anyway, we'd have to know more about the un-screened rest of the population.

The hope with widespread cheap screening would be that it might avoid this. If heart-rate smartwatches become as common as smartphones, we could train our treatment model on everyone, and do better. Or at least this sounds mathematically plausible to me (not an expert).

Whether it will come to pass, or whether doctors will continue to treat as now (anything which an eyeball test says looks like an early stage of a known disease) I don't know.

Not when everyone is wearing one of these watches. That's the point behind one of the parent comments. The traditional way of looking at medical screening is that you only go looking for something if there is a rationale.

So, for a heart monitor, you'd only have the test run if you had another symptom, such as shortness of breath, an irregular heart beat, etc... Otherwise, you will end up with many more false positives, which can be dangerous in and of itself. What if someone who legitimately has a heart condition had an Apple watch and ran the test. But what if that person then ignored the watch because all of their friends tried the same thing, and went to the doctor only to find out it wasn't anything to worry about. Their false positives could have a real impact on someone who had a true positive result.

The more you test, the more you'll find. Now, it's perfectly possible that many of the subjects in this study bought the watch specifically because they suspected that they had a heart condition. In which case, the study pool could be skewed away from the general population.

There is no way to adjust for this. The more people you screen, the more you'll find. Whether or not it's a true positive or false positive is another question. And I'd argue that Apple would need to skew their reporting towards removing false negatives instead of removing false positives, just from a liability point of view. You're much more likely to see Apple sued over this than a typical medical device manufacturer.

When the ratio of true to false positives is at greater than 5:1, it's hard to imagine the false positives causing net harm.

Where do you get 5:1? Maybe I don't know enough about heart disease but the numbers in the article seem pretty hard to parse.

It sounds like many of those who got a bleep from their iWatch subsequently had some effect detected by standard tests. But that isn't the right number. We don't necessarily know that we should treat everyone who fails standard tests. Especially if our previous knowledge about such tests was limited to patients where there was other evidence.

> Where do you get 5:1?

At least 84% of the positives were correct.

> It sounds like many of those who got a bleep from their iWatch subsequently had some effect detected by standard tests. But that isn't the right number. We don't necessarily know that we should treat everyone who fails standard tests. Especially if our previous knowledge about such tests was limited to patients where there was other evidence.

I agree, but there is a very important distinction to make here. When we don't know what to do with the results of a test, it can be wasteful and negative even when the results of the test are completely accurate.

That is the much bigger risk here. That the true positives are a waste of time and net negative all by themselves. They have the real potential for issues here, not the false positives.

Because the true positives outweigh the false positives by so much, for there to be net harm because of false positives, each one would have to do more damage than the effect of five true positives. I doubt that's the case. I'm pretty confident that either the test is an overall good thing, or the test is a bad thing because of true positives.

OK, I think we're fully in agreement here, modulo terminology. You are using "true positive" to mean that the watch correctly predicted a positive score on the medical test. And these are the people who may be treated, some of them unnecessarily.

Not just that it matches the medical test, I'm saying that the watch is objectively correct in detecting an irregular heartbeat. The problem is that we don't know what to do with that data, which is different from the test making a mistake.

There are other situations where it actually is a huge problem that the test makes a mistake some percent of the time. It's important to distinguish the two problems.

Given the choice between

A) Everyone is told nothing.

B) 5% of people are told “Here’s something. It’s serious.” And, 45% of people are told “Here’s something. But, it’s not serious.”

Option B sounds greatly preferable.

Depends... does "Here’s something. But, it’s not serious" take expensive, risky, and invasive medical tests to get to the "not serious" part?

EULA and UI should emphasize that this is not a sure thing, please see a doctor to be sure. Why would anyone get mad at that? My grandpa noticed my dad losing a ton of weight. He told my dad there might be something wrong with him, go check it out. Turns out it was cancer, saved my dad's life. Should my dad be upset at my grandpa for causing emotional stress if it was nothing?

Getting tests when you have symptoms is a completely and totally different topic than widescale testing of asymptomatic individuals without specific risk factors, which is what we are talking about here.

For now, I'd prefer option C) "Here's something. Go ask your doctor if it's serious."