Sign in / up

back to article US watchdog OKs robo-doc AI that spies eye disease all on its own

The first FDA-approved AI system for diagnosing an eye disease caused by diabetes is completely autonomous, and doesn’t require a doctor to interpret the results. Several corporations including Google and DeepMind have been working on building machine-learning algorithms for detecting diabetic retinopathy, a leading cause of …

COMMENTS

Post your comment
House rules Send corrections
Add to 'My topics' unstar *
  1. ratfox

    The device has up to to 87 percent sensitivity – patients who did have the mild diabetic retinopathy were correctly identified; 90 per cent specificity – patients who did not have the disease and were correctly identified as having no eye damage

    Meaning that 13% of patients with the disease are not detected. That's not great, and I'm surprised they are proposing to remove the doctors entirely. Could it be that the doctors are even worse?

    I think that typically, these detection systems err on the safe side – reduce false negatives as much as possible, even if that raises the false positives – and then all those detected as positive go through a more precise and more expensive screening with a human doctor. Maybe here the 10% false positives are already so numerous that they don't want to be more aggressive.

    1 0 Reply
    1. Alan Hope
      Reply Icon

      "Meaning that 13% of patients with the disease are not detected. That's not great, and I'm surprised they are proposing to remove the doctors entirely. Could it be that the doctors are even worse?"

      The reason we know there is a false-negative and false-positive rate is that the diagnostic criteria have been defined by doctors and the false-negatives and positives are then identified by professionals. Qualified medical consultants remain the gold standard and will be so for a very long time.

      What is presented here, is an AI-based pattern-recognition screening system that may improve the pickup rate of, or perhaps be used to help initial training of non-medical "graders".

      2 0 Reply
    2. Cuddles
      Reply Icon

      "Meaning that 13% of patients with the disease are not detected. That's not great, and I'm surprised they are proposing to remove the doctors entirely. Could it be that the doctors are even worse?"

      Of course. Comments like this always seem to appear whenever medical things are discussed, and it seems most people just don't understand how messy and difficult biology is. In this case, a quick look at the first few papers I found gives sensitivity anywhere between 70-90% for specialists, and as bad as 40% for regular GPs, so it looks like the machine is really pretty good even if it may not be perfect.

      It's also worth bearing in mind that this is routine screening - diabetics are recommended to be tested every year or two (depending on the country). Since the problem is gradual degeneration rather than a sudden acute condition, even if it's missed early on it's still likely to be caught before things get too serious.

      2 0 Reply
  2. Alan J. Wylie

    What about all the other diseases?

    Macular degeneration, for one. And what happens if someone dies because of a missed diagnosis. Who do you prosecute then?

    0 2 Reply
    1. Cuddles
      Reply Icon

      Re: What about all the other diseases?

      "Macular degeneration, for one."

      What about them? This is a diagnostic device for screening for a specific condition. Complaining that it doesn't detect every possible illness in the world is just ridiculous. That makes as much sense as complaining that a blood test for ebola can't detect a sprained ankle. If you want a magic machine that can diagnose absolutely everything, you're in the wrong reality.

      "And what happens if someone dies because of a missed diagnosis. Who do you prosecute then?"

      No-one. Diagnoses are never perfect and should not be expected to be. Unless you have a good reason to suspect actual negligence or malpractice, you have absolutely no business thinking about prosecuting anyone. The insistence some people have on assigning blame every time doctors fail to be perfect in every way is probably the biggest problem faced by the medical profession.

      5 0 Reply
      1. Alan J. Wylie
        Reply Icon

        Re: What about all the other diseases?

        That makes as much sense as complaining that a blood test for ebola can't detect a sprained ankle.

        I was referring only to diseases diagnosed by inspecting the retina. What's the point of an automated system when a specialist has to look at it anyway to diagnose other diseases?

        As for the prosecution, sorry - I missed a sarcasm tag. As with the case of Dr Hadiza Bawa-Garba, the case should never have been brought in the first place.

        1 2 Reply
        1. Cuddles
          Reply Icon

          Re: What about all the other diseases?

          "What's the point of an automated system when a specialist has to look at it anyway to diagnose other diseases?"

          A specialist has to look at what? Why would they be trying to diagnose anything else? The current situation is that diabetics get regular screening for this one specific condition they are at risk of. The proposal is that they will continue to get exactly the same screening, but with a computer instead of a human looking at the images involved. Nothing else will change at all. They don't currently call a specialist in to test for hundreds of other random diseases just for the hell of it, and they will continue not doing so in the future regardless of whether a computer is involved or not.

          0 0 Reply
    2. The Mole
      Reply Icon

      Re: What about all the other diseases?

      I imagine that Macular degeneration may be their next project, which is likely to be quicker and easier now they have proved the technology and get it licensed. And (assuming camera settings are compatible) I'm sure they will be able to roll it out as an upgrade to existing users to also detect other conditions at the same time (the incremental processing cost is likely to be insignificant you imagine). But each new condition will require additional FDA approval which is expensive and time consuming.

      1 0 Reply
  3. Skoorb

    This isn't going to be used in hospitals or reduce doctors workload

    Eh. This has been being researched by various companies for years.

    The use case is not in hospital clinics.

    In the UK, and other countries, there is a national screening programme for people with diabetes called the Diabetic Eye Screening Programme.

    This screens everyone with diabetes 12 years and older every year, by taking digital photographs of the retina (and macula). Currently, these are then stuck in a queue to be graded by a human (within a target of 6 weeks), with 10% of the "normal" ones then being regraded by another human and up to 100% of the "abnormal" ones being regraded by a second human.

    The people grading just answer a series of yes/no questions like "is there retinal thickening within 1DD of the centre of the fovea" or "are there new vessels on disc (NVD)". The system then generates a numerical "grade" of the severity of the condition. If the "grade" is high enough, the images are sent to someone qualified (like an eye doctor) to "gatekeep" a referral to Opthalmology, and the priority of any referral.

    Once referred into hospital, the doctor will be looking at the retina and making a decision, not a computer.

    All this system will be used for is to get rid of / act as a check on the current human graders. In most cases, these graders aren't qualified ophthalmologists, but just people hired and sent through a training programme. Anything identified as "abnormal" will still be sent to someone qualified to decide on if referral into hospital is necessary just as now.

    5 0 Reply
  4. Anonymous Coward
    Anonymous Coward

    Does it distinguish between retinopathy and venal branch occlusions? Both produce retinal haemorrhages - but clinicians apparently distinguish between the two problems. You can have the occlusion without having diabetes retinopathy.

    A retinal venal branch occlusion can be picked up by an annual retinopathy scan. In my UK health authority you get transferred to the eye clinic for further diagnosis/treatment. Only when it is judged stable are you transferred back into the annual retinopathy scan cycle.

    1 0 Reply
  5. Flywheel

    Trust

    While this an amazing feat of several different types of engineering, the excitement could be somewhat tainted by those involved: several corporations including Google and DeepMind. Maybe my tinfoil hat is getting extra layers lately, but I'd want to know exactly who will be using my diagnosis data, and for what purpose.

    4 1 Reply
    1. The Mole
      Reply Icon

      Re: Trust

      Perhaps you should lift the tin foil hat a bit and read the entire paragraph. "The first biz to release an AI-based device that can detect the disease – and was approved by the US Food and Drug Administration (FDA) in April – is less well-known, however." That is, those two big companies aren't involved in this invention, that just also happen to be doing research in the area.

      2 0 Reply
    2. Ken Moorhouse Silver badge
      Reply Icon

      Re: Trust

      I imagine the "who" you are referring to is an entity such as an insurance company. If computer erroneously says "yes" (an insurance company would want a weighted summary of parameters, giving a yes/no score), there will come a time when that "false positive" becomes a blot on your insurance record, regardless of what subsequent human experts diagnose, on the flawed basis that computers don't make mistakes. We then come back to the discussion about what algorithms are embedded in the AI, except in the case of eye health, there probably would be no ability to audit the system by tweaking parameters slightly to highlight a discontinuity in the analysis.

      So it may impact on that someone if an insurance claim is made.

      0 0 Reply

POST COMMENT House rules

Not a member of The Register? Create a new account here.

Forgotten password ?

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like