Reply to post: Re: By the time it's ingrained and encoded into a deep learning net...

Re: By the time it's ingrained and encoded into a deep learning net...

Black box algorithms do something with their data. Some of it, they keep. The question is whether it's possible to retrieve any of it. If, for example, patient name was used in the training set and given to the program, there is a good chance it has done something with that data. Maybe it identifies that people named Bob are more likely to have certain health conditions than other names. If you provided it with more information about Bob, it might be able to predict more information. Of course, a good AI developer wouldn't include patient name, as that's an invitation to pollute the data and has historically proven problematic*. But people do do it sometimes and it could therefore be a privacy risk.

*For example, an algorithm trained on medical data to determine the likelihood of a patient having cancer was given the name of the hospital where the patient was receiving treatment. The algorithm was able to determine that patients staying at hospitals with "cancer center" in them were more likely to have or develop cancer. This made the algorithm next to useless, but it also increased the accuracy rate and if we know one thing about AI companies, they like good accuracy rates.