Not a real issue
Firstly you have to have a retraining cycle for ML models anyway - if not for feature/software release reasons then just for "freshness"
Secondly a good model should need much if anything in the way of PII, most use cases are about acting on novel views of small-medium groups of people. Most use cases are not UK Rozzer "find the crims" types.
Thirdly - for Black Box models you would have to prove that the PII has been retained in a format that still makes it PII - with is both unlikely and virtually impossible.
A good one to troll the UK Rozzers or Govt with though...