It is double dipping. I often have to use subtitles on films/tv because of the audio, or just plain old language ambiguity or sound/word similarities.
I don't listen to many audio books, but when do, I often follow text.
So this would both force me to pay twice, and suggest I'm "illegal", for having an additional need *after paying for the content already*.
My solution? I won't actually consume your content if you treat customers that way. I have no qualms over not watching/reading/listening to your media, and thus not paying.