Re: I'm actually surprised that it works on raw samples at all
I actually think that moving to raw samples is a rather bad thing to do. After all that greatly increases the complexity of the problem by adding lots of irrelevant information.
I mean no animal on earth hears by samples, they all hear by intensity over frequency and perhaps phase differences between different bandpass filters. Musicians also don't output samples, but manipulations of instruments.