I know it sounds tired, but I think the biggest innovation was the simple, intuitive UI.
People that hated Apple/iPods always pointed out that Archos and other mp3 players could do everything the iPod could, which is absolutely true. Apple instead introduced a small, highly polished (literally and figuratively) player that was beautiful and easy to use–something no competitor could claim at the time. It wasn't the list of features that was new, it was how easy it was to use all of them.
This was the first electronic gadget that I was able to use nearly 100% of the features without referencing the manual. The manual it came with was scant, at best, but that was just the thing–you didn't need it. The only thing I couldn't figure out on my own was how to get the backlight on when it was dark (hold the play button down for 5 seconds or so, then the backlight stays on long enough to enable it in the settings).
Mine still works, though the battery life is not so great and it mostly sits in a drawer. I loved the mechanical wheel, though it was prone to accidental swipes that resulted in high volume music.