Well given the fact that most games don't need more than 4 cores aside, they tend to bolt on the following background tasks...
* Stream encoding to upload to Twitch, etc.
* Watching streams, youtube
* Downloading torrents, etc
Just because the game only uses a subset of cores doesn't mean the rest of the system isn't churning away on other processes.