"Why use exact powers of 10? Why not use binary numbers? The power of 2 used will still be arbitrary."
The Babylonians used 12 and 60 (can you guess where we still use those?)
The Romans used 12 a lot. 10 was mostly only used for military work.
In both cases it's because the number of divisors make them easier to work with than 10s.