Heat
Switching signals involves converting electrical energy into heat. Compact switch layouts have less area to dissipate the same amounts of heat. 3D structures have even more problems getting the heat away from the switches. Two ways around this are to use less energy per switch transition (which raises other problems in itself such as thermal and electrical noise swamping the signal) and the other is to use materials and switch technologies that don't degrade when hot (which causes knockon problems with packaging and assembly technologies -- a ceramic-encapsulated chip that can run OK at 400 deg C will desolder itself from a conventionally-built planar).
