That is true, but it really depends how large the place was. For example, I am mostly a developer, but I volunteer some system administration for a local charity that I appreciate. They used to have an administrator, but they left and they don't have that many systems. When I arrived to look over the systems and start my work, I found the following:
1. A server that contains a domain controller and shared network folders.
2. A UPS for aforementioned server. Not plugged in to the wall socket or, thankfully, the server.
3. A backup system that seemed to be set up properly. It used removable disks that were swapped out every week, when there was an administrator. Since that admin left, they had one disk inside the system that contained the most recent backup and two disks that contained backups from eight months previous.
4. A firewall that nobody had the access codes to. Nobody knew what this firewall was or wasn't doing and I just wanted to get rid of it once I felt confident to rebuild the network.
This is what happens when there is only one person working on the system and the company lacks the ability to manage that person. The charity is small, the director is nontechnical, and the system was consequently chaotic. There wasn't a clear person at fault, but we could all agree that there was a problem.