It has it's uses
I've had pen testers my and fail one of the sites I look after, just because it had a robots.txt. They didn't look at the contents of it, just that it was there. Robots.txt is a useful tool for controlling the behaviour of legitimate crawlers, it also makes it easy to identify those that ignore it and take remedial action.
Needless to say, it added an extra sense of perspective to some of their other suggestions, most of which I considered bullshit too.