Time...
Is a PITA!
I was seriously considering an atomic clock setup for one of our servers as, for some reason we couldn't fathom, it was consistently 3 minutes slow despite regular NTP calls and irrespective of the time server used. Had I been sure it would have resolved the issue I'd probably have gone ahead but in the end our support company managed to fix it with an obscure patch.
Did 3 minutes really matter? Whilst not as critical as the time for financial trading yes, it did! The factory time clock updated from the server weekly (and when the clocks changed), that three minutes was the difference between some of the operators catching their bus home or waiting an hour for the next one.
Not work but we had an issue with linked electronics (it was a proprietary ring bus system) on a yacht where the time on the chart plotter (which didn't have it's own RTC) had to be manually reset at power up when it was supposed to take the time from the GPS. Having finally managed to track down a USB interface for the network we went into packet sniffing mode and it turned out the scenario was:
Chart Plotter sends Time Request Good so far
GPS responds to Request Still good
Chart Plotter responds to Request Eh? Why is it responding to its own time request?
Chart Plotter receives time from GPS and updates Correct time
Chart Plotter receives time from itself and updates Overwriting the correct time with the default start up time
We were able to update the chart plotter config to stop it responding to time requests but why the heck it was ever enabled to do so was beyond us.