Thursday, October 13, 2005

Clock drift ate my email!

I just spent a bizarre two hours trying to find out why mail would be delivered into inboxes very quickly, but not appear on POP3 for about 10 minutes. The answer really surprised me: clock drift. It turns out that Qmail's POP3 client won't acknowledge the existence of mail with a filesystem timestamp in the future (as soon as the clock reaches the file timestamp, the message is downloadable). We have one server storing all mail, and several POP3 daemons pointing at it over NFS. Server clocks drifted, with the NFS server in the future by about 10 minutes. The result? Mail arrives quickly, but isn't downloadable until the other servers catch up to the delivery time.


Wierd.


Mood: accomplished
Music: None

No comments: