If we're going to try to nail this down, we need all the information. I mean everything.
Complete Asterisk version, CPU Architecture, NAT or Not NAT, Phones that are playing up with explicit firmware versions... Everything.
(And I'm serious about CPU Architecture, is anyone not using 64 bit?)