I had a problem like this when I built a small dev server, and was addressing it using only its IP address. Got some very long response times. When I created a name for the server, entered the name (and IP) in my etc/hosts file, the server response time was much faster.