When IIS responses slow it may not be your IIS configuration issue. For most of the time it could be network accessing issue. Ping command cannot reflect the problem for example, 14ms. User tracert command if your Ping command shows you no trouble. The picture below is showing the tracert to the destination server without problem.
Another picture below is showing the tracert to destination server with huge among of delay. With this situation the IIS will response very slow.