What would cause very high pdd 12 seconds + after 3000 ports.  Wireshark shows reinvites.

I have tried many profiles and even many console apps both seem to have the same results after 2500 to 3000 ports (only 300 are connected)

The server cpu and memory are ½ the server limits.

Any ideas where to look would be great.