Hi everyone,
I've got a strange problem on my servers which has happened 3 times in the last month now...
I have FS listening on 2 IP addresses, bound to bond0 and bond0:0. FS suddently stops processing any calls arriving on one of those IPs.
tcpdump/tshark still shows INVITE packets arriving on my server, but FS doesn't process them and there are no 100 Trying etc replies.
Initially FreeSWITCH still owns the socket, the only odd thing is Send-Q is 744:
ss1:~# netstat -anp | head -n2
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
ss1:~# netstat -anp | grep 5060
tcp 0 0 81.27.101.252:5060 0.0.0.0:* LISTEN 27587/freeswitch
tcp 0 0 81.27.101.233:5060 0.0.0.0:* LISTEN 27587/freeswitch
udp 0 744 81.27.101.252:5060 0.0.0.0:* 27587/freeswitch
udp 0 0 81.27.101.233:5060 0.0.0.0:* 27587/freeswitch
I stop freeswitch so I can restart it:
ss1:~# killall -9 freeswitch
Freeswitch is no longer running (no zombies etc):
ss1:~# ps faux | grep freeswitch
root 20723 0.0 0.0 5164 788 pts/3 S+ 19:56 0:00 \_ grep --exclude-dir=.svn --color freeswitch
But the port is still open!
ss1:~# netstat -anp | grep :5060
udp 0 744 81.27.101.252:5060 0.0.0.0:* -
This means I can't restart FreeSWITCH because the port is already in use! I have to reboot to free the port up.
Has anyone seen this before? Does anyone know what could cause this?
Since it's still in use after FS has stopped, I'm thinking a Linux kernel problem. I'm running Debian Lenny with kernel 2.6.26-2-amd64. The hardware is a NEC server with 2x E5405 xeon (8cores total) and 8gb ram. It's happened on 2 servers, so I'm thinking something in software. I updated kernel a couple of months back so perhaps that's related, before that everything was fine for 2 years.
Thanks everyone...
-Steve