[Freeswitch-users] Threads remain after calling close on Java client

Neil Davis ndavis at inetwork.com
Thu Dec 22 02:18:50 MSK 2011


Hector,



Thanks for your suggestions.  I tried both your fix and modifying the close
method as suggested, but I still have threads hanging around in the
TIMED-WAITING status.  They eventually terminate after about a minute.



For the time being, I’m just stalling the Tomcat shutdown with
Thread.sleeps until all the hanging threads have terminated.   Tomcat is
then able to shutdown gracefully.  I’ll look a little more at the esl code
as I have time and update the mailing list if I find a solution.



Thanks,



Neil Davis



*From:* freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] *On Behalf Of *Hector
Geraldino
*Sent:* Wednesday, December 21, 2011 12:26 PM
*To:* FreeSWITCH Users Help
*Subject:* Re: [Freeswitch-users] Threads remain after calling close on
Java client



Hi Neil,



This doesn’t seem to be the same concurrency issue I had, but I’m attaching
the patch that fixes my issue anyway.  Feel free to test it and send me
back the restuls.



In case it doesn’t work you might try to “manually” close the channel by
modifying the close() method on the
org.freeswitch.esl.client.inbound.Client. Try to do a channel.disconnect();
and channel=null; and see what happens.



*From:* freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] *On Behalf Of *Neil
Davis
*Sent:* Wednesday, December 21, 2011 11:17 AM
*To:* FreeSWITCH Users Help
*Subject:* Re: [Freeswitch-users] Threads remain after calling close on
Java client



Here is a thread dump of my Tomcat process at the point when it is hanging
on shutdown.  There are a number of threads in a “waiting on condition”
state that appear to have to do with the netty package on which the
Freeswitch client is dependent.





2011-12-21 09:05:17

Full thread dump Java HotSpot(TM) Client VM (14.3-b01 mixed mode):



"DestroyJavaVM" prio=6 tid=0x546f7400 nid=0x640 waiting on condition
[0x00000000]

   java.lang.Thread.State: RUNNABLE



   Locked ownable synchronizers:

                - None



"pool-5-thread-16" prio=6 tid=0x544b5800 nid=0x1098 waiting on condition
[0x55e4f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-15" prio=6 tid=0x54f7f400 nid=0x10f0 waiting on condition
[0x54e5f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-14" prio=6 tid=0x544b3c00 nid=0xdb4 waiting on condition
[0x5670f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-13" prio=6 tid=0x544b4000 nid=0x12c8 waiting on condition
[0x566bf000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-12" prio=6 tid=0x544b4800 nid=0xe18 waiting on condition
[0x5666f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-11" prio=6 tid=0x544b3000 nid=0x620 waiting on condition
[0x5661f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-10" prio=6 tid=0x552a1800 nid=0x131c waiting on condition
[0x565cf000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-9" prio=6 tid=0x552a1400 nid=0x1710 waiting on condition
[0x5657f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-8" prio=6 tid=0x552a0c00 nid=0x1094 waiting on condition
[0x5652f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-7" prio=6 tid=0x552a0800 nid=0x1040 waiting on condition
[0x564df000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-6" prio=6 tid=0x552a0000 nid=0x179c waiting on condition
[0x5648f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-5" prio=6 tid=0x5529fc00 nid=0x3e4 waiting on condition
[0x5643f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-4" prio=6 tid=0x5529f400 nid=0x63c waiting on condition
[0x563ef000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-3" prio=6 tid=0x5529f000 nid=0x17e8 waiting on condition
[0x5639f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-2" prio=6 tid=0x5529e800 nid=0x574 waiting on condition
[0x5634f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-4-thread-1" prio=6 tid=0x5529e400 nid=0x10cc waiting on condition
[0x562ff000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                - parking to wait for  <0x09970378> (a
java.util.concurrent.SynchronousQueue$TransferStack)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown
Source)

                at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)

                at java.util.concurrent.SynchronousQueue.poll(Unknown
Source)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-3-thread-1" prio=6 tid=0x546f8800 nid=0x654 waiting on condition
[0x562af000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                - parking to wait for  <0x09970580> (a
java.util.concurrent.SynchronousQueue$TransferStack)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown
Source)

                at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)

                at java.util.concurrent.SynchronousQueue.poll(Unknown
Source)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"pool-5-thread-1" prio=6 tid=0x546f8000 nid=0x594 waiting on condition
[0x5625f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.awaitMatch(LinkedTransferQueue.java:766)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.xfer(LinkedTransferQueue.java:673)

                at
org.jboss.netty.util.internal.LinkedTransferQueue.poll(LinkedTransferQueue.java:1164)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"RMI TCP Connection(2)-10.0.0.22" daemon prio=6 tid=0x551db800 nid=0xf6c
runnable [0x5593f000]

   java.lang.Thread.State: RUNNABLE

                at java.net.SocketInputStream.socketRead0(Native Method)

                at java.net.SocketInputStream.read(Unknown Source)

                at java.io.BufferedInputStream.fill(Unknown Source)

                at java.io.BufferedInputStream.read(Unknown Source)

                - locked <0x08f5c2d0> (a java.io.BufferedInputStream)

                at java.io.FilterInputStream.read(Unknown Source)

                at
sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)

                at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)

                at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - <0x08f62b10> (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)



"JMX server connection timeout 22" daemon prio=6 tid=0x5474fc00 nid=0x424
in Object.wait() [0x558ef000]

   java.lang.Thread.State: TIMED_WAITING (on object monitor)

                at java.lang.Object.wait(Native Method)

                - waiting on <0x08e9bdb0> (a [I)

                at
com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(Unknown
Source)

                - locked <0x08e9bdb0> (a [I)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"RMI Scheduler(0)" daemon prio=6 tid=0x5511d400 nid=0x1420 waiting on
condition [0x5589f000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                - parking to wait for  <0x08e9bdd0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
Source)

                at java.util.concurrent.DelayQueue.take(Unknown Source)

                at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown
Source)

                at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown
Source)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"RMI TCP Connection(idle)" daemon prio=6 tid=0x55176800 nid=0x884 waiting
on condition [0x557ff000]

   java.lang.Thread.State: TIMED_WAITING (parking)

                at sun.misc.Unsafe.park(Native Method)

                - parking to wait for  <0x08ebd088> (a
java.util.concurrent.SynchronousQueue$TransferStack)

                at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
Source)

                at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown
Source)

                at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)

                at java.util.concurrent.SynchronousQueue.poll(Unknown
Source)

                at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
Source)

                at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"RMI TCP Accept-0" daemon prio=6 tid=0x5518bc00 nid=0x163c runnable
[0x557af000]

   java.lang.Thread.State: RUNNABLE

                at java.net.PlainSocketImpl.socketAccept(Native Method)

                at java.net.PlainSocketImpl.accept(Unknown Source)

                - locked <0x08e9e158> (a java.net.SocksSocketImpl)

                at java.net.ServerSocket.implAccept(Unknown Source)

                at java.net.ServerSocket.accept(Unknown Source)

                at
sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(Unknown
Source)

                at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(Unknown
Source)

                at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(Unknown Source)

                at java.lang.Thread.run(Unknown Source)



   Locked ownable synchronizers:

                - None



"GC Daemon" daemon prio=2 tid=0x5464b000 nid=0x1718 in Object.wait()
[0x5497f000]

   java.lang.Thread.State: TIMED_WAITING (on object monitor)

                at java.lang.Object.wait(Native Method)

                - waiting on <0x089b1270> (a sun.misc.GC$LatencyLock)

                at sun.misc.GC$Daemon.run(Unknown Source)

                - locked <0x089b1270> (a sun.misc.GC$LatencyLock)



   Locked ownable synchronizers:

                - None



"Low Memory Detector" daemon prio=6 tid=0x01a12c00 nid=0x14fc runnable
[0x00000000]

   java.lang.Thread.State: RUNNABLE



   Locked ownable synchronizers:

                - None



"CompilerThread0" daemon prio=10 tid=0x01a0f800 nid=0x1ec waiting on
condition [0x00000000]

   java.lang.Thread.State: RUNNABLE



   Locked ownable synchronizers:

                - None



"JDWP Event Helper Thread" daemon prio=6 tid=0x01a01400 nid=0x173c runnable
[0x00000000]

   java.lang.Thread.State: RUNNABLE



   Locked ownable synchronizers:

                - None



"Attach Listener" daemon prio=10 tid=0x019f5000 nid=0x13a4 waiting on
condition [0x00000000]

   java.lang.Thread.State: RUNNABLE



   Locked ownable synchronizers:

                - None



"Signal Dispatcher" daemon prio=10 tid=0x019ea000 nid=0x17b8 runnable
[0x00000000]

   java.lang.Thread.State: RUNNABLE



   Locked ownable synchronizers:

                - None



"Finalizer" daemon prio=8 tid=0x019bb400 nid=0x1720 in Object.wait()
[0x53fcf000]

   java.lang.Thread.State: WAITING (on object monitor)

                at java.lang.Object.wait(Native Method)

                - waiting on <0x089b2858> (a
java.lang.ref.ReferenceQueue$Lock)

                at java.lang.ref.ReferenceQueue.remove(Unknown Source)

                - locked <0x089b2858> (a java.lang.ref.ReferenceQueue$Lock)

                at java.lang.ref.ReferenceQueue.remove(Unknown Source)

                at java.lang.ref.Finalizer$FinalizerThread.run(Unknown
Source)



   Locked ownable synchronizers:

                - None



"Reference Handler" daemon prio=10 tid=0x019ba000 nid=0x53c in
Object.wait() [0x53f7f000]

   java.lang.Thread.State: WAITING (on object monitor)

                at java.lang.Object.wait(Native Method)

                - waiting on <0x089b2878> (a java.lang.ref.Reference$Lock)

                at java.lang.Object.wait(Object.java:485)

                at java.lang.ref.Reference$ReferenceHandler.run(Unknown
Source)

                - locked <0x089b2878> (a java.lang.ref.Reference$Lock)



   Locked ownable synchronizers:

                - None



"VM Thread" prio=10 tid=0x019b7400 nid=0xab8 runnable



"VM Periodic Task Thread" prio=10 tid=0x01a1bc00 nid=0xc7c waiting on
condition



JNI global references: 21240





*From:* freeswitch-users-bounces at lists.freeswitch.org [mailto:
freeswitch-users-bounces at lists.freeswitch.org] *On Behalf Of *Hector
Geraldino
*Sent:* Monday, December 19, 2011 8:29 AM
*To:* FreeSWITCH Users Help
*Subject:* Re: [Freeswitch-users] Threads remain after calling close on
Java client



Hi Neil,



Can you get a thread dump of the tomcat process to try to figure out what
this problem is about? Or at least, try to connect the jconsole to the
tomcat process and get the StackTrace of one of these threads to have a
better idea of what is going on.



IIRC I’ve fixed a couple of bugs for this library, but the patches haven’t
been tested by the main developer (dvarnes)  nor integrated on the
repository (freeswitch-contrib). If this problem can be fixed with my
patched code, I would be happy to share it with you.



Good luck!



*From:* freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] *On Behalf Of *Neil
Davis
*Sent:* Friday, December 16, 2011 7:09 PM
*To:* FreeSWITCH-users at lists.freeswitch.org
*Subject:* [Freeswitch-users] Threads remain after calling close on Java
client



Hi,



I built a web application that connects to Freeswitch using the
org.freeswitch.esl.client.Client.  I connect the Client object from a
Spring annotated service that I call from a Spring controller.  I put the
connected client in my ServletContext, so I can access it later to call
client.cancelEventSubscriptions() and client.close() from my
ServletContextListener contextDestroyed method when Tomcat is shutting down.



The problem I'm having is that even after I call close on the client, there
are still a bunch of active threads that the client has spawned in the
background.  These threads are causing Tomcat to hang when I'm shutting
down.  Can anyone suggest an approach that would enable my application to
disconnect the Freeswitch client when Tomcat is shutting down that would
allow Tomcat to shutdown gracefully?



Below are errors from my Tomcat log for the threads that I have identified
as being related to the Freeswitch client.  I don't know how I can get to
these threads to interrupt them and Client.close() seems to leave them
hanging.



Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-1] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-3-thread-1] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-4-thread-1] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-2] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-3] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-4] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-5] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-6] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-7] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-8] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-9] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-10] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-11] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-12] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-13] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-14] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-15] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads

SEVERE: The web application [/socketspy] appears to have started a thread
named [pool-5-thread-16] but has failed to stop it. This is very likely to
create a memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks

SEVERE: The web application [/socketspy] created a ThreadLocal with key of
type [org.jboss.netty.util.internal.ThreadLocalBoolean] (value
[org.jboss.netty.util.internal.ThreadLocalBoolean at 186e192]) and a value of
type [java.lang.Boolean] (value [false]) but failed to remove it when the
web application was stopped. Threads are going to be renewed over time to
try and avoid a probable memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks

SEVERE: The web application [/socketspy] created a ThreadLocal with key of
type [org.jboss.netty.util.CharsetUtil$1] (value
[org.jboss.netty.util.CharsetUtil$1 at 14d8e1]) and a value of type
[java.util.IdentityHashMap] (value
[{windows-1252=sun.nio.cs.MS1252$Encoder at 373f86}]) but failed to remove it
when the web application was stopped. Threads are going to be renewed over
time to try and avoid a probable memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks

SEVERE: The web application [/socketspy] created a ThreadLocal with key of
type [org.jboss.netty.util.internal.ThreadLocalRandom$1] (value
[org.jboss.netty.util.internal.ThreadLocalRandom$1 at 12bb519]) and a value of
type [org.jboss.netty.util.internal.ThreadLocalRandom] (value
[org.jboss.netty.util.internal.ThreadLocalRandom at 7e9dbc]) but failed to
remove it when the web application was stopped. Threads are going to be
renewed over time to try and avoid a probable memory leak.

Dec 16, 2011 12:04:59 PM org.apache.catalina.loader.WebappClassLoader
checkThreadLocalMapForLeaks

SEVERE: The web application [/socketspy] created a ThreadLocal with key of
type [org.jboss.netty.util.CharsetUtil$1] (value
[org.jboss.netty.util.CharsetUtil$1 at 14d8e1]) and a value of type
[java.util.IdentityHashMap] (value
[{windows-1252=sun.nio.cs.MS1252$Encoder at a5b041}]) but failed to remove it
when the web application was stopped. Threads are going to be renewed over
time to try and avoid a probable memory leak.





Thanks,



Neil Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111221/90af41fa/attachment-0001.html 


Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list