C-Gate 2.9.5 on Linux - thread count

I had C-gate 2.8.0 running on a Linux box (32-bit) which I have just upgraded.

To make the 2.8.0 install stable I had to restart C-Gate nightly - I don't recall exactly why now but I think CPU and memory usage crept up over time and restarting at 3am every morning was a quick fix. With the upgrade to 2.9.5 I've taken that restart out to see if the system can be stable without it.

Checking this morning (it's been running less than 24 hours) C-Gate is now using 15% of available RAM (it was 8% last night), and the number of threads is growing (currently 1265, it was 1261 when I started writing this post).

Suggestions as to what might be causing this?

I have level 9 logs if needed.

(Thread count now 1269...)

 

Log sent to [email protected]

Looking at the log, however, it appears that the extra threads correlate to my application code opening connections on 20024/5 which never get logged as having closed. netstat confirms that I do not have 1000+ active threads (I have 19 in CLOSE_WAIT state at time of writing). I am obviously doing something wrong here and will investigate how these threads are being closed, but I suspect that C-Gate should still be killing those threads at some point?

Mark

 

I was wrong on this point: the application code I thought we had running accessing C-Gate on 20024 hasn't been in use since 2011...

However, what we do have is the "monit" monitoring application configured to check that C-Gate is functioning correctly. It tests that it can connect to 20023 (and receives "201 Service Ready" in response), and that it can connect to 20024/20025/20123. In other words it will make TCP connections to those ports, and then drop the connection without sending anything, as part of a monitoring program to ensure C-Gate is responding correctly.

Is there a reason why this should cause a problem, or is this an issue in C-Gate?

It does appear to be repeatable:

Code:

telnet localhost 20024
Ctrl-]
quit

.. causes the thread count to increase by one. I don't know of any way to close these connections other than dropping the connection in this way?

 

Indeed...

Looking at this on Windows now (to avoid any potential differences with Linux), I started with a thread count of 20 (from "get cgate *").

After telnet to localhost on 20024 and quitting telnet, this was up to 21 threads. netstat reports the inbound state as FIN_WAIT_2 state and the outbound state as CLOSE_WAIT. If I wait until both of these have cleared from the netstat output C-Gate still reports a thread count of 21.

This is repeatable. I'm pretty sure this is a resource leak in C-Gate.

Interestingly if I run the same test with port 20023 (closing telnet rather than issuing a quit command for consistency) the TCP states are the same (as would be expected) but the C-Gate thread count (correctly) goes up to 22 on connection and returns to 21 immediately I kill the connection. I would expect the same to be true of connections to 20024/20025?

Hope this helps!

 

That is quite probably the first time I've had a bug report described as "awesome", you've just made my day! Thanks for that!