listen socket more frequent service

listen socket more frequent service

From an idea by Edwin van den Oetelaar <oetelaar.automatisering@gmail.com>

When testing libwebsockets with ab, Edwin found an unexpected bump in
the distribution of latencies, some connections were held back almost
the whole test duration.

http://ml.libwebsockets.org/pipermail/libwebsockets/2013-January/000006.html

Studying the problem revealed that when there are mass pending connections
amongst many active connections, we do not service the listen socket often
enough to clear the backlog, some seem to get stale violating FIFO ordering.

This patch introduces listen socket service "piggybacking", where every n
normal socket service actions we also check the listen socket and deal with
pending connections there.

Normally, it checks the listen socket gratuitously every 10 normal socket
services.  However, if it finds something waiting, it forces a check on the
next normal socket service too by keeping stats on how often something was
waiting.  If the probability of something waiting each time becomes high,
it will allow up to two waiting connections to be serviced for each normal
socket service.

In that way it has low burden in the normal case, but rapidly adapts by
detecting mass connection loads as found in ab.

Signed-off-by: Andy Green <andy.green@linaro.org>