bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on fds, not...
authorRichard Purdie <richard.purdie@linuxfoundation.org>
Sat, 31 Aug 2013 22:40:55 +0000 (23:40 +0100)
committerRichard Purdie <richard.purdie@linuxfoundation.org>
Sun, 1 Sep 2013 14:51:10 +0000 (15:51 +0100)
commit08bde0a3f3f9ffb351635f19c42f379e29a4320a
tree7c8f8e01a7427f459740b404022906563049afc4
parentd2eacb20c1da3e9bf24f9fc0635227b48ce5677d
bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on fds, not time.sleep()

The existing backend server implementations were inefficient since they
were sleeping for the full length of the timeouts rather than being woken when
there was data ready for them. It was assumed they would wake and perhaps did
when we forked processes directory but that is no longer the case.

This updates both the process and xmlrpc backends to wait using select(). This
does mean we need to pass the file descriptors to wait on from the internals
who know which these file descriptors are but this is a logical improvement.

Tests of a pathaolgical load on the process server of ~420 rapid tasks
executed on a server with BB_NUMBER_THREAD=48  went from a wall clock
measurement of the overall command execution time of 75s to a much more
reasonable 24s.

(Bitbake rev: 9bee497960889d9baa0a4284d79a384b18a8e826)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake/lib/bb/runqueue.py
bitbake/lib/bb/server/process.py
bitbake/lib/bb/server/xmlrpc.py