rate limit calls to posix_fadvise()
authorPádraig Brady <pbrady@fb.com>
Fri, 9 Dec 2016 19:45:34 +0000 (19:45 +0000)
committerPádraig Brady <pbrady@fb.com>
Fri, 9 Dec 2016 19:45:34 +0000 (19:45 +0000)
commitdacd29679633c9b845708e7015bd2c79367a6ea2
tree9807f7c2a9900955bacb148672fe992700b41f8c
parentb6a5e0524c28178985f0d228e9eaa43808dbec3c
rate limit calls to posix_fadvise()

There can be a large kernel overhead involved in POSIX_FADV_DONTNEED.
There is no point in calling this per item logged, so rate limit
to at most once per 2MiB written.

With a simple test program that logs 100K items at WARNING level:

Before:

  $ time strace -c -e fadvise64 log.test \
    -log_dir=/dev/shm -logtofiles=true -logtostderr=false
  % time     seconds  usecs/call     calls    errors syscall
  ------ ----------- ----------- --------- --------- ----------------
  100.00   12.522509         125     99957           fadvise64
  ------ ----------- ----------- --------- --------- ----------------
  real    0m52.671s
  user    0m2.194s
  sys     0m44.022s

After:

  $ time strace -c -e fadvise64 log.test \
    -log_dir=/dev/shm -logtofiles=true -logtostderr=false

  % time     seconds  usecs/call     calls    errors syscall
  ------ ----------- ----------- --------- --------- ----------------
  100.00    0.000759         152         5           fadvise64
  ------ ----------- ----------- --------- --------- ----------------
  real    0m4.206s
  user    0m1.436s
  sys     0m3.153s

Fixes issue #84
src/logging.cc