Lennart Poettering [Wed, 6 Jun 2018 09:58:38 +0000 (11:58 +0200)]
string-util: put together strstrip() from skip_leading_chars() and delete_trailing_chars()
Lennart Poettering [Wed, 6 Jun 2018 09:58:18 +0000 (11:58 +0200)]
process-util: make use of delete_trailing_chars() in get_process_cmdline()
Lennart Poettering [Wed, 6 Jun 2018 09:44:58 +0000 (11:44 +0200)]
update TODO
Michael Prokop [Tue, 5 Jun 2018 22:14:00 +0000 (00:14 +0200)]
man: fix typo
Peter Jones [Tue, 5 Jun 2018 22:15:58 +0000 (18:15 -0400)]
Fix DPI for Logitech M185, M510, and M705. (#9182)
The DPI for three logitech models, the M185, M510, and M705, appear to
have always been different here than what logitech's given specs say.
With my M510, this results in jumpy behavior when transitioning from
fast motion to slow motion, making clicking specific buttons or
highlighting specific text *very* frustrating.
This patch changes those 3 mice to the published resolution, which
resolves the problem with my M510. I have chosen to fix those 3 simply
because they were already grouped together, and all incorrect as
compared to logitech's web site, and as such I have not tried other mice
or investigated if there are other non-matching values in the database
as well.
Signed-off-by: Peter Jones <pjones@redhat.com>
Lennart Poettering [Tue, 5 Jun 2018 19:53:54 +0000 (21:53 +0200)]
util: tighten on_tty() check a bit, also check stderr
Let's detect output redirection a bit better, cover both stdout and
stderr.
Fixes: #9192
Lennart Poettering [Tue, 5 Jun 2018 22:00:58 +0000 (00:00 +0200)]
Merge pull request #9148 from poettering/tidy-late
make PID watching a bit less expensive
Lennart Poettering [Thu, 31 May 2018 13:50:46 +0000 (15:50 +0200)]
core: watch PIDs of scope units right after starting them
Scope units don't have a main or control process we can watch, hence
let's explicitly watch the PIDs contained in them early on, just to make
things more robust and have at least something to watch.
Lennart Poettering [Thu, 31 May 2018 13:41:59 +0000 (15:41 +0200)]
core: rework how we track service and scope PIDs
This reworks how systemd tracks processes on cgroupv1 systems where
cgroup notification is not reliable. Previously, whenever we had reason
to believe that new processes showed up or got removed we'd scan the
cgroup of the scope or service unit for new processes, and would tidy up
the list of PIDs previously watched. This scanning is relatively slow,
and does not scale well. With this change behaviour is changed: instead
of scanning for new/removed processes right away we do this work in a
per-unit deferred event loop job. This event source is scheduled at a
very low priority, so that it is executed when we have time but does not
starve other event sources. This has two benefits: this expensive work is
coalesced, if events happen in quick succession, and we won't delay
SIGCHLD handling for too long.
This patch basically replaces all direct invocation of
unit_watch_all_pids() in scope.c and service.c with invocations of the
new unit_enqueue_rewatch_pids() call which just enqueues a request of
watching/tidying up the PID sets (with one exception: in
scope_enter_signal() and service_enter_signal() we'll still do
unit_watch_all_pids() synchronously first, since we really want to know
all processes we are about to kill so that we can track them properly.
Moreover, all direct invocations of unit_tidy_watch_pids() and
unit_synthesize_cgroup_empty_event() are removed too, when the
unit_enqueue_rewatch_pids() call is invoked, as the queued job will run
those operations too.
All of this is done on cgroupsv1 systems only, and is disabled on
cgroupsv2 systems as cgroup-empty notifications are reliable there, and
we do not need SIGCHLD events to track processes there.
Fixes: #9138
Lennart Poettering [Thu, 31 May 2018 13:40:54 +0000 (15:40 +0200)]
cgroup: tiny log message tweak, say that we ignore one kind of failure
Zbigniew Jędrzejewski-Szmek [Tue, 5 Jun 2018 11:51:07 +0000 (13:51 +0200)]
Merge pull request #9158 from poettering/notify-auto-reload
trigger OnFailure= only if Restart= is not in effect
Zbigniew Jędrzejewski-Szmek [Tue, 5 Jun 2018 07:15:19 +0000 (09:15 +0200)]
man: use entities for fedora number and update URL
Fedora 28 is out already, let's advertise it. While at it, drop "container"
from "f28container" — it's a subdirectory under /var/lib/machines, it's pretty
obvious that's it a container.
To make the switch easier in the future, define the number as an entity.
Evegeny Vereshchagin [Mon, 4 Jun 2018 06:40:43 +0000 (06:40 +0000)]
travis: use Fedora 27 as a base image
It was confirmed experimentally that Fedora 27 is more suitable
for running cov-build than Fedora 28:
https://github.com/systemd/systemd/issues/9186#issuecomment-
394577877.
Evegeny Vereshchagin [Sun, 3 Jun 2018 23:41:09 +0000 (23:41 +0000)]
travis: get python installed while building a docker image
coverity.sh tries to run python, so it should be installed
to make everything work.
According to https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3#Current_status,
python seems to be the right name of the package.
Closes https://github.com/systemd/systemd/issues/9186.
Evgeny Vereshchagin [Mon, 4 Jun 2018 17:53:20 +0000 (20:53 +0300)]
Merge pull request #9184 from msekletar/travis-followups
Travis CI follow ups
Michal Sekletar [Mon, 4 Jun 2018 17:44:07 +0000 (19:44 +0200)]
travis: drop ccache requirement
Michal Sekletar [Mon, 4 Jun 2018 17:37:19 +0000 (19:37 +0200)]
travis: DOCKER_REPOSITORY is no longer set. Don't require it.
Michal Sekletar [Fri, 1 Jun 2018 07:56:13 +0000 (09:56 +0200)]
travis: use Travis CI only for running scheduled Coverity scans
Building and testing of systemd on Fedora Rawhide is now handled by
dedicated Fedora CI. We don't need to duplicate this in Travis CI.
Yu Watanabe [Mon, 4 Jun 2018 12:18:28 +0000 (21:18 +0900)]
conf-parser: fix memleak (#9177)
Fixes CID#1391437.
Closes #9180.
Marcel Hoppe [Thu, 17 May 2018 18:17:47 +0000 (20:17 +0200)]
Update 60-sensor.hwdb
add chuwi hibook with newer bios version
[zj: remove duplicate matrix]
Zbigniew Jędrzejewski-Szmek [Mon, 4 Jun 2018 10:59:22 +0000 (12:59 +0200)]
basic/log: add the log_struct terminator to macro
This way all callers do not need to specify it.
Exhaustively tested by running test-log under valgrind ;)
Lennart Poettering [Mon, 4 Jun 2018 11:45:29 +0000 (13:45 +0200)]
Merge pull request #9176 from keszybz/flags-set
Macro to check if flags are set
Lennart Poettering [Mon, 4 Jun 2018 11:45:03 +0000 (13:45 +0200)]
Merge pull request #9167 from keszybz/ellipsization
Ellipsization fixes based on unit-testing and fuzzing
Lennart Poettering [Mon, 4 Jun 2018 09:58:21 +0000 (11:58 +0200)]
Merge pull request #9147 from keszybz/runtime-enablement
Runtime enablement
Zbigniew Jędrzejewski-Szmek [Mon, 4 Jun 2018 08:29:23 +0000 (10:29 +0200)]
basic/path-util: use FLAGS_SET in one more place
Zbigniew Jędrzejewski-Szmek [Fri, 20 Apr 2018 13:36:20 +0000 (15:36 +0200)]
Add macro for checking if some flags are set
This way we don't need to repeat the argument twice.
I didn't replace all instances. I think it's better to leave out:
- asserts
- comparisons like x & y == x, which are mathematically equivalent, but
here we aren't checking if flags are set, but if the argument fits in the
flags.
Zbigniew Jędrzejewski-Szmek [Fri, 20 Apr 2018 13:21:06 +0000 (15:21 +0200)]
coccinelle: add option to make changes in place
Zbigniew Jędrzejewski-Szmek [Fri, 20 Apr 2018 13:07:42 +0000 (15:07 +0200)]
coccinelle: run spatch just on version-controlled files
Also, allow run-cocinnelle.sh to be started from any directory.
Unfortunately set -x does not work nicely anymore, because the list is
too verbose. Replace it by an echo line.
Lennart Poettering [Mon, 4 Jun 2018 09:35:25 +0000 (11:35 +0200)]
core: go to failure state if the main service process fails and RemainAfterExit=yes (#9159)
Previously, we'd not care about failures that were seen earlier and
remain in "exited" state. This could be triggered if the main process of
a service failed while ExecStartPost= was still running, as in that case
we'd not immediately act on the main process failure because we needed
to wait for ExecStartPost= to finish, before acting on it.
Fixes: #8929
Alan Jenkins [Fri, 1 Jun 2018 18:03:21 +0000 (19:03 +0100)]
login: log session state "closing" (as well as New/Removed)
Let's show a message at the time of logout i.e. entering the "closing"
state, not just e.g. once the user closes `tmux` and the session can be
removed completely. (At least when KillUserProcesses=no applies. My
thinking was we can spare the log noise if we're killing the processes
anyway).
These are two independent events. I think the logout event is quite
significant in the session lifecycle. It will be easier for a user who
does not know logind details to understand why "Removed session" doesn't
appear at logout time, if we have a specific message we can show at this
time :).
Tested using tmux and KillUserProcesses=no. I can also confirm the extra
message doesn't show when using KillUserProcesses=yes. Maybe it looks a
bit mysterious when you use KillOnlyUsers= / KillExcludeUsers=, but
hopefully not alarmingly so.
I was looking at systemd-logind messages on my system, because I can
reproduce two separate problems with Gnome on Fedora 28 where
sessions are unexpectedly in state "closing". (One where a GUI session
limps along in a degraded state[1], and another where spice-vdagent is left
alive after logout, keeping the session around[2]). It logged when
sessions were created and removed, but it didn't log when the session
entered the "closing" state.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1583240#c1
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1583261
Closes #9096
Zbigniew Jędrzejewski-Szmek [Mon, 4 Jun 2018 09:02:25 +0000 (11:02 +0200)]
Merge pull request #9173 from jwrdegoede/hwdb-sensors3
Hwdb sensors3
Zbigniew Jędrzejewski-Szmek [Mon, 4 Jun 2018 08:13:40 +0000 (10:13 +0200)]
Merge pull request #9149 from yuwata/fix-9107
path-util: introduce path_simplify()
Yu Watanabe [Sun, 3 Jun 2018 16:57:21 +0000 (01:57 +0900)]
test: fix function name
Yu Watanabe [Sun, 3 Jun 2018 15:49:41 +0000 (00:49 +0900)]
man: RuntimeDirectory= or friends accept dot contained paths
Yu Watanabe [Sun, 3 Jun 2018 15:36:46 +0000 (00:36 +0900)]
conf-parser: remove redundant utf8-validity check
Yu Watanabe [Sun, 3 Jun 2018 15:30:36 +0000 (00:30 +0900)]
conf-parse: use free_and_replace()
Also removes unnecessary empty lines.
Yu Watanabe [Sun, 3 Jun 2018 15:30:18 +0000 (00:30 +0900)]
conf-parser: reject utf8-invalid lines
Yu Watanabe [Thu, 31 May 2018 14:59:30 +0000 (23:59 +0900)]
test: add test for trailing dot in WorkingDirectory= and RuntimeDirectory=
Yu Watanabe [Sun, 3 Jun 2018 14:59:02 +0000 (23:59 +0900)]
load-fragment: use path_simplify_and_warn() where applicable
Yu Watanabe [Sun, 3 Jun 2018 14:57:30 +0000 (23:57 +0900)]
conf-parse: use path_simplify_and_warn() in config_parse_path()
Yu Watanabe [Fri, 1 Jun 2018 05:11:37 +0000 (14:11 +0900)]
path-util: introduce path_simplify_and_warn()
Yu Watanabe [Sun, 3 Jun 2018 14:54:32 +0000 (23:54 +0900)]
path-util: make path_make_relative() support path including dots
Yu Watanabe [Thu, 31 May 2018 14:39:31 +0000 (23:39 +0900)]
path-util: introduce path_simplify()
The function is similar to path_kill_slashes() but also removes
initial './', trailing '/.', and '/./' in the path.
When the second argument of path_simplify() is false, then it
behaves as the same as path_kill_slashes(). Hence, this also
replaces path_kill_slashes() with path_simplify().
Zbigniew Jędrzejewski-Szmek [Sun, 3 Jun 2018 02:18:23 +0000 (04:18 +0200)]
network: avoid temporary variables for parsing, use TAKE_PTR (#9166)
We don't need a temporary variable when parsing just one number, because
our parsing functions do not touch the output variable on error.
TAKE_PTR is more expressive than 'n = NULL'.
Zbigniew Jędrzejewski-Szmek [Sat, 2 Jun 2018 15:08:46 +0000 (17:08 +0200)]
test-ellipsize: add tests for ellipsize_mem, fix bugs
First, ellipsize() and ellipsize_mem() should not read past the input
buffer. Those functions take an explicit length for the input data, so they
should not assume that the buffer is terminated by a nul.
Second, ellipsization was off in various cases where wide on multi-byte
characters were used.
We had some basic test for ellipsize(), but apparently it wasn't enough to
catch more serious cases.
Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8686.
Zbigniew Jędrzejewski-Szmek [Sat, 2 Jun 2018 15:52:07 +0000 (17:52 +0200)]
basic/string-util: make ellipsize() inline
Once the redundant check is removed, it's a very simple wrapper around
ellipsize_mem().
Zbigniew Jędrzejewski-Szmek [Sat, 2 Jun 2018 14:18:59 +0000 (16:18 +0200)]
Merge pull request #9156 from shr-project/jansa/gcc8
time-util: fix build with gcc8 -Werror=format-truncation=
Hans de Goede [Sat, 2 Jun 2018 13:47:27 +0000 (15:47 +0200)]
hwdb: Add accelerometer orientation quirk for the Onda V891w tablet
Add accelerometer orientation quirk for the Onda V891w tablet, this
does a partial match on the BIOS version because the other strings are
somewhat generic.
The BIOS match is done for both the W89* and D89* BIOS versions which are
for the Windows only and Dual boot versions respectively.
Hans de Goede [Sat, 2 Jun 2018 10:09:43 +0000 (12:09 +0200)]
hwdb: Add accelerometer orientation quirk for the Nuvision/TMAX TM800W560L
Add accelerometer orientation quirk for the Nuvision/TMAX TM800W560L
8" Windows signature edition tablet.
Yu Watanabe [Fri, 1 Jun 2018 09:38:04 +0000 (18:38 +0900)]
netdev: fix parser for VRF.Table=
This effectively reverts
f98dd1e70750f09a85d9236ad3621b249ad59629 (#6704).
Fixes #9150.
Hans de Goede [Mon, 28 May 2018 14:39:16 +0000 (16:39 +0200)]
hwdb: Add accelerometer orientation quirk for the Archos 80 Cesium tablet.
Add accelerometer orientation quirk for the Archos 80 Cesium 8" windows
tablet.
Hans de Goede [Sun, 20 May 2018 15:36:13 +0000 (17:36 +0200)]
hwdb: Extend Point of View TAB-P800W accelerometer quirk for another version
Extend the accelerometer quirk for the Point of View TAB-P800W to also
match the BIOS info on the Point of View TAB-P800W (v2.0).
Zbigniew Jędrzejewski-Szmek [Sat, 2 Jun 2018 08:22:10 +0000 (10:22 +0200)]
test-time-util: print names of test functions
This makes it easier to find the right spot in the long output.
Martin Jansa [Fri, 1 Jun 2018 15:22:28 +0000 (15:22 +0000)]
time-util: fix build with gcc8 -Werror=format-truncation=
* it fails with gcc8 when -O1 or -Os is used (and -ftree-vrp which is added by -O2 and higher isn't used)
../git/src/basic/time-util.c: In function 'format_timespan':
../git/src/basic/time-util.c:508:46: error: '%0*llu' directive output between 1 and
2147483647 bytes may cause result to exceed 'INT_MAX' [-Werror=format-truncation=]
"%s"USEC_FMT".%0*"PRI_USEC"%s",
^~~~
../git/src/basic/time-util.c:508:60: note: format string is defined here
"%s"USEC_FMT".%0*"PRI_USEC"%s",
../git/src/basic/time-util.c:508:46: note: directive argument in the range [0,
18446744073709551614]
"%s"USEC_FMT".%0*"PRI_USEC"%s",
^~~~
../git/src/basic/time-util.c:507:37: note: 'snprintf' output 4 or more bytes (assuming
2147483651) into a destination of size
4294967295
k = snprintf(p, l,
^~~~~~~~~~~~~~
"%s"USEC_FMT".%0*"PRI_USEC"%s",
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
p > buf ? " " : "",
~~~~~~~~~~~~~~~~~~~
a,
~~
j,
~~
b,
~~
table[i].suffix);
~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
[zj: change 'char' to 'signed char']
Mike Gilbert [Thu, 31 May 2018 03:06:33 +0000 (23:06 -0400)]
basic: set errno in raw_clone() on sparc
sparc sets the carry bit when a syscall fails. Use this information to
set errno and return -1 as appropriate.
The added test case calls raw_clone() with flags known to be invalid
according to the clone(2) manpage.
Zbigniew Jędrzejewski-Szmek [Fri, 1 Jun 2018 21:34:53 +0000 (23:34 +0200)]
Merge pull request #9018 from poettering/get-process-comm-is-print
some extra safety checks for get_process_comm()
Lennart Poettering [Thu, 17 May 2018 01:50:35 +0000 (21:50 -0400)]
process-util: also filter non-printable characters in get_process_com()
We already do that in get_process_cmdline(), which is very similar in
behaviour otherwise. Hence, let's be safe and also filter them in
get_process_comm(). Let's try to retain as much information as we can
though and escape rather than suppress unprintable characters. Let's not
increase comm names beyond the kernel limit on such names however.
Also see discussion about this here:
https://marc.info/?l=linux-api&m=
152649570404881&w=2
Lennart Poettering [Fri, 1 Jun 2018 19:45:23 +0000 (21:45 +0200)]
string-util: tweak cellescape() a bit
For short buffer sizes cellescape() was a bit wasteful, as it might
suffice to to drop a single character to find enough place for the full
four byte ellipsis, if that one character was a four character escape.
With this rework we'll guarantee to drop the minimum number of
characters from the end to fit in the ellipsis.
If the buffers we write to are large this doesn't matter much. However,
if they are short (as they are when talking about the process comm
field) then it starts to matter that we put as much information as we
can in the space we get.
Lennart Poettering [Fri, 1 Jun 2018 19:45:06 +0000 (21:45 +0200)]
escape: add an explanatory comment about buffer sizes
Lennart Poettering [Fri, 1 Jun 2018 19:43:43 +0000 (21:43 +0200)]
missing: define kernel internal limit TASK_COMM_LEN in userspace too
We already use it at two places, and we are about to add one too.
Arbitrary literally hardcoded limits suck.
Lennart Poettering [Thu, 17 May 2018 02:14:59 +0000 (22:14 -0400)]
process-util: mention that wait_for_terminate_with_timeout() should be called with SIGCHLD blocked
Lennart Poettering [Fri, 1 Jun 2018 17:13:38 +0000 (19:13 +0200)]
update NEWS to explain new OnFailure= behaviour
Lennart Poettering [Fri, 1 Jun 2018 17:06:19 +0000 (19:06 +0200)]
core: don't trigger OnFailure= deps when a unit is going to restart
This adds a flags parameter to unit_notify() which can be used to pass
additional notification information to the function. We the make the old
reload_failure boolean parameter one of these flags, and then add a new
flag that let's unit_notify() if we are configured to restart the
service.
Note that this adjusts behaviour of systemd to match what the docs say.
Fixes: #8398
Lennart Poettering [Fri, 1 Jun 2018 17:04:37 +0000 (19:04 +0200)]
core: when we can't enqueue OnFailure= job show full error message
Let's ask for the full error message and show it, there's really no
reason to just show the crappy errno error.
Susant Sahani [Fri, 1 Jun 2018 14:22:12 +0000 (19:52 +0530)]
networkd: enable to set IFF_ALLMULTI to network device (#9146)
networkd: allow setting set IFF_ALLMULTI flag on network devices
Ian Miell [Fri, 1 Jun 2018 12:41:05 +0000 (13:41 +0100)]
Minor spelling error - "Our of"->"Out of"
Zbigniew Jędrzejewski-Szmek [Thu, 31 May 2018 13:28:07 +0000 (15:28 +0200)]
systemctl: when removing enablement or mask symlinks, cover both /run and /etc
'systemctl disable --runtime' would disable a unit, but only if it was enabled
with '--runtime', and silently do nothing if the unit was enabled persistently.
And similarly 'systemctl disable' would do nothing if the unit was enabled in
/run. This just doesn't seem useful.
This pathch changes enable/disable and mask/unmask to be asymmetrical. enable
and mask create symlinks in /etc or /run, depending on whether --runtime was
specified. disable and unmask remove symlinks from both locations. --runtime
cannot be specified for the disable and unmask verbs.
The advantage is that 'disable' now means that the unit is disabled, period.
And similarly for 'unmask', all masks are removed.
Similarly for preset and preset-all, they now cannot be called with --runtime,
and are asymmetrical: when they enable a unit, symlinks are created in /etc.
When they disable a unit, all symlinks are nuked.
$ systemctl --root=/ enable bluetooth
Created symlink /etc/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ --runtime enable bluetooth
Created symlink /run/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /run/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ disable bluetooth
Removed /run/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /run/systemd/system/dbus-org.bluez.service.
Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /etc/systemd/system/dbus-org.bluez.service.
$ systemctl --root=/ disable --runtime bluetooth
--runtime cannot be used with disable
$ systemctl --root=/ mask --runtime bluetooth
Created symlink /run/systemd/system/bluetooth.service → /dev/null.
$ systemctl --root=/ mask bluetooth
Created symlink /etc/systemd/system/bluetooth.service → /dev/null.
$ systemctl --root=/ unmask bluetooth
Removed /run/systemd/system/bluetooth.service.
Removed /etc/systemd/system/bluetooth.service.
$ systemctl --root=/ unmask --runtime bluetooth
--runtime cannot be used with unmask
$ systemctl --root=/ --runtime enable bluetooth
Created symlink /run/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /run/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ enable bluetooth
Created symlink /etc/systemd/system/dbus-org.bluez.service → /usr/lib/systemd/system/bluetooth.service.
Created symlink /etc/systemd/system/bluetooth.target.wants/bluetooth.service → /usr/lib/systemd/system/bluetooth.service.
$ systemctl --root=/ preset bluetooth
Removed /run/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /run/systemd/system/dbus-org.bluez.service.
Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service.
Removed /etc/systemd/system/dbus-org.bluez.service.
$ systemctl --root=/ preset --runtime bluetooth
--runtime cannot be used with preset
$ systemctl preset-all --runtime
--runtime cannot be used with preset-all
Zbigniew Jędrzejewski-Szmek [Thu, 31 May 2018 13:32:49 +0000 (15:32 +0200)]
systemctl: remove newlines
This file is long enough already, we don't need extra vertical whitespace.
Zbigniew Jędrzejewski-Szmek [Thu, 31 May 2018 13:16:06 +0000 (15:16 +0200)]
Fix reporting of enabled-runtime units
We would always report them as "enabled", because path_is_config() includes
/run, despite the name.
Fixes #9122.
Zbigniew Jędrzejewski-Szmek [Wed, 30 May 2018 11:22:18 +0000 (11:22 +0000)]
core/dbus: simplify bus_done_{api,system} functions
destroy_bus already has a check for NULL, so we don't need to repeat it here.
Zbigniew Jędrzejewski-Szmek [Tue, 29 May 2018 15:01:53 +0000 (17:01 +0200)]
journal: small simplification
Lennart Poettering [Tue, 29 May 2018 15:12:39 +0000 (17:12 +0200)]
efi: explicitly cast physical address to UINTN when converting to/from pointers
UINTN is the integer type equalling the native ptr size. Let's fix the
casting warnings described in #7788 by casting the the pointers and
values to this type first. That way we cast integers to the right size
first before turning them into pointers, and pointers are first
covnerted to integers of the right size before converting them into
integers.
Not tested, since I lack i386 EFI systems, but I think this is simple
enough to be correct event without testing.
Fixes: #7788
Felipe Sateler [Tue, 22 May 2018 19:08:57 +0000 (15:08 -0400)]
build-sys: Add option to link systemctl statically
Systemctl is special because it is required for many tasks that may need to
be performed when the system is not fully configured and/or partially
broken:
1. Installing/Uninstalling services during OS installs and upgrades
2. Shutting down the system
Therefore reduce the number of dependencies that systemctl pulls in, by
not linking to systemd-shared. This brings a bit of resilience to
systemctl (and its aliases shutdown, reboot, etc), by linking against
less external libraries.
Because this extra resilience comes at a cost of approximately 580 KB
extra space, this is done behind a meson build option.
Lennart Poettering [Thu, 31 May 2018 13:33:44 +0000 (15:33 +0200)]
Merge pull request #9014 from keszybz/fuzz-journal-remote
A fuzzer for journal-remote
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 17:03:03 +0000 (19:03 +0200)]
journal-remote: do not send _BOOT_ID twice
Also remove the comma from the comment everywhere, I think the comma
unnecessarilly put emphasis on the clause after the comma.
Fixes #9090.
Reproducer:
systemd-journal-remote --split-mode=none -o /tmp/msg6.journal --trust=all --listen-http=8080
systemd-journal-upload -u http://localhost:8080
journalctl --file /tmp/msg6.journal -o verbose -n1
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 09:35:07 +0000 (11:35 +0200)]
journal-remote: parse the _BOOT_ID field and use the value when writing entries
The boot id is stored twice, and different code paths use either one or the
other. So we need to store it both in the header and as a field for full
compatibility.
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 09:27:58 +0000 (11:27 +0200)]
journal: allow boot_id to be passed to journal_append_entry()
In this commit, this is done only in testing code, i.e. there is
no functional change apart from tests.
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 09:08:44 +0000 (11:08 +0200)]
journal: remove unused args from journal_file_copy_entry()
Zbigniew Jędrzejewski-Szmek [Thu, 24 May 2018 06:37:47 +0000 (08:37 +0200)]
shared/logs-show: be more careful before using a _SOURCE_REALTIME_TIMESTAMP entry
journalctl -o short would display those entries, but journalctl -o short-full
would refuse. If the entry is bad, just fall back to the receive-side realtime
timestamp like we would if it was completely missing.
Zbigniew Jędrzejewski-Szmek [Thu, 24 May 2018 11:11:12 +0000 (13:11 +0200)]
journal: refuse an entry with invalid timestamp fields
The journal verification functions would reject such an entry. It would probably
still display fine (because we prefer _SOURCE_REALTIME_TIMESTAMP= if present), but
it seems wrong to create an entry that would not pass verification.
Zbigniew Jędrzejewski-Szmek [Thu, 24 May 2018 07:36:56 +0000 (09:36 +0200)]
Use const char* for timestamp strings which we don't plan to modify
Makes the intent a bit clearer.
Zbigniew Jędrzejewski-Szmek [Mon, 21 May 2018 18:39:09 +0000 (20:39 +0200)]
Always allow timestamps to be printed
If the timestamp is above 9999-12-30, (or 2038-something-something on 32 bit),
use XXXX-XX-XX XX:XX:XX as the replacement.
The problem with refusing to print timestamps is that our code accepts such
timestamps, so we can't really just refuse to process them afterwards. Also, it
makes journal files non-portable, because suddently we might completely refuse
to print entries which are totally OK on a different machine.
Zbigniew Jędrzejewski-Szmek [Mon, 21 May 2018 12:42:42 +0000 (14:42 +0200)]
fuzz-journal-remote: write to /dev/null not stdout
This makes the fuzzing much more efficient. Optionally provide output is
$SYSTEMD_FUZZ_OUTPUT is set, which makes debugging of any failures much easier.
The case from
056129deb73df17ece4212db39d2ca0842d9a49c is still detected properly.
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:51:28 +0000 (22:51 +0200)]
basic/journal-importer: "trusted" fields in binary format are not supported
The parser never accepted "__"-prefixed fields in binary format, but there was
a comment questioning this decision. Let's make it official, and remove the
comment.
Also, for clarity, let's move the dunder field parsing after the field
verification check. This doesn't change much, because invalid fields cannot be
known special fields, but is seems cleaner to first verify the validity of the
name, and then check if it is one of the known ones.
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:45:22 +0000 (22:45 +0200)]
basic/journal-importer: reject any field names that journald would reject
$ build-asan/fuzz-journal-remote test/fuzz-regressions/fuzz-journal-remote/crash-
96dee870ea66d03e89ac321eee28ea63a9b9aa45
...
Ignoring invalid field: "S\020"
Ignoring invalid field: "S\020"
...
If the field name includes nul bytes, we won't print all of the name.
But that seems enough of a corner case to ignore.
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:13:08 +0000 (22:13 +0200)]
shared/logs-show: use _cleanup_
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:06:23 +0000 (22:06 +0200)]
shared/logs-show: fix mixup between length-based memory duplication and string operations
We'd look for a '=' separator using memchr, i.e. ignoring any nul bytes in the
string, but then do a strndup, which would terminate on any nul byte, and then
again do a memcmp, which would access memory past the chunk allocated by strndup.
Of course, we probably shouldn't allow keys with nul bytes in them. But we
currently do, so there might be journal files like that out there. So let's fix
the journal-reading code first.
Evgeny Vereshchagin [Sun, 20 May 2018 13:07:07 +0000 (15:07 +0200)]
basic/random-util: do not use getrandom() under msan
`fuzz-journal-remote` seems to be failing under `msan` as soon as it starts:
$ sudo infra/helper.py run_fuzzer systemd fuzz-journal-remote
Running: docker run --rm -i --privileged -e FUZZING_ENGINE=libfuzzer -v /home/vagrant/oss-fuzz/build/out/systemd:/out -t gcr.io/oss-fuzz-base/base-runner run_fuzzer fuzz-journal-remote
Using seed corpus: fuzz-journal-remote_seed_corpus.zip
/out/fuzz-journal-remote -rss_limit_mb=2048 -timeout=25 /tmp/fuzz-journal-remote_corpus -max_len=65536 < /dev/null
INFO: Seed:
3380449479
INFO: Loaded 2 modules (36336 inline 8-bit counters): 36139 [0x7ff36ea31d39, 0x7ff36ea3aa64), 197 [0x9998c8, 0x99998d),
INFO: Loaded 2 PC tables (36336 PCs): 36139 [0x7ff36ea3aa68,0x7ff36eac7d18), 197 [0x999990,0x99a5e0),
INFO: 2 files found in /tmp/fuzz-journal-remote_corpus
INFO: seed corpus: files: 2 min: 4657b max: 7790b total: 12447b rss: 97Mb
Uninitialized bytes in __interceptor_pwrite64 at offset 24 inside [0x7fffdd4d7230, 240)
==15==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x7ff36e685e8a in journal_file_init_header /work/build/../../src/systemd/src/journal/journal-file.c:436:13
#1 0x7ff36e683a9d in journal_file_open /work/build/../../src/systemd/src/journal/journal-file.c:3333:21
#2 0x7ff36e68b8f6 in journal_file_open_reliably /work/build/../../src/systemd/src/journal/journal-file.c:3520:13
#3 0x4a3f35 in open_output /work/build/../../src/systemd/src/journal-remote/journal-remote.c:70:13
#4 0x4a34d0 in journal_remote_get_writer /work/build/../../src/systemd/src/journal-remote/journal-remote.c:136:21
#5 0x4a550f in get_source_for_fd /work/build/../../src/systemd/src/journal-remote/journal-remote.c:183:13
#6 0x4a46bd in journal_remote_add_source /work/build/../../src/systemd/src/journal-remote/journal-remote.c:235:13
#7 0x4a271c in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-journal-remote.c:36:9
#8 0x4f27cc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:524:13
#9 0x4efa0b in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/libfuzzer/FuzzerLoop.cpp:448:3
#10 0x4f8e96 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:732:7
#11 0x4f9f73 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:752:3
#12 0x4bf329 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:756:6
#13 0x4ac391 in main /src/libfuzzer/FuzzerMain.cpp:20:10
#14 0x7ff36d14982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
#15 0x41f9d8 in _start (/out/fuzz-journal-remote+0x41f9d8)
Uninitialized value was stored to memory at
#0 0x7ff36e61cd41 in sd_id128_randomize /work/build/../../src/systemd/src/libsystemd/sd-id128/sd-id128.c:288:16
#1 0x7ff36e685cec in journal_file_init_header /work/build/../../src/systemd/src/journal/journal-file.c:426:13
#2 0x7ff36e683a9d in journal_file_open /work/build/../../src/systemd/src/journal/journal-file.c:3333:21
#3 0x7ff36e68b8f6 in journal_file_open_reliably /work/build/../../src/systemd/src/journal/journal-file.c:3520:13
#4 0x4a3f35 in open_output /work/build/../../src/systemd/src/journal-remote/journal-remote.c:70:13
#5 0x4a34d0 in journal_remote_get_writer /work/build/../../src/systemd/src/journal-remote/journal-remote.c:136:21
#6 0x4a550f in get_source_for_fd /work/build/../../src/systemd/src/journal-remote/journal-remote.c:183:13
#7 0x4a46bd in journal_remote_add_source /work/build/../../src/systemd/src/journal-remote/journal-remote.c:235:13
#8 0x4a271c in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-journal-remote.c:36:9
#9 0x4f27cc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:524:13
#10 0x4efa0b in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/libfuzzer/FuzzerLoop.cpp:448:3
#11 0x4f8e96 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:732:7
#12 0x4f9f73 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:752:3
#13 0x4bf329 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:756:6
#14 0x4ac391 in main /src/libfuzzer/FuzzerMain.cpp:20:10
#15 0x7ff36d14982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
Uninitialized value was created by an allocation of 't' in the stack frame of function 'sd_id128_randomize'
#0 0x7ff36e61cb00 in sd_id128_randomize /work/build/../../src/systemd/src/libsystemd/sd-id128/sd-id128.c:274
SUMMARY: MemorySanitizer: use-of-uninitialized-value /work/build/../../src/systemd/src/journal/journal-file.c:436:13 in journal_file_init_header
Exiting
MS: 0 ; base unit:
0000000000000000000000000000000000000000
artifact_prefix='./'; Test unit written to ./crash-
847911777b3096783f4ee70a69ab6d28380c810b
[vagrant@localhost oss-fuzz]$ sudo infra/helper.py check_build --sanitizer=memory systemd
Running: docker run --rm -i --privileged -e FUZZING_ENGINE=libfuzzer -e SANITIZER=memory -v /home/vagrant/oss-fuzz/build/out/systemd:/out -t gcr.io/oss-fuzz-base/base-runner test_all
INFO: performing bad build checks for /out/fuzz-dhcp-server.
INFO: performing bad build checks for /out/fuzz-journal-remote.
INFO: performing bad build checks for /out/fuzz-unit-file.
INFO: performing bad build checks for /out/fuzz-dns-packet.
4 fuzzers total, 0 seem to be broken (0%).
Check build passed.
It's a false positive which is most likely caused by
https://github.com/google/sanitizers/issues/852. I think it could be got around
by avoiding `getrandom` when the code is compiled with `msan`
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 09:42:55 +0000 (11:42 +0200)]
fuzz-journal-remote: try all output modes
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 09:09:07 +0000 (11:09 +0200)]
basic/journal-importer: escape & ellipsize bad data in log entries
We shouldn't just log arbitrary stuff, in particular newlines and control chars
Now:
Unknown dunder line __CURSORFACILITY=6\nSYSLOG_IDENTIFIER=/USR/SBIN/CRON\nMES…, ignoring.
Unknown dunder line __REALTIME_TIME[TAMP=
1404101101501874\n__MONOTONIC_TIMEST…, ignoring.
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 08:55:21 +0000 (10:55 +0200)]
basic/string-util: add a convenience function to cescape mostly-ascii fields
It's not supposed to be the most efficient, but instead fast and simple to use.
I kept the logic in ellipsize_mem() to use unicode ellipsis even in non-unicode
locales. I'm not quite convinced things should be this way, especially that with
this patch it'd actually be simpler to always use "…" in unicode locale and "..."
otherwise, but Lennart wanted it this way for some reason.
Zbigniew Jędrzejewski-Szmek [Thu, 31 May 2018 11:38:53 +0000 (13:38 +0200)]
Merge pull request #9145 from poettering/itsy-bitsy-fixes
trivial fixes
Thomas H. P. Andersen [Thu, 31 May 2018 11:30:10 +0000 (13:30 +0200)]
sysctl.d: request ECN on both in and outgoing connections (#9143)
To further avoid bufferbloat Explicit Congestion Notification (ECN)
should be enabled for both in and outgoing connections. The kernel
default is to enable it when requested for incoming connections, but
not to request it on outgoing connections. This patch enables it for
both.
A long time ago enabling these was causing problems, but these issues
have since been dealt with.
Fixes #9087.
Lennart Poettering [Tue, 29 May 2018 14:55:46 +0000 (16:55 +0200)]
man: say more explicitly what to do in the SD_JOURNAL_INVALIDATE event
This rewords the section, explicitly distuingishing the cases of clients
that only want a continious log stream (which can simply treat
SD_JOURNAL_INVALIDATE the same way as SD_JOURNAL_APPEND) and those which
want to represent on screen the full state of the log data on disk.
This is an alternative to a part of PR #9060, but keeps an explanation
of the destinction of handling depending on the type of client.
Fixes: #8963
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 08:04:24 +0000 (10:04 +0200)]
basic/journal-importer: do not write non-unicode char to log
The type of cescape_char() is changed to int to make it easier to use
in "%.*s". We know the value is between 1 and 4, so size_t is overkill.
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 07:27:30 +0000 (09:27 +0200)]
journal-remote: when an entry is rejected with -EBADMSG, do not rotate the journal file
Something is wrong with the entry (probably a missing timestamp), so no point
in rotating. But suppress the error in process_source(), so that the processing
of the data stream continues.
Also, just return 0 from writer_write() on success, the only caller doesn't
care.
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 07:07:58 +0000 (09:07 +0200)]
journal-remote: verify realtime and monotonic timestamps early
We would accept any value, and then journal_file_check_object() would reject
the whole entry. Let's just ignore the field.
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 10:05:07 +0000 (12:05 +0200)]
fuzz-journal-remote: a fuzzer for journal-remote over-the-wire input
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 15:21:51 +0000 (17:21 +0200)]
shared/logs-show: export show_journal()
This is a nice function to output some journal entries without much ado.
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 15:12:53 +0000 (17:12 +0200)]
journal: rename output_journal to show_journal_entry
We have show_journal, and output_journal, and it's not immediately clear
how they related. Rename the first to show that it just prints one entry.