platform/upstream/systemd.git
6 years agonetwork: avoid temporary variables for parsing, use TAKE_PTR (#9166)
Zbigniew Jędrzejewski-Szmek [Sun, 3 Jun 2018 02:18:23 +0000 (04:18 +0200)]
network: avoid temporary variables for parsing, use TAKE_PTR (#9166)

We don't need a temporary variable when parsing just one number, because
our parsing functions do not touch the output variable on error.

TAKE_PTR is more expressive than 'n = NULL'.

6 years agoMerge pull request #9156 from shr-project/jansa/gcc8
Zbigniew Jędrzejewski-Szmek [Sat, 2 Jun 2018 14:18:59 +0000 (16:18 +0200)]
Merge pull request #9156 from shr-project/jansa/gcc8

time-util: fix build with gcc8 -Werror=format-truncation=

6 years agonetdev: fix parser for VRF.Table=
Yu Watanabe [Fri, 1 Jun 2018 09:38:04 +0000 (18:38 +0900)]
netdev: fix parser for VRF.Table=

This effectively reverts f98dd1e70750f09a85d9236ad3621b249ad59629 (#6704).

Fixes #9150.

6 years agotest-time-util: print names of test functions
Zbigniew Jędrzejewski-Szmek [Sat, 2 Jun 2018 08:22:10 +0000 (10:22 +0200)]
test-time-util: print names of test functions

This makes it easier to find the right spot in the long output.

6 years agotime-util: fix build with gcc8 -Werror=format-truncation=
Martin Jansa [Fri, 1 Jun 2018 15:22:28 +0000 (15:22 +0000)]
time-util: fix build with gcc8 -Werror=format-truncation=

* it fails with gcc8 when -O1 or -Os is used (and -ftree-vrp which is added by -O2 and higher isn't used)

../git/src/basic/time-util.c: In function 'format_timespan':
../git/src/basic/time-util.c:508:46: error: '%0*llu' directive output between 1 and 2147483647 bytes may cause result to exceed 'INT_MAX' [-Werror=format-truncation=]
                                              "%s"USEC_FMT".%0*"PRI_USEC"%s",
                                              ^~~~
../git/src/basic/time-util.c:508:60: note: format string is defined here
                                              "%s"USEC_FMT".%0*"PRI_USEC"%s",
../git/src/basic/time-util.c:508:46: note: directive argument in the range [0, 18446744073709551614]
                                              "%s"USEC_FMT".%0*"PRI_USEC"%s",
                                              ^~~~
../git/src/basic/time-util.c:507:37: note: 'snprintf' output 4 or more bytes (assuming 2147483651) into a destination of size 4294967295
                                 k = snprintf(p, l,
                                     ^~~~~~~~~~~~~~
                                              "%s"USEC_FMT".%0*"PRI_USEC"%s",
                                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                              p > buf ? " " : "",
                                              ~~~~~~~~~~~~~~~~~~~
                                              a,
                                              ~~
                                              j,
                                              ~~
                                              b,
                                              ~~
                                              table[i].suffix);
                                              ~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors

[zj: change 'char' to 'signed char']

6 years agobasic: set errno in raw_clone() on sparc
Mike Gilbert [Thu, 31 May 2018 03:06:33 +0000 (23:06 -0400)]
basic: set errno in raw_clone() on sparc

sparc sets the carry bit when a syscall fails. Use this information to
set errno and return -1 as appropriate.

The added test case calls raw_clone() with flags known to be invalid
according to the clone(2) manpage.

6 years agoMerge pull request #9018 from poettering/get-process-comm-is-print
Zbigniew Jędrzejewski-Szmek [Fri, 1 Jun 2018 21:34:53 +0000 (23:34 +0200)]
Merge pull request #9018 from poettering/get-process-comm-is-print

some extra safety checks for get_process_comm()

6 years agoprocess-util: also filter non-printable characters in get_process_com()
Lennart Poettering [Thu, 17 May 2018 01:50:35 +0000 (21:50 -0400)]
process-util: also filter non-printable characters in get_process_com()

We already do that in get_process_cmdline(), which is very similar in
behaviour otherwise. Hence, let's be safe and also filter them in
get_process_comm(). Let's try to retain as much information as we can
though and escape rather than suppress unprintable characters. Let's not
increase comm names beyond the kernel limit on such names however.

Also see discussion about this here:

https://marc.info/?l=linux-api&m=152649570404881&w=2

6 years agostring-util: tweak cellescape() a bit
Lennart Poettering [Fri, 1 Jun 2018 19:45:23 +0000 (21:45 +0200)]
string-util: tweak cellescape() a bit

For short buffer sizes cellescape() was a bit wasteful, as it might
suffice to to drop a single character to find enough place for the full
four byte ellipsis, if that one character was a four character escape.
With this rework we'll guarantee to drop the minimum number of
characters from the end to fit in the ellipsis.

If the buffers we write to are large this doesn't matter much. However,
if they are short (as they are when talking about the process comm
field) then it starts to matter that we put as much information as we
can in the space we get.

6 years agoescape: add an explanatory comment about buffer sizes
Lennart Poettering [Fri, 1 Jun 2018 19:45:06 +0000 (21:45 +0200)]
escape: add an explanatory comment about buffer sizes

6 years agomissing: define kernel internal limit TASK_COMM_LEN in userspace too
Lennart Poettering [Fri, 1 Jun 2018 19:43:43 +0000 (21:43 +0200)]
missing: define kernel internal limit TASK_COMM_LEN in userspace too

We already use it at two places, and we are about to add one too.
Arbitrary literally hardcoded limits suck.

6 years agoprocess-util: mention that wait_for_terminate_with_timeout() should be called with...
Lennart Poettering [Thu, 17 May 2018 02:14:59 +0000 (22:14 -0400)]
process-util: mention that wait_for_terminate_with_timeout() should be called with SIGCHLD blocked

6 years agonetworkd: enable to set IFF_ALLMULTI to network device (#9146)
Susant Sahani [Fri, 1 Jun 2018 14:22:12 +0000 (19:52 +0530)]
networkd: enable to set IFF_ALLMULTI to network device (#9146)

networkd: allow setting set IFF_ALLMULTI flag on network devices

6 years agoMinor spelling error - "Our of"->"Out of"
Ian Miell [Fri, 1 Jun 2018 12:41:05 +0000 (13:41 +0100)]
Minor spelling error - "Our of"->"Out of"

6 years agoefi: explicitly cast physical address to UINTN when converting to/from pointers
Lennart Poettering [Tue, 29 May 2018 15:12:39 +0000 (17:12 +0200)]
efi: explicitly cast physical address to UINTN when converting to/from pointers

UINTN is the integer type equalling the native ptr size. Let's fix the
casting warnings described in #7788 by casting the the pointers and
values to this type first. That way we cast integers to the right size
first before turning them into pointers, and pointers are first
covnerted to integers of the right size before converting them into
integers.

Not tested, since I lack i386 EFI systems, but I think this is simple
enough to be correct event without testing.

Fixes: #7788

6 years agobuild-sys: Add option to link systemctl statically
Felipe Sateler [Tue, 22 May 2018 19:08:57 +0000 (15:08 -0400)]
build-sys: Add option to link systemctl statically

Systemctl is special because it is required for many tasks that may need to
be performed when the system is not fully configured and/or partially
broken:

1. Installing/Uninstalling services during OS installs and upgrades
2. Shutting down the system

Therefore reduce the number of dependencies that systemctl pulls in, by
not linking to systemd-shared. This brings a bit of resilience to
systemctl (and its aliases shutdown, reboot, etc), by linking against
less external libraries.

Because this extra resilience comes at a cost of approximately 580 KB
extra space, this is done behind a meson build option.

6 years agoMerge pull request #9014 from keszybz/fuzz-journal-remote
Lennart Poettering [Thu, 31 May 2018 13:33:44 +0000 (15:33 +0200)]
Merge pull request #9014 from keszybz/fuzz-journal-remote

A fuzzer for journal-remote

6 years agojournal-remote: do not send _BOOT_ID twice
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 17:03:03 +0000 (19:03 +0200)]
journal-remote: do not send _BOOT_ID twice

Also remove the comma from the comment everywhere, I think the comma
unnecessarilly put emphasis on the clause after the comma.

Fixes #9090.

Reproducer:
systemd-journal-remote --split-mode=none -o /tmp/msg6.journal --trust=all --listen-http=8080
systemd-journal-upload -u http://localhost:8080
journalctl --file /tmp/msg6.journal -o verbose -n1

6 years agojournal-remote: parse the _BOOT_ID field and use the value when writing entries
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 09:35:07 +0000 (11:35 +0200)]
journal-remote: parse the _BOOT_ID field and use the value when writing entries

The boot id is stored twice, and different code paths use either one or the
other. So we need to store it both in the header and as a field for full
compatibility.

6 years agojournal: allow boot_id to be passed to journal_append_entry()
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 09:27:58 +0000 (11:27 +0200)]
journal: allow boot_id to be passed to journal_append_entry()

In this commit, this is done only in testing code, i.e. there is
no functional change apart from tests.

6 years agojournal: remove unused args from journal_file_copy_entry()
Zbigniew Jędrzejewski-Szmek [Sun, 27 May 2018 09:08:44 +0000 (11:08 +0200)]
journal: remove unused args from journal_file_copy_entry()

6 years agoshared/logs-show: be more careful before using a _SOURCE_REALTIME_TIMESTAMP entry
Zbigniew Jędrzejewski-Szmek [Thu, 24 May 2018 06:37:47 +0000 (08:37 +0200)]
shared/logs-show: be more careful before using a _SOURCE_REALTIME_TIMESTAMP entry

journalctl -o short would display those entries, but journalctl -o short-full
would refuse. If the entry is bad, just fall back to the receive-side realtime
timestamp like we would if it was completely missing.

6 years agojournal: refuse an entry with invalid timestamp fields
Zbigniew Jędrzejewski-Szmek [Thu, 24 May 2018 11:11:12 +0000 (13:11 +0200)]
journal: refuse an entry with invalid timestamp fields

The journal verification functions would reject such an entry. It would probably
still display fine (because we prefer _SOURCE_REALTIME_TIMESTAMP= if present), but
it seems wrong to create an entry that would not pass verification.

6 years agoUse const char* for timestamp strings which we don't plan to modify
Zbigniew Jędrzejewski-Szmek [Thu, 24 May 2018 07:36:56 +0000 (09:36 +0200)]
Use const char* for timestamp strings which we don't plan to modify

Makes the intent a bit clearer.

6 years agoAlways allow timestamps to be printed
Zbigniew Jędrzejewski-Szmek [Mon, 21 May 2018 18:39:09 +0000 (20:39 +0200)]
Always allow timestamps to be printed

If the timestamp is above 9999-12-30, (or 2038-something-something on 32 bit),
use XXXX-XX-XX XX:XX:XX as the replacement.

The problem with refusing to print timestamps is that our code accepts such
timestamps, so we can't really just refuse to process them afterwards. Also, it
makes journal files non-portable, because suddently we might completely refuse
to print entries which are totally OK on a different machine.

6 years agofuzz-journal-remote: write to /dev/null not stdout
Zbigniew Jędrzejewski-Szmek [Mon, 21 May 2018 12:42:42 +0000 (14:42 +0200)]
fuzz-journal-remote: write to /dev/null not stdout

This makes the fuzzing much more efficient. Optionally provide output is
$SYSTEMD_FUZZ_OUTPUT is set, which makes debugging of any failures much easier.

The case from 056129deb73df17ece4212db39d2ca0842d9a49c is still detected properly.

6 years agobasic/journal-importer: "trusted" fields in binary format are not supported
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:51:28 +0000 (22:51 +0200)]
basic/journal-importer: "trusted" fields in binary format are not supported

The parser never accepted "__"-prefixed fields in binary format, but there was
a comment questioning this decision. Let's make it official, and remove the
comment.

Also, for clarity, let's move the dunder field parsing after the field
verification check. This doesn't change much, because invalid fields cannot be
known special fields, but is seems cleaner to first verify the validity of the
name, and then check if it is one of the known ones.

6 years agobasic/journal-importer: reject any field names that journald would reject
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:45:22 +0000 (22:45 +0200)]
basic/journal-importer: reject any field names that journald would reject

$ build-asan/fuzz-journal-remote test/fuzz-regressions/fuzz-journal-remote/crash-96dee870ea66d03e89ac321eee28ea63a9b9aa45
...
Ignoring invalid field: "S\020"
Ignoring invalid field: "S\020"
...

If the field name includes nul bytes, we won't print all of the name.
But that seems enough of a corner case to ignore.

6 years agoshared/logs-show: use _cleanup_
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:13:08 +0000 (22:13 +0200)]
shared/logs-show: use _cleanup_

6 years agoshared/logs-show: fix mixup between length-based memory duplication and string operations
Zbigniew Jędrzejewski-Szmek [Sun, 20 May 2018 20:06:23 +0000 (22:06 +0200)]
shared/logs-show: fix mixup between length-based memory duplication and string operations

We'd look for a '=' separator using memchr, i.e. ignoring any nul bytes in the
string, but then do a strndup, which would terminate on any nul byte, and then
again do a memcmp, which would access memory past the chunk allocated by strndup.

Of course, we probably shouldn't allow keys with nul bytes in them. But we
currently do, so there might be journal files like that out there. So let's fix
the journal-reading code first.

6 years agobasic/random-util: do not use getrandom() under msan
Evgeny Vereshchagin [Sun, 20 May 2018 13:07:07 +0000 (15:07 +0200)]
basic/random-util: do not use getrandom() under msan

`fuzz-journal-remote` seems to be failing under `msan` as soon as it starts:

$ sudo infra/helper.py run_fuzzer systemd fuzz-journal-remote
Running: docker run --rm -i --privileged -e FUZZING_ENGINE=libfuzzer -v /home/vagrant/oss-fuzz/build/out/systemd:/out -t gcr.io/oss-fuzz-base/base-runner run_fuzzer fuzz-journal-remote
Using seed corpus: fuzz-journal-remote_seed_corpus.zip
/out/fuzz-journal-remote -rss_limit_mb=2048 -timeout=25 /tmp/fuzz-journal-remote_corpus -max_len=65536 < /dev/null
INFO: Seed: 3380449479
INFO: Loaded 2 modules   (36336 inline 8-bit counters): 36139 [0x7ff36ea31d39, 0x7ff36ea3aa64), 197 [0x9998c8, 0x99998d),
INFO: Loaded 2 PC tables (36336 PCs): 36139 [0x7ff36ea3aa68,0x7ff36eac7d18), 197 [0x999990,0x99a5e0),
INFO:        2 files found in /tmp/fuzz-journal-remote_corpus
INFO: seed corpus: files: 2 min: 4657b max: 7790b total: 12447b rss: 97Mb
Uninitialized bytes in __interceptor_pwrite64 at offset 24 inside [0x7fffdd4d7230, 240)
==15==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7ff36e685e8a in journal_file_init_header /work/build/../../src/systemd/src/journal/journal-file.c:436:13
    #1 0x7ff36e683a9d in journal_file_open /work/build/../../src/systemd/src/journal/journal-file.c:3333:21
    #2 0x7ff36e68b8f6 in journal_file_open_reliably /work/build/../../src/systemd/src/journal/journal-file.c:3520:13
    #3 0x4a3f35 in open_output /work/build/../../src/systemd/src/journal-remote/journal-remote.c:70:13
    #4 0x4a34d0 in journal_remote_get_writer /work/build/../../src/systemd/src/journal-remote/journal-remote.c:136:21
    #5 0x4a550f in get_source_for_fd /work/build/../../src/systemd/src/journal-remote/journal-remote.c:183:13
    #6 0x4a46bd in journal_remote_add_source /work/build/../../src/systemd/src/journal-remote/journal-remote.c:235:13
    #7 0x4a271c in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-journal-remote.c:36:9
    #8 0x4f27cc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:524:13
    #9 0x4efa0b in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/libfuzzer/FuzzerLoop.cpp:448:3
    #10 0x4f8e96 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:732:7
    #11 0x4f9f73 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:752:3
    #12 0x4bf329 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:756:6
    #13 0x4ac391 in main /src/libfuzzer/FuzzerMain.cpp:20:10
    #14 0x7ff36d14982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #15 0x41f9d8 in _start (/out/fuzz-journal-remote+0x41f9d8)

  Uninitialized value was stored to memory at
    #0 0x7ff36e61cd41 in sd_id128_randomize /work/build/../../src/systemd/src/libsystemd/sd-id128/sd-id128.c:288:16
    #1 0x7ff36e685cec in journal_file_init_header /work/build/../../src/systemd/src/journal/journal-file.c:426:13
    #2 0x7ff36e683a9d in journal_file_open /work/build/../../src/systemd/src/journal/journal-file.c:3333:21
    #3 0x7ff36e68b8f6 in journal_file_open_reliably /work/build/../../src/systemd/src/journal/journal-file.c:3520:13
    #4 0x4a3f35 in open_output /work/build/../../src/systemd/src/journal-remote/journal-remote.c:70:13
    #5 0x4a34d0 in journal_remote_get_writer /work/build/../../src/systemd/src/journal-remote/journal-remote.c:136:21
    #6 0x4a550f in get_source_for_fd /work/build/../../src/systemd/src/journal-remote/journal-remote.c:183:13
    #7 0x4a46bd in journal_remote_add_source /work/build/../../src/systemd/src/journal-remote/journal-remote.c:235:13
    #8 0x4a271c in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-journal-remote.c:36:9
    #9 0x4f27cc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:524:13
    #10 0x4efa0b in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/libfuzzer/FuzzerLoop.cpp:448:3
    #11 0x4f8e96 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:732:7
    #12 0x4f9f73 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:752:3
    #13 0x4bf329 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:756:6
    #14 0x4ac391 in main /src/libfuzzer/FuzzerMain.cpp:20:10
    #15 0x7ff36d14982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

  Uninitialized value was created by an allocation of 't' in the stack frame of function 'sd_id128_randomize'
    #0 0x7ff36e61cb00 in sd_id128_randomize /work/build/../../src/systemd/src/libsystemd/sd-id128/sd-id128.c:274

SUMMARY: MemorySanitizer: use-of-uninitialized-value /work/build/../../src/systemd/src/journal/journal-file.c:436:13 in journal_file_init_header
Exiting
MS: 0 ; base unit: 0000000000000000000000000000000000000000
artifact_prefix='./'; Test unit written to ./crash-847911777b3096783f4ee70a69ab6d28380c810b
[vagrant@localhost oss-fuzz]$ sudo infra/helper.py check_build --sanitizer=memory systemd
Running: docker run --rm -i --privileged -e FUZZING_ENGINE=libfuzzer -e SANITIZER=memory -v /home/vagrant/oss-fuzz/build/out/systemd:/out -t gcr.io/oss-fuzz-base/base-runner test_all
INFO: performing bad build checks for /out/fuzz-dhcp-server.
INFO: performing bad build checks for /out/fuzz-journal-remote.
INFO: performing bad build checks for /out/fuzz-unit-file.
INFO: performing bad build checks for /out/fuzz-dns-packet.
4 fuzzers total, 0 seem to be broken (0%).
Check build passed.

It's a false positive which is most likely caused by
https://github.com/google/sanitizers/issues/852. I think it could be got around
by avoiding `getrandom` when the code is compiled with `msan`

6 years agofuzz-journal-remote: try all output modes
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 09:42:55 +0000 (11:42 +0200)]
fuzz-journal-remote: try all output modes

6 years agobasic/journal-importer: escape & ellipsize bad data in log entries
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 09:09:07 +0000 (11:09 +0200)]
basic/journal-importer: escape & ellipsize bad data in log entries

We shouldn't just log arbitrary stuff, in particular newlines and control chars
Now:
Unknown dunder line __CURSORFACILITY=6\nSYSLOG_IDENTIFIER=/USR/SBIN/CRON\nMES…, ignoring.
Unknown dunder line __REALTIME_TIME[TAMP=1404101101501874\n__MONOTONIC_TIMEST…, ignoring.

6 years agobasic/string-util: add a convenience function to cescape mostly-ascii fields
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 08:55:21 +0000 (10:55 +0200)]
basic/string-util: add a convenience function to cescape mostly-ascii fields

It's not supposed to be the most efficient, but instead fast and simple to use.

I kept the logic in ellipsize_mem() to use unicode ellipsis even in non-unicode
locales. I'm not quite convinced things should be this way, especially that with
this patch it'd actually be simpler to always use "…" in unicode locale and "..."
otherwise, but Lennart wanted it this way for some reason.

6 years agoMerge pull request #9145 from poettering/itsy-bitsy-fixes
Zbigniew Jędrzejewski-Szmek [Thu, 31 May 2018 11:38:53 +0000 (13:38 +0200)]
Merge pull request #9145 from poettering/itsy-bitsy-fixes

trivial fixes

6 years agosysctl.d: request ECN on both in and outgoing connections (#9143)
Thomas H. P. Andersen [Thu, 31 May 2018 11:30:10 +0000 (13:30 +0200)]
sysctl.d: request ECN on both in and outgoing connections (#9143)

To further avoid bufferbloat Explicit Congestion Notification (ECN)
should be enabled for both in and outgoing connections. The kernel
default is to enable it when requested for incoming connections, but
not to request it on outgoing connections. This patch enables it for
both.

A long time ago enabling these was causing problems, but these issues
have since been dealt with.

Fixes #9087.

6 years agoman: say more explicitly what to do in the SD_JOURNAL_INVALIDATE event
Lennart Poettering [Tue, 29 May 2018 14:55:46 +0000 (16:55 +0200)]
man: say more explicitly what to do in the SD_JOURNAL_INVALIDATE event

This rewords the section, explicitly distuingishing the cases of clients
that only want a continious log stream (which can simply treat
SD_JOURNAL_INVALIDATE the same way as SD_JOURNAL_APPEND) and those which
want to represent on screen the full state of the log data on disk.

This is an alternative to a part of PR #9060, but keeps an explanation
of the destinction of handling depending on the type of client.

Fixes: #8963

6 years agobasic/journal-importer: do not write non-unicode char to log
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 08:04:24 +0000 (10:04 +0200)]
basic/journal-importer: do not write non-unicode char to log

The type of cescape_char() is changed to int to make it easier to use
in "%.*s". We know the value is between 1 and 4, so size_t is overkill.

6 years agojournal-remote: when an entry is rejected with -EBADMSG, do not rotate the journal...
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 07:27:30 +0000 (09:27 +0200)]
journal-remote: when an entry is rejected with -EBADMSG, do not rotate the journal file

Something is wrong with the entry (probably a missing timestamp), so no point
in rotating. But suppress the error in process_source(), so that the processing
of the data stream continues.

Also, just return 0 from writer_write() on success, the only caller doesn't
care.

6 years agojournal-remote: verify realtime and monotonic timestamps early
Zbigniew Jędrzejewski-Szmek [Thu, 17 May 2018 07:07:58 +0000 (09:07 +0200)]
journal-remote: verify realtime and monotonic timestamps early

We would accept any value, and then journal_file_check_object() would reject
the whole entry. Let's just ignore the field.

6 years agofuzz-journal-remote: a fuzzer for journal-remote over-the-wire input
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 10:05:07 +0000 (12:05 +0200)]
fuzz-journal-remote: a fuzzer for journal-remote over-the-wire input

6 years agoshared/logs-show: export show_journal()
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 15:21:51 +0000 (17:21 +0200)]
shared/logs-show: export show_journal()

This is a nice function to output some journal entries without much ado.

6 years agojournal: rename output_journal to show_journal_entry
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 15:12:53 +0000 (17:12 +0200)]
journal: rename output_journal to show_journal_entry

We have show_journal, and output_journal, and it's not immediately clear
how they related. Rename the first to show that it just prints one entry.

6 years agojournal: allow writing journal files even if machine-id is missing
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 12:02:24 +0000 (14:02 +0200)]
journal: allow writing journal files even if machine-id is missing

The code to open journal files seems like the wrong place to enforce this. We
already check during boot and refuse to boot if machine-id is missing, no need
to enforce this here. In particular, it seems better to write logs from
journald even if they are not completely functional rather than refuse to
operate at all, and systemd-journal-remote also writes journal files and may
even be run on a system without systemd at all.

The docker image that oss-fuzz uses has an empty /etc/machine-id. Obviously
this is an error in the docker, but docker is fact of life, and it seems better
for systemd-journal-remote to work in such an incomplete environment.

6 years agosd-id128: return -ENOMEDIUM on null id
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 11:55:12 +0000 (13:55 +0200)]
sd-id128: return -ENOMEDIUM on null id

We currently return -ENOMEDIUM when /etc/machine-id is empty, and -EINVAL when
it is all zeros. But -EINVAL is also used for invalid args. The distinction
between empty and all-zero is not very important, let's use the same return
code.

Also document -ENOENT and -ENOMEDIUM since they can be a bit surprising.

6 years agofuzz-journal-remote: allow fuzzer to be built without µhttpd
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 11:30:37 +0000 (13:30 +0200)]
fuzz-journal-remote: allow fuzzer to be built without µhttpd

journal-remote still requires µhttpd, but things are easier if the fuzzer
can be built without.

6 years agoshared/dissect-image: avoid warning about unused function when compiled w/o blkid
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 10:40:07 +0000 (12:40 +0200)]
shared/dissect-image: avoid warning about unused function when compiled w/o blkid

6 years agolog-generator: make message size configurable, add short options
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 10:05:43 +0000 (12:05 +0200)]
log-generator: make message size configurable, add short options

Also remove "b''" from the generated MESSAGE= field.

6 years agojournal-remote: export handle_raw_source()
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 09:36:52 +0000 (11:36 +0200)]
journal-remote: export handle_raw_source()

6 years agojournal-remote: split out µhttpd support and main() into a separate file
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 08:21:58 +0000 (10:21 +0200)]
journal-remote: split out µhttpd support and main() into a separate file

This is in preparation to reusing the RemoteServer in other concepts.
I tried to keep changes to minimum:
- arg_* global variables are now passed as state in RemoteServer
- exported functions get the "journal_remote_" prefix
- some variables are renamed

In particular, there is an ugly global RemoveServer* variable. It was originally
added because µhttpd did not allow state to be passed to the callbacks. I'm not
sure if this has been remediated in µhttpd, but either way, this is not changed
here, the global variable is only renamed for clarity.

6 years agojournal: rewrap function args
Zbigniew Jędrzejewski-Szmek [Wed, 16 May 2018 06:24:33 +0000 (08:24 +0200)]
journal: rewrap function args

6 years agotree-wide: drop some double newlines
Lennart Poettering [Thu, 31 May 2018 10:19:34 +0000 (12:19 +0200)]
tree-wide: drop some double newlines

6 years agococcinelle says we should use EOPNOTSUPP rather than ENOTSUP
Lennart Poettering [Thu, 31 May 2018 10:06:54 +0000 (12:06 +0200)]
coccinelle says we should use EOPNOTSUPP rather than ENOTSUP

6 years agoportablectl: no need to validate profile name before checking whether it is 'help'
Lennart Poettering [Thu, 31 May 2018 10:06:37 +0000 (12:06 +0200)]
portablectl: no need to validate profile name before checking whether it is 'help'

6 years agoportablectl: support 'help' parameter on --copy=
Lennart Poettering [Thu, 31 May 2018 10:06:12 +0000 (12:06 +0200)]
portablectl: support 'help' parameter on --copy=

6 years agoportablectl: don't join strv if we don't want to display it
Lennart Poettering [Thu, 31 May 2018 10:05:53 +0000 (12:05 +0200)]
portablectl: don't join strv if we don't want to display it

6 years agoportablectl: coccinelle says we should use TAKE_PTR() here, do so
Lennart Poettering [Thu, 31 May 2018 10:05:20 +0000 (12:05 +0200)]
portablectl: coccinelle says we should use TAKE_PTR() here, do so

6 years agotree-wide: some O_NDELAY → O_NONBLOCK fixes
Lennart Poettering [Thu, 31 May 2018 10:04:37 +0000 (12:04 +0200)]
tree-wide: some O_NDELAY → O_NONBLOCK fixes

Somehow the coccinelle script misses these, hence fix them manually.

6 years agoMerge pull request #9115 from yuwata/rfe-8491
Lennart Poettering [Thu, 31 May 2018 09:54:01 +0000 (11:54 +0200)]
Merge pull request #9115 from yuwata/rfe-8491

locale: re-read configuration files if necessary

6 years agoMerge pull request #9144 from yuwata/sysusers-spec
Lennart Poettering [Thu, 31 May 2018 09:41:29 +0000 (11:41 +0200)]
Merge pull request #9144 from yuwata/sysusers-spec

sysusers: support specifier expansion for GECOS and home directory

6 years agoMerge pull request #9131 from poettering/memory-startswith
Lennart Poettering [Thu, 31 May 2018 09:21:43 +0000 (11:21 +0200)]
Merge pull request #9131 from poettering/memory-startswith

add new memory_startswith() helper and make use of it everywhere

6 years agoMerge pull request #9083 from yuwata/empty-assignment
Lennart Poettering [Thu, 31 May 2018 09:13:52 +0000 (11:13 +0200)]
Merge pull request #9083 from yuwata/empty-assignment

load-fragment: make many settings accept empty string

6 years agolocale: assume locale is for LANG= when variable name is not specified
Yu Watanabe [Thu, 31 May 2018 07:27:07 +0000 (16:27 +0900)]
locale: assume locale is for LANG= when variable name is not specified

Closes #6070.

6 years agosmack: make mac_smack_fix() deal somewhat sensible with non-absolute paths
Lennart Poettering [Wed, 30 May 2018 16:21:58 +0000 (18:21 +0200)]
smack: make mac_smack_fix() deal somewhat sensible with non-absolute paths

This tries to improve the mac_smack_fix() logic a bit, by properly
handling non-absolute paths.

It's still pretty broken though, which is sad for security technology:
non-normalized paths (for example "/usr/../dev/sda") will still not be
treated correctly. I am not sure how to fix that properly though, and I
don't understand SMACK well enough to do so. This fix hence just fixes
to most obvious glaring issue.

6 years agoMerge pull request #9142 from yuwata/coverity-fixes
Lennart Poettering [Thu, 31 May 2018 08:50:10 +0000 (10:50 +0200)]
Merge pull request #9142 from yuwata/coverity-fixes

Coverity fixes

6 years agodoc: update TODO
Yu Watanabe [Thu, 31 May 2018 08:39:13 +0000 (17:39 +0900)]
doc: update TODO

6 years agoman: mention that systemd-sysusers understand the specifier.
Yu Watanabe [Thu, 31 May 2018 08:37:15 +0000 (17:37 +0900)]
man: mention that systemd-sysusers understand the specifier.

6 years agosysusers: support specifier expansion for GECOS and home directory fields
Yu Watanabe [Thu, 31 May 2018 08:36:22 +0000 (17:36 +0900)]
sysusers: support specifier expansion for GECOS and home directory fields

6 years agolocale: re-read configuration files if changed
Yu Watanabe [Thu, 31 May 2018 07:22:05 +0000 (16:22 +0900)]
locale: re-read configuration files if changed

Closes #8491.

6 years agolocale: use _cleanup_free_ and free_and_replace
Yu Watanabe [Tue, 29 May 2018 16:35:26 +0000 (01:35 +0900)]
locale: use _cleanup_free_ and free_and_replace

6 years agolocale: check xkb data before requesting authentication
Yu Watanabe [Tue, 29 May 2018 06:49:21 +0000 (15:49 +0900)]
locale: check xkb data before requesting authentication

6 years agolocale: return earlier if no operation will be done
Yu Watanabe [Thu, 31 May 2018 07:15:21 +0000 (16:15 +0900)]
locale: return earlier if no operation will be done

6 years agoload-fragment: use free_and_replace()
Yu Watanabe [Thu, 31 May 2018 04:27:06 +0000 (13:27 +0900)]
load-fragment: use free_and_replace()

6 years agoload-fragment: add missing oom check
Yu Watanabe [Thu, 31 May 2018 04:26:24 +0000 (13:26 +0900)]
load-fragment: add missing oom check

Fixes CID#1391379.

6 years agoportable: fix memleak
Yu Watanabe [Thu, 31 May 2018 04:16:47 +0000 (13:16 +0900)]
portable: fix memleak

Fixes CID#1391382.

6 years agocore/socket: fix coding style
Yu Watanabe [Tue, 29 May 2018 04:32:05 +0000 (13:32 +0900)]
core/socket: fix coding style

6 years agoload-fragment: move macro-defined config parsers
Yu Watanabe [Thu, 31 May 2018 02:04:52 +0000 (11:04 +0900)]
load-fragment: move macro-defined config parsers

6 years agoload-fragment: use DEFINE_CONFIG_PARSE_*() macros
Yu Watanabe [Tue, 29 May 2018 03:57:06 +0000 (12:57 +0900)]
load-fragment: use DEFINE_CONFIG_PARSE_*() macros

6 years agoload-fragment: make IPTOS= accept the empty string
Yu Watanabe [Tue, 29 May 2018 03:52:14 +0000 (12:52 +0900)]
load-fragment: make IPTOS= accept the empty string

6 years agoload-fragment: make SocketProtocol= accept the empty string
Yu Watanabe [Tue, 29 May 2018 03:49:47 +0000 (12:49 +0900)]
load-fragment: make SocketProtocol= accept the empty string

6 years agonamespace: rename parse_protect_{home,system}_or_bool() to protect_{home,system}_or_b...
Yu Watanabe [Tue, 29 May 2018 03:44:57 +0000 (12:44 +0900)]
namespace: rename parse_protect_{home,system}_or_bool() to protect_{home,system}_or_bool_to_string()

Hence, we can define config_parse_protect_{home,system}() by using
DEFINE_CONFIG_PARSE_ENUM() macro.

6 years agosocket-util: rename parse_socket_address_bind_ipv6_only_or_bool() to socket_address_b...
Yu Watanabe [Tue, 29 May 2018 03:40:28 +0000 (12:40 +0900)]
socket-util: rename parse_socket_address_bind_ipv6_only_or_bool() to socket_address_bind_ipv6_only_or_bool_from_string()

Hence, we can define config_parse_socket_bind() by using
DEFINE_CONFIG_PARSE_ENUM() macro.

6 years agoconf-parser: introduce DEFINE_CONFIG_PARSE*() macros
Yu Watanabe [Sat, 26 May 2018 16:39:12 +0000 (01:39 +0900)]
conf-parser: introduce DEFINE_CONFIG_PARSE*() macros

This introduces several macros for defining config parsers.
Also this fixes errno in DEFINE_CONFIG_PARSE_ENUM() and _ENUMV()
and makes the log level lower when a duplicated item is
specified to the settings parsed by the function defined by
DEFINE_CONFIG_PARSE_ENUMV().

6 years agoload-fragment: drop config_parse_no_new_privileges() and use config_parse_bool()...
Yu Watanabe [Fri, 25 May 2018 03:52:12 +0000 (12:52 +0900)]
load-fragment: drop config_parse_no_new_privileges() and use config_parse_bool() instead

6 years agoload-fragment: make WorkingDirectory= accept the empty string
Yu Watanabe [Fri, 25 May 2018 03:25:41 +0000 (12:25 +0900)]
load-fragment: make WorkingDirectory= accept the empty string

6 years agoload-fragment: use parse_sec_fix_0() instead of updating the value later
Yu Watanabe [Fri, 25 May 2018 03:17:52 +0000 (12:17 +0900)]
load-fragment: use parse_sec_fix_0() instead of updating the value later

6 years agoload-fragment: use config_parse_sec_fix_0() for TimeoutStopSec=
Yu Watanabe [Fri, 25 May 2018 03:16:24 +0000 (12:16 +0900)]
load-fragment: use config_parse_sec_fix_0() for TimeoutStopSec=

6 years agoload-fragment: always include config_parse_warn_compat() in the table used by unit_du...
Yu Watanabe [Fri, 25 May 2018 02:54:59 +0000 (11:54 +0900)]
load-fragment: always include config_parse_warn_compat() in the table used by unit_dump_config_items()

6 years agoload-fragment: drop unused function config_parse_sysv_priority()
Yu Watanabe [Fri, 25 May 2018 02:53:39 +0000 (11:53 +0900)]
load-fragment: drop unused function config_parse_sysv_priority()

6 years agoload-fragment: make CPUSchedulingPolicy= accept the empty string
Yu Watanabe [Fri, 25 May 2018 02:36:10 +0000 (11:36 +0900)]
load-fragment: make CPUSchedulingPolicy= accept the empty string

6 years agoload-fragment: make IOScheduling{Class,Priority}= accept the empty string
Yu Watanabe [Fri, 25 May 2018 02:25:43 +0000 (11:25 +0900)]
load-fragment: make IOScheduling{Class,Priority}= accept the empty string

6 years agoload-fragment: setting empty string to Nice= resets the previous assignments
Yu Watanabe [Fri, 25 May 2018 01:52:47 +0000 (10:52 +0900)]
load-fragment: setting empty string to Nice= resets the previous assignments

6 years agohwdb: consolidate multiple razer evdev sections
Jared Kazimir [Wed, 30 May 2018 15:30:09 +0000 (11:30 -0400)]
hwdb: consolidate multiple razer evdev sections

6 years agoMerge pull request #9132 from poettering/sd-bus-slot-set-floating
Lennart Poettering [Wed, 30 May 2018 22:58:41 +0000 (00:58 +0200)]
Merge pull request #9132 from poettering/sd-bus-slot-set-floating

make sure we don't leak bus slots when sd_bus_add_match_async() is used

6 years agoFix pattern to detect distribution
Sergio Lindo Mansilla [Fri, 18 May 2018 14:24:36 +0000 (16:24 +0200)]
Fix pattern to detect distribution

With operator '=', when in /etc/os-release:
ID_LIKE="opensuse suse" # Tumbleweed since at least 2018-04-24
or
ID_LIKE="suse opensuse" # Leap 15

6 years agosd-bus: make add match method callback slot "floating"
Lennart Poettering [Wed, 30 May 2018 14:35:36 +0000 (16:35 +0200)]
sd-bus: make add match method callback slot "floating"

When we allocate an asynchronous match object we will allocate an
asynchronous bus call object to install the match server side.
Previously the call slot would be created as regular slot, i.e.
non-floating which meant installing the match even if it was itself
floating would result in a non-floating slot to be created internally,
which ultimately would mean the sd_bus object would be referenced by it,
and thus never be freed.

Let's fix that by making the match method callback floating in any case
as we have no interest in leaving the bus allocated beyond the match
slot.

Fixes: #8551

6 years agobus-slot: for bus slot objects with no explicit description use the match string...
Lennart Poettering [Wed, 30 May 2018 14:34:06 +0000 (16:34 +0200)]
bus-slot: for bus slot objects with no explicit description use the match string as description

Let's make debugging a but easier with implicit descriptions for some
match objects.

6 years agoman: document the new sd_bus_slot_set_floating() call
Lennart Poettering [Wed, 30 May 2018 14:33:12 +0000 (16:33 +0200)]
man: document the new sd_bus_slot_set_floating() call

Also extend the memory management description of sd-bus highlighting the
effect of "floating" slot objects a bit.

6 years agosd-bus: add new sd_bus_slot_set_floating() call
Lennart Poettering [Wed, 30 May 2018 14:29:33 +0000 (16:29 +0200)]
sd-bus: add new sd_bus_slot_set_floating() call

This new call allows explicit control of the "floating" state of a bus
slot object. This is useful for creating a bus slot object first,
retaining a reference to it, using it for making changes to the slot
object (for example, set a description) and then handing it over to
sd-bus for lifecycle management.

It's also useful to fix #8551.

6 years agoman: fix minor typo
Lennart Poettering [Wed, 30 May 2018 14:28:28 +0000 (16:28 +0200)]
man: fix minor typo