platform/upstream/libxkbcommon.git
4 years agotest: add "how to type" demo program
Ran Benita [Fri, 20 Mar 2020 16:29:03 +0000 (18:29 +0200)]
test: add "how to type" demo program

The program takes a unicode codepoint and an RMLVO and prints out all
key + modifier combinations that would result in that codepoint.

The program was written to exercise the new
xkb_keymap_key_get_mods_for_level() function. It's handy and can be
extended in several ways, but enough for now.

Example:

    $ ./build/how-to-type -l us,il,ru 0x41 | column -ts $'\t'
    keysym: A (0x41)
    KEYCODE  KEY NAME  LAYOUT#  LAYOUT NAME   LEVEL#  MODIFIERS
    38       AC01      1        English (US)  2       [ Shift ]
    38       AC01      1        English (US)  2       [ Lock ]
    38       AC01      2        Hebrew        2       [ Shift ]
    38       AC01      2        Hebrew        2       [ Lock ]

    $ ./build/how-to-type -l de -v neo 0x3b6 | column -ts $'\t'
    keysym: Greek_zeta (0x7e6)
    KEYCODE  KEY NAME  LAYOUT#  LAYOUT NAME     LEVEL#  MODIFIERS
    56       AB05      1        German (Neo 2)  4       [ Shift Mod5 ]
    56       AB05      1        German (Neo 2)  4       [ Shift Mod2 Mod3 Mod5 ]
    56       AB05      1        German (Neo 2)  4       [ Shift Lock Mod5 ]
    56       AB05      1        German (Neo 2)  4       [ Lock Mod2 Mod3 Mod5 ]

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agodoc: add missing `@since` tags
Ran Benita [Fri, 20 Mar 2020 17:15:48 +0000 (19:15 +0200)]
doc: add missing `@since` tags

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoAPI to query modifier set required to type a keysym
Jaroslaw Kubik [Wed, 5 Feb 2020 16:42:06 +0000 (17:42 +0100)]
API to query modifier set required to type a keysym

The new API is useful to implement features like auto-type and
desktop automation. Since the inputs for these features is usually
specified in terms of the symbols that need to be typed, the
implementation needs to be able to invert the keycode->keysym
transformation and produce a sequence of keycodes that can be used
to type the requested character(s).

4 years agoSupport translation Unicode codepoints to keysyms
Jaroslaw Kubik [Wed, 12 Feb 2020 22:44:42 +0000 (23:44 +0100)]
Support translation Unicode codepoints to keysyms

In order to support features like auto-type and UI automation, the
relevant tools need to be able to invert the keycode->keysym->text
transformation. In order to facilitate that, a new API was added.
It allows querying the keysyms that correspond to particular Unicode
codepoints. For all practical purposes, it can be thought of as an
inverse of xkb_keysym_to_utf32().

4 years agobuild: fix build with byacc
Ran Benita [Tue, 18 Feb 2020 12:12:20 +0000 (14:12 +0200)]
build: fix build with byacc

We apparently broke byacc support in the switch to meson.

byacc only supports short option names. And to make things fun, bison
only supports long option for `--defines`.

Fixes: https://github.com/xkbcommon/libxkbcommon/issues/133
Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agodoc: must -> should
Ran Benita [Sun, 16 Feb 2020 11:32:09 +0000 (13:32 +0200)]
doc: must -> should

Using an out-of-range is not an error, just useless.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoBump version to 0.10.0 upstream_bkup_20210105 upstream_work_20200225 xkbcommon-0.10.0
Ran Benita [Sat, 18 Jan 2020 21:08:28 +0000 (23:08 +0200)]
Bump version to 0.10.0

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agodoc/rules-format.txt: document include support
Ran Benita [Sat, 18 Jan 2020 21:06:58 +0000 (23:06 +0200)]
doc/rules-format.txt: document include support

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agodocs: add missing reference to $XDG_CONFIG_HOME
Ran Benita [Sat, 18 Jan 2020 20:59:58 +0000 (22:59 +0200)]
docs: add missing reference to $XDG_CONFIG_HOME

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoCI: Add build instructions for Windows
Adrian Perez de Castro [Mon, 12 Aug 2019 20:14:25 +0000 (23:14 +0300)]
CI: Add build instructions for Windows

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoMSVC: Provide an implementation of gettimeofday()
Adrian Perez de Castro [Mon, 5 Aug 2019 13:57:45 +0000 (16:57 +0300)]
MSVC: Provide an implementation of gettimeofday()

4 years agoMSVC: Provide implementations of [un]setenv()
Adrian Perez de Castro [Mon, 5 Aug 2019 12:53:04 +0000 (15:53 +0300)]
MSVC: Provide implementations of [un]setenv()

Reference:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/putenv-s-wputenv-s

4 years agoMSVC: Provide implementations of test_{dis,en}able_stdin_echo
Adrian Perez de Castro [Mon, 5 Aug 2019 12:44:33 +0000 (15:44 +0300)]
MSVC: Provide implementations of test_{dis,en}able_stdin_echo

This provides implementations of the test_enable_stdin_echo and
test_disable_stdin_echo which do not require <termios.h>, which is
not available on Windows.

4 years agoMSVC: Use <io.h> as an alternative for <unistd.h>
Adrian Perez de Castro [Mon, 5 Aug 2019 10:52:18 +0000 (13:52 +0300)]
MSVC: Use <io.h> as an alternative for <unistd.h>

Only the input/output functions from <unistd.h> options are used, so
using <io.h> when building with MSVC should be enough. The inclusion
of the header in context-priv.c does not seem to be needed (tested
on GNU/Linux) and so it is removed.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agobuild: Skip building some tests on MSVC for now
Adrian Perez de Castro [Mon, 5 Aug 2019 10:51:09 +0000 (13:51 +0300)]
build: Skip building some tests on MSVC for now

This is a stopgap measure to quickly get tests building with MSVC for
now, at some point the tests could be rewritten to avoid using getopt()
and mkdtemp() or to ship an implementation.

4 years agobuild: add some defines for MSVC to allow it to be unixy
Adrian Perez de Castro [Mon, 5 Aug 2019 10:37:23 +0000 (13:37 +0300)]
build: add some defines for MSVC to allow it to be unixy

[ran: combined some commits]

4 years agoci: use consistent python for all jobs
Ran Benita [Sat, 28 Dec 2019 14:05:04 +0000 (16:05 +0200)]
ci: use consistent python for all jobs

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest/data: ensure files are checked out with LF, not CRLF
Ran Benita [Sat, 28 Dec 2019 13:52:20 +0000 (15:52 +0200)]
test/data: ensure files are checked out with LF, not CRLF

The tests stringcomp and buffercomp do binary comparison on some files;
if the files are changed to CRLF on checkout, the tests fail.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoCI: some tweaks
Adrian Perez de Castro [Mon, 12 Aug 2019 20:14:25 +0000 (23:14 +0300)]
CI: some tweaks

[ran: extracted generic bits from Windows commit.]

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoxkbcomp/rules: support \r\n line endings
Ran Benita [Sat, 28 Dec 2019 12:19:22 +0000 (14:19 +0200)]
xkbcomp/rules: support \r\n line endings

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest/atom: use correct format specifier for size_t
Ran Benita [Sat, 28 Dec 2019 12:11:27 +0000 (14:11 +0200)]
test/atom: use correct format specifier for size_t

From MSVC:

test\atom.c(98): note: consider using '%zu' in the format string
test\atom.c(98): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(100): note: consider using '%zu' in the format string
test\atom.c(100): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(114): note: consider using '%zu' in the format string
test\atom.c(114): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(128): note: consider using '%zu' in the format string
test\atom.c(128): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(130): note: consider using '%zu' in the format string
test\atom.c(130): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 1 has type 'size_t'
test\atom.c(137): note: consider using '%zu' in the format string
test\atom.c(137): warning C4477: 'fprintf' : format string '%lu' requires an argument of type 'unsigned long', but variadic argument 2 has type 'size_t'

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoOpen files in binary mode
Ran Benita [Sat, 28 Dec 2019 11:49:40 +0000 (13:49 +0200)]
Open files in binary mode

This turns off some misfeatures on Windows, and does nothing on POSIX.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest/common: avoid double // in path
Ran Benita [Sat, 28 Dec 2019 11:40:38 +0000 (13:40 +0200)]
test/common: avoid double // in path

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest/common: simplify test_get_path()
Ran Benita [Sat, 28 Dec 2019 11:32:02 +0000 (13:32 +0200)]
test/common: simplify test_get_path()

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoxkbcomp: remove cast which triggers warning on gcc
Ran Benita [Fri, 27 Dec 2019 20:08:57 +0000 (22:08 +0200)]
xkbcomp: remove cast which triggers warning on gcc

Will need some other way to take care of the warning on MSVC.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoxkbcomp: make a couple of casts explicit to mark them as checked
Ran Benita [Fri, 27 Dec 2019 19:51:34 +0000 (21:51 +0200)]
xkbcomp: make a couple of casts explicit to mark them as checked

This acknowledges some "possible loss of data cast" warnings from MSVC.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest/context: use a more portable directory-exists check
Ran Benita [Fri, 27 Dec 2019 13:47:15 +0000 (15:47 +0200)]
test/context: use a more portable directory-exists check

MSVC doesn't have opendir/closedir.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoxkbcomp/keywords: regenerate with newer gperf
Ran Benita [Fri, 27 Dec 2019 13:10:10 +0000 (15:10 +0200)]
xkbcomp/keywords: regenerate with newer gperf

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoparser: fix format string for int64_t
Ran Benita [Fri, 27 Dec 2019 12:51:31 +0000 (14:51 +0200)]
parser: fix format string for int64_t

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoOnly add GCC diagnostic pragmas when compiler is GCC compatible
Ran Benita [Fri, 27 Dec 2019 13:03:10 +0000 (15:03 +0200)]
Only add GCC diagnostic pragmas when compiler is GCC compatible

Avoid "unknown pragma" warnings on other compilers.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoxkbcomp: fix wrong return type
Ran Benita [Fri, 27 Dec 2019 12:52:49 +0000 (14:52 +0200)]
xkbcomp: fix wrong return type

Detected by MSVC:

xkbcomp\xkbcomp.c(111): warning C4047: 'return': 'bool' differs in levels of indirection from 'void *'

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoparser: use int64_t for all numbers
Ran Benita [Fri, 27 Dec 2019 12:17:55 +0000 (14:17 +0200)]
parser: use int64_t for all numbers

Don't use int which can have different size on different machines.
Also avoid some warnings from MSVC:

xkbcomp/parser.y(760): warning C4244: '=': conversion from 'int64_t' to 'int', possible loss of data
xkbcomp/parser.y(761): warning C4244: '=': conversion from 'int64_t' to 'int', possible loss of data
xkbcomp/parser.y(767): warning C4244: '=': conversion from 'int64_t' to 'int', possible loss of data

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoscanner-utils: avoid possible implicit truncating of line/column
Ran Benita [Fri, 27 Dec 2019 12:06:47 +0000 (14:06 +0200)]
scanner-utils: avoid possible implicit truncating of line/column

This increases the size of the struct a bit but it's not very important.

Fixes these MSVC warnings:

src\scanner-utils.h(112): warning C4267: '+=': conversion from 'size_t' to 'unsigned int', possible loss of data
src\scanner-utils.h(147): warning C4267: '+=': conversion from 'size_t' to 'unsigned int', possible loss of data

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoutils: move macro defines to before they're used
Ran Benita [Fri, 27 Dec 2019 11:55:24 +0000 (13:55 +0200)]
utils: move macro defines to before they're used

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoutils: fix typo in strndup fallback
Ran Benita [Fri, 27 Dec 2019 11:52:51 +0000 (13:52 +0200)]
utils: fix typo in strndup fallback

Fixup 93a1305 - we will have CI for this soon.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agomeson.build: Take win_bison as a possible variant for Bison
Adrian Perez de Castro [Mon, 12 Aug 2019 22:17:51 +0000 (01:17 +0300)]
meson.build: Take win_bison as a possible variant for Bison

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agobuild: include config.h manually
Ran Benita [Fri, 27 Dec 2019 11:03:20 +0000 (13:03 +0200)]
build: include config.h manually

Previously we included it with an `-include` compiler directive. But
that's not portable. And it's better to be explicit anyway.

Every .c file should have `include "config.h"` first thing.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoProvide a fallback implementation of [v]asprintf()
Adrian Perez de Castro [Mon, 5 Aug 2019 13:18:05 +0000 (16:18 +0300)]
Provide a fallback implementation of [v]asprintf()

Some environments (e.g. Windows + MSVC) do not provide asprintf() or
vasprintf(). This tries to detect their presence, and provides suitable
fallback implementations when not available.

4 years agoProvide a fallback implementation of strndup()
Adrian Perez de Castro [Mon, 5 Aug 2019 13:07:57 +0000 (16:07 +0300)]
Provide a fallback implementation of strndup()

Some environments (e.g. Windows + MSVC) do not provide strndup(), this
tries to detect its presence and provide a fallback implementation when
not available.

[ran: some tweaks]

4 years agoUse built-in istr[n]cmp() instead of strcase[n]cmp()
Adrian Perez de Castro [Mon, 5 Aug 2019 12:59:20 +0000 (15:59 +0300)]
Use built-in istr[n]cmp() instead of strcase[n]cmp()

This avoids the problem that MSVC does not provide strcasecmp() nor
strncasecmp(), and at the same time avoids potential problems due to
locale configuration by using istrcmp() and istrncmp() which are
already in the source tree and written to cover only ASCII.

4 years agoutils: use MIN/MAX instead of min/max
Ran Benita [Fri, 27 Dec 2019 10:34:49 +0000 (12:34 +0200)]
utils: use MIN/MAX instead of min/max

min/max symbols conflict on some systems (msvc), so just use the macros.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoxkbcomp: downgrade "Symbol added to modifier map for multiple modifiers" log to a...
Ran Benita [Fri, 27 Dec 2019 10:22:37 +0000 (12:22 +0200)]
xkbcomp: downgrade "Symbol added to modifier map for multiple modifiers" log to a warning

This condition happens in xkeyboard-config keymaps and seems hard to
fix. Currently it incessantly spams people's logs who have no idea what
to do about it. So downgrade to "warning" level, so it doesn't show up
by default.

When working on keymaps, set `XKB_LOG_LEVEL=debug XKB_LOG_VERBOSITY=10`
to see all possible messages.

Refs https://github.com/xkbcommon/libxkbcommon/issues/111
Fixes https://github.com/xkbcommon/libxkbcommon/issues/128
Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agorules: eliminate an extra fopen/fclose cycle
Peter Hutterer [Wed, 13 Nov 2019 01:38:39 +0000 (11:38 +1000)]
rules: eliminate an extra fopen/fclose cycle

FindXkbFileInPath() opens the file so we're guaranteed that the file not only
exists, but that we can read it. Changing that would alter behavior so instead
let's just pass that file handle along and do the same for include files.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: add include statements to rules files
Peter Hutterer [Tue, 3 Sep 2019 01:23:14 +0000 (11:23 +1000)]
rules: add include statements to rules files

The majority use-case for extending XKB on a machine is to override one or a
few keys with custom keycodes, not to define whole layouts.

Previously, we relied on the rules file to be a single file, making it hard to
extend. libxkbcommon parses $XDG_CONFIG_HOME/xkb/ but that only works as long
as there is a rule that matches the user-specified RMLVO. This works for MLV
but not for options which don't have a wildcard defined. Users have to copy
the whole rules file and then work from there - not something easy to extend
and maintain.

This patch adds a new ! include directive to rules files that allows including
another file. The file path must be without quotes and may not start with the
literal "include". Two directives are supported, %H to $HOME and %S for the
system-installed rules directory (usually /usr/share/X11/xkb/rules).

A user would typically use a custom rules file like this:

! option                =       symbols
  custom:foo            =       +custom(foo)
  custom:bar            =       +custom(baz)

! include %S/evdev

Where the above defines the two options and then includes the system-installed
evdev rule. Since most current implementations default to loading the "evdev"
ruleset, it's best to name this $XDG_CONFIG_HOME/xkb/rules/evdev, but any
valid name is allowed.

The include functionally replaces the line with the content of the included
file which means the behavior of rules files is maintained. Specifically,
custom options must be defined before including another file because the first
match usually wins. In other words, the following ruleset will not assign
my_model as one would expect:

! include %S/evdev

! model                 =  symbols
  my_model              =  +custom(foo)

The default evdev ruleset has wildcards for model and those match before the
my_model is hit.

The actual resolved components need only be in one of the XKB lookup
directories, e.g. for the example above:

$ cat $XDG_CONFIG_HOME/xkb/symbols/custom
partial alphanumeric_keys
xkb_symbols "foo" {
    key <TLDE> {        [      VoidSymbol ]       };
};

partial alphanumeric_keys
xkb_symbols "baz" {
    key <AB01> {        [      k, K ]       };
};

This can then be loaded with the XKB option "custom:foo,custom:bar".

The use of "custom" is just as an example, there are no naming requirements
beyond avoiding already-used ones. Also note the bar/baz above - the option
names don't have to match the component names.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agocontext: add a helper function to return the default system include path
Peter Hutterer [Mon, 30 Sep 2019 05:39:08 +0000 (15:39 +1000)]
context: add a helper function to return the default system include path

No functional changes but we'll need that same lookup in the rules file
include handling in a future patch.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: move the matcher result handling to the caller
Peter Hutterer [Tue, 12 Nov 2019 05:27:22 +0000 (15:27 +1000)]
rules: move the matcher result handling to the caller

This shouldn't be processed in the matcher itself, especially in the glorious
future when we can have nested matchers. Only handle this once in the caller
to the original parsed file.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: put the scanner on the stack
Peter Hutterer [Tue, 12 Nov 2019 04:19:55 +0000 (14:19 +1000)]
rules: put the scanner on the stack

This allows nesting the scanner for the future !include directive.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: simplify an error path
Peter Hutterer [Tue, 12 Nov 2019 05:25:16 +0000 (15:25 +1000)]
rules: simplify an error path

Initialize to NULL so we don't have to care about whether the cleanups can be
called or not.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: rename a variable from 's' to 'str'
Peter Hutterer [Tue, 12 Nov 2019 04:39:47 +0000 (14:39 +1000)]
rules: rename a variable from 's' to 'str'

To avoid name conflicts with a future patch.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: drop the matcher_err() macro and use scanner_err directly
Peter Hutterer [Tue, 12 Nov 2019 04:17:44 +0000 (14:17 +1000)]
rules: drop the matcher_err() macro and use scanner_err directly

No functional changes, this is what the macro expanded to anyway. Prep work
for putting the scanner on the stack and removing it from the matcher struct.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agorules: factor out the function to parse a rules file
Peter Hutterer [Tue, 3 Sep 2019 00:56:01 +0000 (10:56 +1000)]
rules: factor out the function to parse a rules file

No functional changes, this just makes the part to parse a single rules file
re-usable.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agokeysym: handle ssharp in XConvertCase()
Peter Hutterer [Fri, 13 Dec 2019 03:35:25 +0000 (13:35 +1000)]
keysym: handle ssharp in XConvertCase()

lowercase: LATIN SMALL LETTER SHARP S (U+00DF)
uppercase: LATIN CAPITAL LETTER SHARP S (U+1E9E)

The uppercase sharp s (XK_ssharp) is a relatively recent addition to unicode
but was added to the relevant keyboard layouts in xkeyboard-config-2.25
(d1411e5e95c)
https://gitlab.freedesktop.org/xkeyboard-config/xkeyboard-config/issues/144

Alas, the CapsLock behavior was broken on the finnish layout (maybe others).
This was due XConvertCase() never returning the uppercase characters.

Let's make this function return the right lower/upper symbols for the sharp s
and hope that the world won't get any worse because of it.

Corresponding Xlib issue:
https://gitlab.freedesktop.org/xorg/lib/libx11/issues/110

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agomeson: remove redundant malloc scribbling
Ran Benita [Sat, 14 Dec 2019 11:45:35 +0000 (13:45 +0200)]
meson: remove redundant malloc scribbling

Turns out meson already sets this (at least MALLOC_PERTURB) on its own
for the `test` target.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoparser: fix the remaining pointer chasing
Ran Benita [Sat, 14 Dec 2019 11:44:33 +0000 (13:44 +0200)]
parser: fix the remaining pointer chasing

Fix the TODO added in 7c42945.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest: let rmlvo-to-kccgst take long options like rmlvo-to-keymap
Peter Hutterer [Wed, 13 Nov 2019 03:42:11 +0000 (13:42 +1000)]
test: let rmlvo-to-kccgst take long options like rmlvo-to-keymap

The short options were left for backwards compatibility.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agoparser: fix quadratic pointer chasing
Ran Benita [Wed, 13 Nov 2019 20:41:38 +0000 (22:41 +0200)]
parser: fix quadratic pointer chasing

In the AST, lists (e.g. the list of statements in a file) are kept in
singly-linked lists -- each AST node has a `next` pointer available for
this purpose.

Previously, a node was added to the list by starting from the head,
chasing to the last, and appending. So creating a list of length N would
take ~N^2/2 pointer dereferences.

Now, we always (temporarily) keep the last as well, so appending is O(1)
instead of O(N).

Given a keymap

    xkb_keymap {
    xkb_keycodes {
    minimum = 8;
    minimum = 8;
    minimum = 8;
    minimum = 8;
    minimum = 8;
    [... repeated N times ...]
    };
    xkb_types {};
    xkb_compat {};
    xkb_symbols {};
    };

The compilation times are

N       | Before   | After
--------|----------|-------
10,000  | 0.407s   | 0.006s
20,000  | 1.851s   | 0.015s
30,000  | 5.737s   | 0.021s
40,000  | 12.759s  | 0.023s
50,000  | 21.489s  | 0.035s
60,000  | 40.473s  | 0.041s
70,000  | 53.336s  | 0.039s
80,000  | 72.485s  | 0.044s
90,000  | 94.703s  | 0.048s
100,000 | 118.390s | 0.057s

Another option is to ditch the linked lists and use arrays instead. I
got it to work, but its more involved and allocation heavy so turns out
to be worse without further optimizations.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoparser: remove an unneeded check
Ran Benita [Wed, 13 Nov 2019 21:37:47 +0000 (23:37 +0200)]
parser: remove an unneeded check

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agodocs: update the include path documentation
Peter Hutterer [Wed, 13 Nov 2019 23:52:21 +0000 (09:52 +1000)]
docs: update the include path documentation

Missing from e23f1061b2b9c8d40ba7797a5ffbd6d474d604e0 and
3a91788d9254b4571ff883439931f497d8a0663f.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agocompat: reject interpret modifier predicate with more than one value
Ran Benita [Tue, 12 Nov 2019 20:31:46 +0000 (22:31 +0200)]
compat: reject interpret modifier predicate with more than one value

Given

    interpret ISO_Level3_Shift+AnyOf(all,extraneous) { ... };

Previously, extraneous (and further) was ignored. Now it's rejected.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoexpr: fix log message on some unexpected expression types
Ran Benita [Tue, 12 Nov 2019 20:09:19 +0000 (22:09 +0200)]
expr: fix log message on some unexpected expression types

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoReplace some tabs that sneaked in with spaces
Ran Benita [Tue, 12 Nov 2019 20:06:02 +0000 (22:06 +0200)]
Replace some tabs that sneaked in with spaces

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoparser: fix merge mode only applied to first vmod in a virtual_modifiers statement
Ran Benita [Tue, 12 Nov 2019 18:34:31 +0000 (20:34 +0200)]
parser: fix merge mode only applied to first vmod in a virtual_modifiers statement

Given

    augment virtual_modifiers NumLock,Alt,LevelThree

Previously it was expanded (directly in the parser) to

    augment virtual_modifiers NumLock;
    virtual_modifiers Alt;
    virtual_modifiers LevelThree;

Now it expands to

    augment virtual_modifiers NumLock;
    augment virtual_modifiers Alt;
    augment virtual_modifiers LevelThree;

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoast: use a separate expr struct for action list
Ran Benita [Tue, 12 Nov 2019 18:04:13 +0000 (20:04 +0200)]
ast: use a separate expr struct for action list

Currently it's under UnaryExpr, which just doesn't make sense.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoast-build: get rid of unhelpful macro
Ran Benita [Tue, 12 Nov 2019 17:16:08 +0000 (19:16 +0200)]
ast-build: get rid of unhelpful macro

Straightforward code is better here.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: a string is greater than its prefix
Ran Benita [Sat, 9 Nov 2019 20:07:15 +0000 (22:07 +0200)]
atom: a string is greater than its prefix

Bug accidentally introduced in 9a92b46.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: combine atom_intern() and atom_lookup()
Ran Benita [Sat, 9 Nov 2019 19:25:01 +0000 (21:25 +0200)]
atom: combine atom_intern() and atom_lookup()

Use an "add" bool parameter instead. This simplifies the code a bit.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: correct iteration count in hash function
Ran Benita [Sat, 9 Nov 2019 11:47:16 +0000 (13:47 +0200)]
atom: correct iteration count in hash function

Fixup of ccab349 - unlike the commit message, hash a byte twice instead
of zero times, which is probably better. This is how it was before.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: describe how this odd data structure works
Ran Benita [Sat, 9 Nov 2019 11:12:02 +0000 (13:12 +0200)]
atom: describe how this odd data structure works

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: use a better hash function
Ran Benita [Sat, 9 Nov 2019 10:43:04 +0000 (12:43 +0200)]
atom: use a better hash function

FNV-1a instead of the djb2-like one from before.

Keep the unrolling since it seems quite beneficial, even though it loses
one byte if the length is odd...

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: style changes
Ran Benita [Sat, 9 Nov 2019 09:49:25 +0000 (11:49 +0200)]
atom: style changes

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: remove handling of garbage input
Ran Benita [Sat, 9 Nov 2019 09:39:17 +0000 (11:39 +0200)]
atom: remove handling of garbage input

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: use explicit size for fingerprint
Ran Benita [Sat, 9 Nov 2019 09:33:45 +0000 (11:33 +0200)]
atom: use explicit size for fingerprint

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: replace an avoidable strlen
Ran Benita [Fri, 8 Nov 2019 22:20:45 +0000 (00:20 +0200)]
atom: replace an avoidable strlen

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoatom: remove redundant field
Ran Benita [Fri, 8 Nov 2019 22:05:59 +0000 (00:05 +0200)]
atom: remove redundant field

The field is redundant.

Due to alignment, this will only save memory on 32bit architectures.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest/atom: increase iteration count and print random seed on failure
Ran Benita [Fri, 8 Nov 2019 20:40:13 +0000 (22:40 +0200)]
test/atom: increase iteration count and print random seed on failure

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agoparser: get rid of "stealing" atoms
Ran Benita [Sat, 2 Nov 2019 11:31:44 +0000 (13:31 +0200)]
parser: get rid of "stealing" atoms

This requires (well, at least implemented by) casting away `const` which
is undefined behavior, and clang started to warn about it.

The micro optimization didn't save too many allocations, anyway.

Signed-off-by: Ran Benita <ran@unusedvar.com>
4 years agotest: remove a superfluous string-is-null check
Peter Hutterer [Tue, 5 Nov 2019 03:33:11 +0000 (13:33 +1000)]
test: remove a superfluous string-is-null check

A few lines above we check path_rel[0], so any null pointer will blow up
before we get here.

Found by coverity

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
4 years agotest: fix a potential memory leak
Peter Hutterer [Tue, 5 Nov 2019 03:22:49 +0000 (13:22 +1000)]
test: fix a potential memory leak

Found by coverity

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: bring back the progress bar
Ran Benita [Fri, 1 Nov 2019 08:45:43 +0000 (10:45 +0200)]
test: xkeyboard-config: bring back the progress bar

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agotest: xkeyboard-config: invoke the python3 command (#120)
Peter Hutterer [Fri, 1 Nov 2019 08:41:16 +0000 (18:41 +1000)]
test: xkeyboard-config: invoke the python3 command (#120)

python3 is always python3, but python could be python2 in some cases. Or just
missing (e.g. RHEL8).

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: add missing variant tests
Peter Hutterer [Fri, 1 Nov 2019 01:09:16 +0000 (11:09 +1000)]
test: xkeyboard-config: add missing variant tests

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: use universal_newlines instead of decode
Peter Hutterer [Thu, 31 Oct 2019 23:54:29 +0000 (09:54 +1000)]
test: xkeyboard-config: use universal_newlines instead of decode

This way stdin/stdout of the process are opened in text mode and we don't need
manually decode.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: flake8 fixes
Peter Hutterer [Wed, 30 Oct 2019 02:03:48 +0000 (12:03 +1000)]
test: xkeyboard-config: flake8 fixes

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: handle keyboard interrupts correctly
Peter Hutterer [Wed, 30 Oct 2019 01:22:49 +0000 (11:22 +1000)]
test: xkeyboard-config: handle keyboard interrupts correctly

In python multiprocessing, each process needs to handle (and ignore) the
KeyboardInterrupt to avoid exception logging. This is a separate patch for
easier reviewing, the first hunks merely re-indent all of the
xkbcommontool/xkbcomp functions into a try/except KeyboardInterrupt block.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: print to stderr on failure, stdout otherwise
Peter Hutterer [Wed, 30 Oct 2019 00:53:58 +0000 (10:53 +1000)]
test: xkeyboard-config: print to stderr on failure, stdout otherwise

This is a change in behavior and requires any automated callers to adjust
accordingly. Still, much easier to get the errors that way rather than it
being mixed into a thousands-of-lines output file.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: add a multiprocessing.Pool() to speed up the test
Peter Hutterer [Tue, 29 Oct 2019 06:06:10 +0000 (16:06 +1000)]
test: xkeyboard-config: add a multiprocessing.Pool() to speed up the test

Collect all options into a dictionary, then process that as async actions
through a process pool. This of course requires collecting the various print
statements to avoid mangled output.

This dropped the time to completion from around 14 min to 8 min on my local
machine (unscientific single run only for the original timing).

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agotest: xkeyboard-config: use argparse for the path and the tool selection
Peter Hutterer [Wed, 30 Oct 2019 01:15:49 +0000 (11:15 +1000)]
test: xkeyboard-config: use argparse for the path and the tool selection

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agoscripts/makeheader: slight simplification
Ran Benita [Thu, 31 Oct 2019 19:21:35 +0000 (21:21 +0200)]
scripts/makeheader: slight simplification

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agoscripts/makeheader: allow overriding the prefix path of the X11 headers
Sebastian Wick [Thu, 31 Oct 2019 16:04:49 +0000 (17:04 +0100)]
scripts/makeheader: allow overriding the prefix path of the X11 headers
with X11_HEADERS_PREFIX

Signed-off-by: Sebastian Wick <sebastian@sebastianwick.net>
5 years agotest: add test for the various default include paths
Peter Hutterer [Mon, 28 Oct 2019 01:40:56 +0000 (11:40 +1000)]
test: add test for the various default include paths

All tests create a temporary directory, set up the environment for that
directory and then check the include paths for the presence of that directory,
ideally in the right position of the list.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agoUse XDG_CONFIG_HOME as first XKB search path
Peter Hutterer [Fri, 25 Oct 2019 04:36:16 +0000 (14:36 +1000)]
Use XDG_CONFIG_HOME as first XKB search path

Use $XDG_CONFIG_HOME/xkb as the primary lookup path for XKB rules. Same
motivation as in 3a91788d9254b, however the XDG directories are more standard
and recommended these days than application-specific dotfiles.

The XDG spec says to fall back to $HOME/.config where XDG_CONFIG_HOME is not
set so we implement that behavior as well.

Fixes #112

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
5 years agoBump version to 0.9.1 xkbcommon-0.9.1
Ran Benita [Sun, 20 Oct 2019 20:07:52 +0000 (23:07 +0300)]
Bump version to 0.9.1

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agocontext: Don't fail to create the context if HOME isn't available
Jan Alexander Steffens (heftig) [Sun, 20 Oct 2019 19:06:19 +0000 (19:06 +0000)]
context: Don't fail to create the context if HOME isn't available

E.g. when Mutter has CAP_SYS_NICE and thus secure_getenv returns NULL.

Fixes https://bugs.archlinux.org/task/64191

[ran: changed to ignore error]
Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agoREADME: update my email address
Ran Benita [Fri, 18 Oct 2019 21:55:21 +0000 (00:55 +0300)]
README: update my email address

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agoBump version to 0.9.0 xkbcommon-0.9.0
Ran Benita [Fri, 18 Oct 2019 21:33:09 +0000 (00:33 +0300)]
Bump version to 0.9.0

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agobuild: remove leftover autotools file
Ran Benita [Fri, 18 Oct 2019 21:41:16 +0000 (00:41 +0300)]
build: remove leftover autotools file

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agotest: fix printf("%s", NULL) in error path
Ran Benita [Fri, 18 Oct 2019 21:37:48 +0000 (00:37 +0300)]
test: fix printf("%s", NULL) in error path

    ../test/common.c: In function ‘test_get_path’:
    ../test/common.c:171:9: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
      171 |         fprintf(stderr, "Failed to allocate path (%d chars) for %s\n",
          |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      172 |                 (int) path_len, path);
          |                 ~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agokeymap-dump: use consistent capitalization for "Group<N>"
Ran Benita [Wed, 16 Oct 2019 07:32:19 +0000 (10:32 +0300)]
keymap-dump: use consistent capitalization for "Group<N>"

It's used capitalized everywhere except a couple places.

Signed-off-by: Ran Benita <ran@unusedvar.com>
5 years agokeymap-dump: fix invalid names used for levels above 8
Ran Benita [Wed, 16 Oct 2019 07:27:12 +0000 (10:27 +0300)]
keymap-dump: fix invalid names used for levels above 8

xkbcomp only accepts the "Level" prefix for a level name for levels 1 to
8, but the keymap dumping code added it always, e.g. "Level15".

The plain integer, e.g. "8", "15" is always accepted, so just use that.

Fixes https://github.com/xkbcommon/libxkbcommon/issues/113

Signed-off-by: Ran Benita <ran@unusedvar.com>
Reported-by: progandy
5 years agocontext: move ~/.xkb to before XKB_CONFIG_ROOT in the default include path
Ran Benita [Thu, 3 Oct 2019 14:27:00 +0000 (17:27 +0300)]
context: move ~/.xkb to before XKB_CONFIG_ROOT in the default include path

Previously, the default include path was XKB_CONFIG_ROOT:~/.xkb.

The ~/.xkb include path is intended to allow the local user to customize
their keymaps without having to modify system paths.

But usually, the user only wants to customize specific parts. When
XKB_CONFIG_ROOT is first, the user can only customize through the "entry
point" (the RMLVO). When ~/.xkb is first, the user can drop in a file
and it will override the system one.

The impetus for this change is the rules file. "evdev" is hard-coded
everywhere, so it not often not possible to change to something else.
And the rules files determines how the rest of the RMLVO is interpreted.
So, to enable customization, we have these options:

A: System includes user.
B: User includes system.
C: Library goes over both in one or the other order.

Option A is problematic due to backward compatibility and is also
unnatural.

Option B gives the user control and is backward compatible, so that's
what we choose. This is also how Compose files are handled, and that
seems to work fine in the wild.

Option C is actually less flexible than B, and more complicated.

(The rules file format doesn't have an include statement yet, but it's
planned).

Signed-off-by: Ran Benita <ran@unusedvar.com>