* glib/gpattern.c: add UTF-8 support.
* tests/patterntest.c: add UTF-8 and equality tests.
* docs/reference/glib/Makefile.am (MKDB_OPTIONS): Add --sgml-mode.
* docs/reference/glib/tmpl/patterns.sgml: Document UTF-8 support.
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/gpattern.c: add UTF-8 support.
+
+ * tests/patterntest.c: add UTF-8 and equality tests.
+
Wed Nov 14 07:34:24 2001 Tim Janik <timj@gtk.org>
* glib/galloca.h (g_newa): provide g_newa(ctype, count) on top of
+2001-11-14 Matthias Clasen <matthiasc@poet.de>
+
+ * glib/Makefile.am (MKDB_OPTIONS): Add --sgml-mode.
+
+ * glib/tmpl/patterns.sgml: Document UTF-8 support.
+
Wed Nov 14 03:19:49 2001 Tim Janik <timj@gtk.org>
- * gobject/tmp/param_value_types.sgml: more docs.
+ * gobject/tmpl/param_value_types.sgml: more docs.
Tue Nov 13 21:31:58 2001 Tim Janik <timj@gtk.org>
- * gobject/tmp/param_value_types.sgml: list parameter and
+ * gobject/tmpl/param_value_types.sgml: list parameter and
value types.
* gobject/tmpl/gparamspec.sgml: more docs for g_param_spec*()
SCAN_OPTIONS=--deprecated-guards="G_DISABLE_DEPRECATED"
# Extra options to supply to gtkdoc-mkdb
-MKDB_OPTIONS=
+MKDB_OPTIONS=--sgml-mode
# Extra options to supply to gtkdoc-fixref
FIXXREF_OPTIONS=
be escaped to include them literally in a pattern.
</para>
<para>
-The pattern matcher is restricted to ASCII and will not work correctly with
-multibyte UTF-8 characters in the pattern or in the string to match.
-</para>
-<para>
When multiple strings must be matched against the same pattern, it
is better to compile the pattern to a #GPatternSpec using
g_pattern_spec_new() and use g_pattern_match_string() instead of
g_pattern_match_simple(). This avoids the overhead of repeated
pattern compilation.
-</para>
-
-<!-- ##### SECTION See_Also ##### -->
-<para>
-g_strreverse()
-</para>
<!-- ##### STRUCT GPatternSpec ##### -->
<para>
A <structname>GPatternSpec</structname> is the 'compiled' form of a pattern.
-This structure is opaque and its fields and cannot be accessed.
+This structure is opaque and its fields cannot be accessed directly.
</para>
Compiles a pattern to a #GPatternSpec.
</para>
-@pattern: a zero terminated string.
+@pattern: a zero-terminated UTF-8 encoded string.
@Returns: a newly-allocated #GPatternSpec.
this is more efficient if the reversed version of the string to be matched is
not at hand, as g_pattern_match() will only construct it if the compiled pattern
requires reverse matches.
-Note that, if the user code will (possibly) match a string against a multitude of
-patterns containing wildcards, chances are high that some patterns will require
-a reversed string. In this case, it's more efficient to provide the reversed
-string to avoid multiple constructions thereof in the various calls to
+</para>
+<para>
+Note that, if the user code will (possibly) match a string against a multitude
+of patterns containing wildcards, chances are high that some patterns will
+require a reversed string. In this case, it's more efficient to provide the
+reversed string to avoid multiple constructions thereof in the various calls to
g_pattern_match().
</para>
+<para>
+Note also that the reverse of a UTF-8 encoded string can in general
+<emphasis>not</emphasis> be obtained by <function>g_strreverse()</function>.
+This works only if the string doesn't contain any multibyte characters.
+Glib doesn't currently offer a function to reverse UTF-8 encoded strings.
+</para>
@pspec: a #GPatternSpec.
@string_length: the length of @string.
-@string: the string to match.
+@string: the UTF-8 encoded string to match.
@string_reversed: the reverse of @string or %NULL.
@Returns: %TRUE if @string matches @pspec.
</para>
@pspec: a #GPatternSpec.
-@string: the string to match.
+@string: the UTF-8 encoded string to match.
@Returns: %TRUE if @string matches @pspec.
repetively.
</para>
-@pattern: the pattern.
-@string: the string to match.
+@pattern: the UTF-8 encoded pattern.
+@string: the UTF-8 encoded string to match.
@Returns: %TRUE if @string matches @pspec.
#include "gmacros.h"
#include "gmessages.h"
#include "gmem.h"
-#include "gutils.h" /* inline hassle */
+#include "gunicode.h"
+#include "gutils.h"
#include <string.h>
/* keep enum and structure of gpattern.c and patterntest.c in sync */
G_MATCH_EXACT, /* "AAAAA" */
G_MATCH_LAST
} GMatchType;
+
struct _GPatternSpec
{
GMatchType match_type;
guint pattern_length;
+ guint min_length;
gchar *pattern;
};
/* --- functions --- */
-static inline void
-instring_reverse (guint length,
- gchar *str)
+/**
+ * g_utf8_reverse:
+ * string: a UTF-8 string.
+ *
+ * Reverses a UTF-8 string. The @string must be valid UTF-8 encoded text.
+ * (Use g_utf8_validate() on all text before trying to use UTF-8
+ * utility functions with it.)
+ *
+ * Note that unlike g_strreverse(), this function returns
+ * newly-allocated memory, which should be freed with g_free() when
+ * no longer needed.
+ *
+ * Returns: a newly-allocated string which is the reverse of @string.
+ */
+static gchar *
+g_utf8_reverse (guint len, const gchar *string)
{
- gchar *f, *l, *b;
+ gchar *result;
+ const gchar *p;
+ gchar *m, *r, skip;
- f = str;
- l = str + length - 1;
- b = str + length / 2;
- while (f < b)
+ result = g_new (gchar, len + 1);
+ r = result + len;
+ p = string;
+ while (*p)
{
- gchar tmp = *l;
-
- *l-- = *f;
- *f++ = tmp;
+ skip = g_utf8_skip[*(guchar*)p];
+ r -= skip;
+ for (m = r; skip; skip--)
+ *m++ = *p++;
}
-}
-
-static inline gchar*
-strdup_reverse (guint length,
- const gchar *str)
-{
- gchar *t, *dest = g_new (gchar, length + 1);
+ result[len] = 0;
- t = dest + length;
- *t-- = 0;
- while (t >= dest)
- *t-- = *str++;
- return dest;
+ return result;
}
static inline gboolean
case '?':
if (!*string)
return FALSE;
- string++;
+ string = g_utf8_next_char (string);
break;
case '*':
{
if (!*string)
return FALSE;
- string++;
+ string = g_utf8_next_char (string);
}
}
while (ch == '*' || ch == '?');
{
if (!*string)
return FALSE;
- string++;
+ string = g_utf8_next_char (string);
}
string++;
if (g_pattern_ph_match (pattern, string))
g_return_val_if_fail (pspec != NULL, FALSE);
g_return_val_if_fail (string != NULL, FALSE);
+ if (pspec->min_length > string_length)
+ return FALSE;
+
switch (pspec->match_type)
{
- gboolean result;
- gchar *tmp;
case G_MATCH_ALL:
return g_pattern_ph_match (pspec->pattern, string);
case G_MATCH_ALL_TAIL:
return g_pattern_ph_match (pspec->pattern, string_reversed);
else
{
- tmp = strdup_reverse (string_length, string);
+ gboolean result;
+ gchar *tmp;
+ tmp = g_utf8_reverse (string_length, string);
result = g_pattern_ph_match (pspec->pattern, tmp);
g_free (tmp);
return result;
}
case G_MATCH_HEAD:
- if (pspec->pattern_length > string_length)
- return FALSE;
- else if (pspec->pattern_length == string_length)
+ if (pspec->pattern_length == string_length)
return strcmp (pspec->pattern, string) == 0;
else if (pspec->pattern_length)
return strncmp (pspec->pattern, string, pspec->pattern_length) == 0;
else
return TRUE;
case G_MATCH_TAIL:
- if (pspec->pattern_length > string_length)
- return FALSE;
- else if (pspec->pattern_length == string_length)
- {
- if (string_reversed)
- return strcmp (pspec->pattern, string_reversed) == 0;
- else
- {
- tmp = strdup_reverse (string_length, string);
- result = strcmp (pspec->pattern, tmp) == 0;
- g_free (tmp);
- return result;
- }
- }
- else if (pspec->pattern_length)
- {
- if (string_reversed)
- return strncmp (pspec->pattern, string_reversed, pspec->pattern_length) == 0;
- else
- {
- tmp = strdup_reverse (string_length, string);
- result = strncmp (pspec->pattern, tmp, pspec->pattern_length) == 0;
- g_free (tmp);
- return result;
- }
- }
+ if (pspec->pattern_length)
+ return strcmp (pspec->pattern, string + (string_length - pspec->pattern_length)) == 0;
else
return TRUE;
case G_MATCH_EXACT:
gboolean seen_joker = FALSE, seen_wildcard = FALSE, more_wildcards = FALSE;
gint hw_pos = -1, tw_pos = -1, hj_pos = -1, tj_pos = -1;
gboolean follows_wildcard = FALSE;
+ guint pending_jokers = 0;
const gchar *s;
gchar *d;
guint i;
/* canonicalize pattern and collect necessary stats */
pspec = g_new (GPatternSpec, 1);
pspec->pattern_length = strlen (pattern);
+ pspec->min_length = 0;
pspec->pattern = g_new (gchar, pspec->pattern_length + 1);
d = pspec->pattern;
for (i = 0, s = pattern; *s != 0; s++)
tw_pos = i;
break;
case '?':
- if (hj_pos < 0)
- hj_pos = i;
- tj_pos = i;
- /* fall through */
+ pending_jokers++;
+ pspec->min_length++;
+ continue;
default:
+ for (; pending_jokers; pending_jokers--, i++) {
+ *d++ = '?';
+ if (hj_pos < 0)
+ hj_pos = i;
+ tj_pos = i;
+ }
follows_wildcard = FALSE;
+ pspec->min_length++;
break;
}
*d++ = *s;
i++;
}
+ for (; pending_jokers; pending_jokers--) {
+ *d++ = '?';
+ if (hj_pos < 0)
+ hj_pos = i;
+ tj_pos = i;
+ }
*d++ = 0;
seen_joker = hj_pos >= 0;
seen_wildcard = hw_pos >= 0;
if (pspec->pattern[0] == '*')
{
pspec->match_type = G_MATCH_TAIL;
- instring_reverse (pspec->pattern_length, pspec->pattern);
- pspec->pattern[--pspec->pattern_length] = 0;
+ memmove (pspec->pattern, pspec->pattern + 1, --pspec->pattern_length);
+ pspec->pattern[pspec->pattern_length] = 0;
return pspec;
}
if (pspec->pattern[pspec->pattern_length - 1] == '*')
pspec->match_type = tw_pos > hw_pos ? G_MATCH_ALL_TAIL : G_MATCH_ALL;
else /* seen_joker */
pspec->match_type = tj_pos > hj_pos ? G_MATCH_ALL_TAIL : G_MATCH_ALL;
- if (pspec->match_type == G_MATCH_ALL_TAIL)
- instring_reverse (pspec->pattern_length, pspec->pattern);
+ if (pspec->match_type == G_MATCH_ALL_TAIL) {
+ gchar *tmp = pspec->pattern;
+ pspec->pattern = g_utf8_reverse (pspec->pattern_length, pspec->pattern);
+ g_free (tmp);
+ }
return pspec;
}
* Free Software Foundation, Inc., 59 Temple Place - Suite 330,
* Boston, MA 02111-1307, USA.
*/
-
-#include <string.h>
-
#include "glib.h"
+#include "glib/gpattern.h"
+
/* keep enum and structure of gpattern.c and patterntest.c in sync */
typedef enum
G_MATCH_EXACT, /* "AAAAA" */
G_MATCH_LAST
} GMatchType;
+
struct _GPatternSpec
{
GMatchType match_type;
guint pattern_length;
+ guint min_length;
gchar *pattern;
};
}
}
+/* this leakes memory, but we don't care */
+
+#define utf8(str) g_convert (str, -1, "Latin1", "UTF-8", NULL, NULL, NULL)
+#define latin1(str) g_convert (str, -1, "UTF-8", "Latin1", NULL, NULL, NULL)
+
static gboolean
test_compilation (gchar *src,
GMatchType match_type,
- gchar *pattern)
+ gchar *pattern,
+ guint min)
{
GPatternSpec *spec;
- g_print ("compiling \"%s\" \t", src);
+ g_print ("compiling \"%s\" \t", utf8(src));
spec = g_pattern_spec_new (src);
-
+
if (spec->match_type != match_type)
{
g_print ("failed \t(match_type: %s, expected %s)\n",
if (strcmp (spec->pattern, pattern) != 0)
{
g_print ("failed \t(pattern: \"%s\", expected \"%s\")\n",
- spec->pattern,
- pattern);
+ utf8(spec->pattern),
+ utf8(pattern));
return FALSE;
}
return FALSE;
}
+ if (spec->min_length != min)
+ {
+ g_print ("failed \t(min_length: %d, expected %d)\n",
+ spec->min_length,
+ min);
+ return FALSE;
+ }
+
g_print ("passed (%s: \"%s\")\n",
match_type_name (spec->match_type),
spec->pattern);
gchar *string,
gboolean match)
{
- g_print ("matching \"%s\" against \"%s\" \t", string, pattern);
+ g_print ("matching \"%s\" against \"%s\" \t", utf8(string), utf8(pattern));
if (g_pattern_match_simple (pattern, string) != match)
{
GPatternSpec *p2 = g_pattern_spec_new (pattern2);
gboolean equal = g_pattern_spec_equal (p1, p2);
- g_print ("comparing \"%s\" with \"%s\" \t", pattern1, pattern2);
+ g_print ("comparing \"%s\" with \"%s\" \t", utf8(pattern1), utf8(pattern2));
if (expected != equal)
{
g_print ("failed \t{%s, %u, \"%s\"} %s {%s, %u, \"%s\"}\n",
- match_type_name (p1->match_type), p1->pattern_length, p1->pattern,
+ match_type_name (p1->match_type), p1->pattern_length, utf8(p1->pattern),
expected ? "!=" : "==",
- match_type_name (p2->match_type), p2->pattern_length, p2->pattern);
+ match_type_name (p2->match_type), p2->pattern_length, utf8(p2->pattern));
}
else
g_print ("passed (%s)\n", equal ? "equal" : "unequal");
return expected == equal;
}
-#define TEST_COMPILATION(src, type, pattern) { \
+#define TEST_COMPILATION(src, type, pattern, min) { \
total++; \
- if (test_compilation (src, type, pattern)) \
+ if (test_compilation (latin1(src), type, latin1(pattern), min)) \
passed++; \
else \
failed++; \
#define TEST_MATCH(pattern, string, match) { \
total++; \
- if (test_match (pattern, string, match)) \
+ if (test_match (latin1(pattern), latin1(string), match)) \
passed++; \
else \
failed++; \
#define TEST_EQUAL(pattern1, pattern2, match) { \
total++; \
- if (test_equal (pattern1, pattern2, match)) \
+ if (test_equal (latin1(pattern1), latin1(pattern2), match)) \
passed++; \
else \
failed++; \
gint total = 0;
gint passed = 0;
gint failed = 0;
- gchar *string;
+ gchar *string, *pattern;
- TEST_COMPILATION("*A?B*", G_MATCH_ALL, "*A?B*");
- TEST_COMPILATION("ABC*DEFGH", G_MATCH_ALL_TAIL, "HGFED*CBA");
- TEST_COMPILATION("ABCDEF*GH", G_MATCH_ALL, "ABCDEF*GH");
- TEST_COMPILATION("ABC**?***??**DEF*GH", G_MATCH_ALL, "ABC*?*??*DEF*GH");
- TEST_COMPILATION("*A?AA", G_MATCH_ALL_TAIL, "AA?A*");
- TEST_COMPILATION("ABCD*", G_MATCH_HEAD, "ABCD");
- TEST_COMPILATION("*ABCD", G_MATCH_TAIL, "DCBA");
- TEST_COMPILATION("ABCDE", G_MATCH_EXACT, "ABCDE");
+ TEST_COMPILATION("*A?B*", G_MATCH_ALL, "*A?B*", 3);
+ TEST_COMPILATION("ABC*DEFGH", G_MATCH_ALL_TAIL, "HGFED*CBA", 8);
+ TEST_COMPILATION("ABCDEF*GH", G_MATCH_ALL, "ABCDEF*GH", 8);
+ TEST_COMPILATION("ABC**?***??**DEF*GH", G_MATCH_ALL, "ABC*???DEF*GH", 11);
+ TEST_COMPILATION("*A?AA", G_MATCH_ALL_TAIL, "AA?A*", 4);
+ TEST_COMPILATION("ABCD*", G_MATCH_HEAD, "ABCD", 4);
+ TEST_COMPILATION("*ABCD", G_MATCH_TAIL, "ABCD", 4);
+ TEST_COMPILATION("ABCDE", G_MATCH_EXACT, "ABCDE", 5);
+ TEST_COMPILATION("A?C?E", G_MATCH_ALL, "A?C?E", 5);
+ TEST_COMPILATION("*?x", G_MATCH_ALL_TAIL, "x?*", 2);
+ TEST_COMPILATION("?*x", G_MATCH_ALL_TAIL, "x?*", 2);
+ TEST_COMPILATION("*?*x", G_MATCH_ALL_TAIL, "x?*", 2);
+ TEST_COMPILATION("x*??", G_MATCH_ALL_TAIL, "??*x", 3);
TEST_EQUAL ("*A?B*", "*A?B*", TRUE);
TEST_EQUAL ("A*BCD", "A*BCD", TRUE);
TEST_EQUAL ("*YZ", "*YZ", TRUE);
TEST_EQUAL ("A1x", "A1x", TRUE);
TEST_EQUAL ("AB*CD", "AB**CD", TRUE);
+ TEST_EQUAL ("AB*?*CD", "AB*?CD", TRUE);
+ TEST_EQUAL ("AB*?CD", "AB?*CD", TRUE);
TEST_EQUAL ("AB*CD", "AB*?*CD", FALSE);
TEST_EQUAL ("ABC*", "ABC?", FALSE);
TEST_MATCH("?*x", "x", FALSE);
TEST_MATCH("*?*x", "yx", TRUE);
TEST_MATCH("*?*x", "xxxx", TRUE);
-
- string = g_convert ("Äx", -1, "UTF-8", "Latin1", NULL, NULL, NULL);
- TEST_MATCH("*x", string, TRUE);
- TEST_MATCH("?x", string, TRUE);
- TEST_MATCH("??x", string, FALSE);
+ TEST_MATCH("x*??", "xyzw", TRUE);
+ TEST_MATCH("*x", "Äx", TRUE);
+ TEST_MATCH("?x", "Äx", TRUE);
+ TEST_MATCH("??x", "Äx", FALSE);
+ TEST_MATCH("abäö", "abäö", TRUE);
+ TEST_MATCH("abäö", "abao", FALSE);
+ TEST_MATCH("ab?ö", "abäö", TRUE);
+ TEST_MATCH("ab?ö", "abao", FALSE);
+ TEST_MATCH("abä?", "abäö", TRUE);
+ TEST_MATCH("abä?", "abao", FALSE);
+ TEST_MATCH("ab??", "abäö", TRUE);
+ TEST_MATCH("ab*", "abäö", TRUE);
+ TEST_MATCH("ab*ö", "abäö", TRUE);
+ TEST_MATCH("ab*ö", "abaöxö", TRUE);
g_print ("\n%u tests passed, %u failed\n", passed, failed);
projects / platform / upstream / glib.git / commitdiff