Paul "LeoNerd" Evans [Sat, 28 Jan 2012 15:02:48 +0000 (15:02 +0000)]
Don't use 1906 to test strptime/mktime because it yields a negative time_t value
Paul "LeoNerd" Evans [Thu, 29 Dec 2011 23:38:56 +0000 (23:38 +0000)]
Bump version number in POSIX.pm
Paul "LeoNerd" Evans [Fri, 23 Dec 2011 18:17:12 +0000 (18:17 +0000)]
Added some docs about new POSIX::strptime()
Paul "LeoNerd" Evans [Wed, 21 Dec 2011 00:10:41 +0000 (00:10 +0000)]
mktime() before returning result from strptime(), to ensure wday/yday/isdst fields are correct
Paul "LeoNerd" Evans [Fri, 23 Dec 2011 17:50:43 +0000 (17:50 +0000)]
Accept strptime \$str, "format" to use/set pos() magic at parsing position
Paul "LeoNerd" Evans [Sun, 18 Dec 2011 23:25:03 +0000 (23:25 +0000)]
Initial hack at strptime(); just literal strings for now
Paul "LeoNerd" Evans [Sat, 28 Jan 2012 15:12:47 +0000 (15:12 +0000)]
Ammend comment referring to init_tm() to point to its correct location
Karl Williamson [Sat, 11 Feb 2012 21:54:24 +0000 (14:54 -0700)]
perldelta: Note is_utf8_char_buf() and is_utf8_char()
Karl Williamson [Sat, 11 Feb 2012 21:40:02 +0000 (14:40 -0700)]
Change =head1 to =head2 in perldelta
Karl Williamson [Sat, 11 Feb 2012 21:20:56 +0000 (14:20 -0700)]
Deprecate is_utf8_char()
This function assumes that there is enough space in the buffer to read
however many bytes are indicated by the first byte in the alleged UTF-8
encoded string. This may not be true, and so it can read beyond the
buffer end. is_utf8_char_buf() should be used instead.
Karl Williamson [Sat, 11 Feb 2012 21:04:39 +0000 (14:04 -0700)]
Add is_utf8_char_buf()
This function is to replace is_utf8_char(), and requires an extra
parameter to ensure that it doesn't read beyond the end of the buffer.
Convert is_utf8_char() and the only place in the Perl core to use the
new one, assuming in each that there is enough space.
Thanks to Jarkko Hietaniemi for suggesting this function name
Dominic Hargreaves [Sat, 11 Feb 2012 21:18:06 +0000 (14:18 -0700)]
ExtUtils::Install: Fix POD error
Dominic Hargreaves [Sat, 11 Feb 2012 21:13:07 +0000 (14:13 -0700)]
Term-Cap/Cap.pm: Fix POD errors
Karl Williamson [Sat, 11 Feb 2012 20:43:26 +0000 (13:43 -0700)]
intrpvar.h: Rmv no longer used PL_ variable
Commit
24caacbccae7b938deecdcc3f13dd66c9c6a684e removed all uses of this
variable, but failed to remove it.
Karl Williamson [Sat, 11 Feb 2012 17:49:06 +0000 (10:49 -0700)]
regcomp.c: /[[:lower:]]/i should match the same as /\p{Lower}/i
Same for [[:upper:]] and \p{Upper}. These were matching instead all of
[[:alpha:]] or \p{Alpha}. What /\p{Lower}/i and /\p{Upper}/i match instead
is \p{Cased}, and so that is what these should match.
Craig A. Berry [Sat, 11 Feb 2012 20:17:12 +0000 (14:17 -0600)]
General-purpose symbol shortening for VMS.
Some folks like to write long sentences and then use them as
variable names, which doesn't come up that often, but when it
does, the build on VMS falls down hard if any of the resulting
symbols is longer than 31 characters. The problem is not for
the compiler, which when using /NAMES=SHORTENED (which we now do
by default) will shorten the symbols, but for the linker, which
must have an exact list of the symbol names to be exported when
creating the perlshr.exe shareable image.
That list of potentially shortened symbols goes in a linker options
file created by vms/gen_shrfls.pl. Until now we had no recourse
but to hard-code there a mapping of long symbols to shortened ones,
but the AUTODIN-II polynomial used by the compiler to do the
shortening is (partially) documented under the help for CC/NAMES,
and it was possible to extrapolate from there and create a pure-
Perl implementation that mimics precisely what the C compiler (and
the C++ compiler under "extern C" declarations) use for shortening
long symbol names.
Symbols like Perl__it_was_the_best_of_times_it_was_the_worst_of_times
can now be created freely without causing the VMS linker to seize up.
Karl Williamson [Sat, 11 Feb 2012 20:20:45 +0000 (13:20 -0700)]
regcomp.c: Remove outdated #undef
Karl Williamson [Sat, 11 Feb 2012 17:00:11 +0000 (10:00 -0700)]
mktables: Update comments, variable names
Commit
d11155ec2b4e3f6cf952e2a25615aec506a8e296 changed the format of
some of the generated tables, but I left some of the old comments and
variable names the same in order to not make this already large commit
bigger. This updates these to reflect the new format.
It also refactors one 'if' statement to not use a block.
Karl Williamson [Fri, 10 Feb 2012 17:35:48 +0000 (10:35 -0700)]
regcomp.c: Remove duplicate inversion list
\h and \p{XPosixBlank} contain the same code points, so there is no need
to have both of them.
Zefram [Sat, 11 Feb 2012 17:14:45 +0000 (17:14 +0000)]
handle conditional definition of PL_check_mutex
PL_check_mutex only exists on threading builds, so makedef.pl needs to
know about that conditionality to produce correct export lists.
Zefram [Sat, 11 Feb 2012 11:05:51 +0000 (11:05 +0000)]
add wrap_op_checker() API function
This function provides a convenient and thread-safe way for modules to
hook op checking.
Zefram [Sat, 11 Feb 2012 08:35:35 +0000 (08:35 +0000)]
restore ExtUtils-ParseXS portability to Perl 5.6
Zefram [Sat, 11 Feb 2012 08:30:03 +0000 (08:30 +0000)]
update ExtUtils-ParseXS to CPAN version 3.15
Ensure that every module in the ExtUtils-ParseXS distribution has
a $VERSION.
Karl Williamson [Fri, 10 Feb 2012 23:18:23 +0000 (16:18 -0700)]
UCD.t: white-space only
This outdents some statements that are no longer enclosed in a block
Karl Williamson [Fri, 10 Feb 2012 23:10:12 +0000 (16:10 -0700)]
mktables: Fix up some comments in the generated files
These were incorrectly stating that some tables are accessible via
Unicode::UCD, and giving the wrong name in some instances.
Karl Williamson [Fri, 10 Feb 2012 22:13:10 +0000 (15:13 -0700)]
Unicode::UCD::prop_invmap: Store Nv property as adjusted type
By converting this property to requiring adjustments to get the proper
values, its storage size decreases by more than half.
Karl Williamson [Fri, 10 Feb 2012 21:50:18 +0000 (14:50 -0700)]
Unicode::UCD::prop_invmap(): New improved API
Thanks to Tony Cook for suggesting this.
The API is changed from returning deltas of code points, to storing the
actual correct values, but requiring adjustments for the non-initial
elements in a range, as explained in the pod.
This makes the data less confusing to look at, and gets rid of
inconsistencies if we didn't make the same sort of deltas for entries
that were, e.g. arrays of code points.
Karl Williamson [Thu, 9 Feb 2012 18:01:43 +0000 (11:01 -0700)]
Unicode::UCD: move common directory to subroutine
All the files that should ever be read by the subroutine will be found
in the unicore directory, so can specify it in the subroutine instead of
in each call to it. This makes things slightly easier in future
commits.
Karl Williamson [Wed, 8 Feb 2012 17:44:00 +0000 (10:44 -0700)]
Unicode::UCD: pod and comment nits
One comment is out-dated, also moves a line of code so that the comments
flow better.
Craig A. Berry [Thu, 9 Feb 2012 23:33:51 +0000 (17:33 -0600)]
Avoid null pointer dereference in tovmsspec.
Before
360732b5267d5, when dirend was either never set at all or
set to NULL, the routine always returned early before executing
the parts that look at dirend. But after that change it became
possible to dereference a null dirend pointer. Let's not do that.
Karl Williamson [Sun, 5 Feb 2012 23:17:54 +0000 (16:17 -0700)]
perrebackslash, perlrecharclass: Note locale effects
This adds text to specify what happens under 'use locale'.
Karl Williamson [Sun, 5 Feb 2012 22:36:08 +0000 (15:36 -0700)]
Allow [[:blank:]] to work under locale
This takes advantage of the recently added Configure probe, and if the
platform has an isblank library function, calls that under locale. This
now matches the documentation
Karl Williamson [Sun, 5 Feb 2012 00:54:01 +0000 (17:54 -0700)]
Use system isascii() when available under locale
We have code that assumes that ASCII should be locale dependent, but it
was missing its final link. This supplies that, and makes the code work
as documented. I thought it better to do that then to document yet
another exception.
Karl Williamson [Sun, 5 Feb 2012 00:49:10 +0000 (17:49 -0700)]
regcomp.c: Rmv no-longer used variables and code
Karl Williamson [Sun, 5 Feb 2012 00:44:56 +0000 (17:44 -0700)]
reg_posixcc.t: Add tests for \v, \V, \h, \H
Karl Williamson [Sun, 5 Feb 2012 00:08:58 +0000 (17:08 -0700)]
regcomp.c: Use compiled-in inversion lists
This uses the compiled inversion lists to generate Posix character
classes and things like \v, \s inside bracketed character classes.
This paves the way for future optimizations, and fixes the bug which has
no formal bug number that /[[:ascii:]]/i matched non-Ascii characters,
such as the Kelvin sign, unlike /\p{ascii}/i.
Karl Williamson [Sat, 4 Feb 2012 18:42:58 +0000 (11:42 -0700)]
Add compile-time inversion lists for POSIX classes
These will be used in regcomp.c to replace the existing bit-wise
handling of these, enabling subsequent optimizations.
These are compiled-in, and hence affect the memory footprint of every
program, including those that don't use Unicode. The lists that aren't
tiny are therefore currently restricted to only the Latin1 range;
anything needed beyond that will have to be read in at execution time,
just as before.
The design allows for easy conversion from Latin1 to use the full
Unicode range, should it be deemed desirable for some or all of these.
Karl Williamson [Sat, 4 Feb 2012 05:01:03 +0000 (22:01 -0700)]
regcomp.c: Use compile-time invlists
This creates three simple compile-time inversion lists from the data
that has been generated in a previous commit, and uses two of them.
Three PL_ variables are used to store them.
Karl Williamson [Sat, 7 Jan 2012 19:10:41 +0000 (12:10 -0700)]
Add regen/mk_invlists.pl, charclass_invlists.h
This will be used to generate compile-time inversion lists in a C hdr
file that can be included in programs for initialization speed
Three simple inversion lists are included in this initial commit
Karl Williamson [Sat, 7 Jan 2012 19:34:02 +0000 (12:34 -0700)]
regcomp.c: Add ability to have compiled-in inversion lists
This adds a routine that will take a C array and quickly create an
inversion list that points to that array. Thus the array had better be
exactly the internal form that is required for an inversion list. To
make sure that this doesn't get out of sync, a new field in the list's
header is created that is a combination of
version-number/inversion-list-type.
Karl Williamson [Mon, 9 Jan 2012 19:12:00 +0000 (12:12 -0700)]
handy.h: Add comment
Karl Williamson [Fri, 3 Feb 2012 21:51:49 +0000 (14:51 -0700)]
t/re/re_tests: Add Todo test
This bug was spotted by Tom Christiansen, but no bug report has been
written. /[[:ascii:]]/i should match the same set of code points as
/\p{ASCII}/i. But it is matching things outside the ASCII range
Karl Williamson [Fri, 3 Feb 2012 22:08:10 +0000 (15:08 -0700)]
regcomp.c: Use new complement union
Previous commits have changed the API of invlist_union so the first
parameter can be NULL, and are to take the complement of the 2nd. We
can take advantage of that to simplify this code.
Karl Williamson [Fri, 3 Feb 2012 18:40:34 +0000 (11:40 -0700)]
regcomp.c: Add ability to take union of a complement
Previous commits have added the ability to the inversion list
intersection routine to take the complement of one of its inputs.
Likewise, for unions, this will be a frequent paradigm, and it is
cheaper to do the complement of an input in the routine than to
construct a new temporary that is the desired complement, and throw it
away.
Karl Williamson [Fri, 3 Feb 2012 17:57:33 +0000 (10:57 -0700)]
regcomp.c: _invlist_subtract() becomes a macro
This function is no longer necessary, as it is just a call to the newly
created _invlist_intersection_maybe_complement_2nd() with the correct
parameters.
Karl Williamson [Fri, 3 Feb 2012 17:32:15 +0000 (10:32 -0700)]
regcomp.c: Add ability to take intersection of complement
It turns out that it is a common paradigm to want to take the
intersection of an inversion list with the complement of another
inversion list. In fact, this is the how to subtract the second
inversion list from the first, as what remains in the first after the
subtraction is everything in it that is not in the second.
It also turns out that it adds very few cycles to an intersection to
complement one (or both, should we choose to) of the operands. By
adding this capability, we don't have to create a copy of the inverted
operand beforehand, just to throw it away.
Karl Williamson [Fri, 3 Feb 2012 21:55:45 +0000 (14:55 -0700)]
regcomp.c: Use NULL first parameter to _invlist_union
A previous commit has changed _invlist_union() to accept a NULL
first parameter. This takes advantage of that to simplify some code.
Karl Williamson [Thu, 12 Jan 2012 18:09:39 +0000 (11:09 -0700)]
regcomp.c: Chg invlist_union() to accept NULL first param
It is common in a loop to keep adding inversion lists to a current
running total. But the first time through, the current union list needs
to be initialized from NULL. This puts that code in the function
instead of the callers each having to do it.
Reini Urban [Thu, 9 Feb 2012 06:44:56 +0000 (07:44 +0100)]
Add new email address for Reini Urban
Robin Barker [Thu, 9 Feb 2012 06:42:15 +0000 (07:42 +0100)]
Silence compiler warnings
Cf. RT #110208.
- Remove missing unused variables: op.c, regcomp.c
- Silence -Wformat type error: sv.c
- Cast first part of (,) expression as (void): gv.c
Tony Cook [Thu, 9 Feb 2012 01:54:43 +0000 (12:54 +1100)]
README.cygwin now has fewer pod issues, update known_pod_issues.dat
Reini Urban [Wed, 8 Feb 2012 23:58:21 +0000 (17:58 -0600)]
add Cygwin::sync_winenv [perl #110190]
Cygwin::sync_winenv should fix [CPAN #65052], ADODB missing
%COMMONPROGRAMFILES%.
sync_winenv code and solution by Chris Day.
Reini Urban [Wed, 8 Feb 2012 23:51:39 +0000 (17:51 -0600)]
Revise perlcygwin.pod for cygwin-1.7
cygwin does not support windows prior NT5 anymore.
cygwin paths are now all UTF8 encoded, special pathname
characters and names are now allowed.
Removed outdated cruft and fix wrong parts.
Chris 'BinGOs' Williams [Wed, 8 Feb 2012 18:24:00 +0000 (18:24 +0000)]
Update Module-Metadata to CPAN version 1.000009
[DELTA]
1.0.9 2012-02-08 12:00:00
- API of 'provides' changed to require a 'version' argument to future
proof the function against CPAN Meta Spec changes (DAGOLDEN)
- Fatal errors now use 'croak' instead of 'die'; Carp added as
prerequisite (DAGOLDEN)
Nicholas Clark [Tue, 7 Feb 2012 16:58:50 +0000 (17:58 +0100)]
Clean up Pod::Html's cache as part of distclean on *nix.
This has been done on win32 since commit
b4a415570dc258dd in 2006.
However, the new Pod::Html caching code only uses one file, pod2htmd.tmp,
so remove now redundant cleanup code for pod2htmi.tmp.
Nicholas Clark [Tue, 7 Feb 2012 16:07:33 +0000 (17:07 +0100)]
Move lib/Pod/t/eol.t to ext/Pod-Html, as it's testing Pod::Html.
Nicholas Clark [Tue, 7 Feb 2012 15:59:23 +0000 (16:59 +0100)]
Convert triplicated code in lib/Pod/t/eol.t to a loop.
Nicholas Clark [Tue, 7 Feb 2012 15:20:18 +0000 (16:20 +0100)]
Refactor lib/Pod/t/eol.t
* use variables for the names of temporary files
* use lexicals for file handles
* check the return value of close
* use is() rather than ok() with ==
[possibly still dubious that it's using unpack checksums for comparison,
instead of SHAs or simply File::Compare]
Nicholas Clark [Tue, 7 Feb 2012 14:46:03 +0000 (15:46 +0100)]
The cleanup code in lib/Pod/t/eol.t needs updating to track Pod::Html changes.
eol.t gained code to clean up temporary files it generated as part of commit
0ec158f4b0db050a in 2002. The temporary file names used by Pod::Html were
changed by commit
33869856bc668ad8 in 2003, but eol.t had never been updated.
Nicholas Clark [Wed, 8 Feb 2012 13:38:01 +0000 (14:38 +0100)]
Add PERL_RELOCATABLE_INCPUSH to PL_non_bincompat_options, and hence -V output.
PERL_RELOCATABLE_INCPUSH was added in commit
f31c6eed22759301. As it
causes a small specific behaviour change that isn't visible in any other
configuration option, it seems useful to indicate that the perl binary has
been compiled with it, to aid debugging. (Specifically, debugging of "Why
isn't this working?" when it failed to be enabled.)
Chris 'BinGOs' Williams [Wed, 8 Feb 2012 09:51:30 +0000 (09:51 +0000)]
Update Parse-CPAN-Meta to CPAN version 1.4402
[DELTA]
1.4402 Tue Feb 07 2012
- Minor maintenance: standardized newlines to Unix style
Chris 'BinGOs' Williams [Wed, 8 Feb 2012 09:45:12 +0000 (09:45 +0000)]
Update CPAN-Meta-YAML to CPAN version 0.007
[DELTA]
0.007 2012-02-07 22:42:42 EST5EDT
- Generated from ADAMK/YAML-Tiny-1.50.tar.gz
- Documentation fix to replace missing abstract
Chris 'BinGOs' Williams [Wed, 8 Feb 2012 09:29:02 +0000 (09:29 +0000)]
Update Module-Metadata to CPAN version 1.000008
[DELTA]
1.0.8 2012-02-07 22:30:00
- Adds 'provides' method to generate a CPAN META provides data structure
correctly; use of package_versions_from_directory is discouraged (DAGOLDEN)
Reini Urban [Mon, 6 Feb 2012 20:29:46 +0000 (14:29 -0600)]
fix cygwin -Uuseithreads hints
Before -Uuseithreads resulted in a threaded perl. Now
-Uuseithreads and -Uusethreads DWIM
Nicholas Clark [Tue, 7 Feb 2012 16:18:36 +0000 (17:18 +0100)]
is_duplicate_pod() and get_pod_metadata() in pod_lib.pl modified $_
Reported by Reini Urban in RT #110078, installperl generated a warning and
failed to install CORE.pod, because is_duplicate_pod() modified $_
is_duplicate_pod() and get_pod_metadata() now both localise $_, to ensure
that they don't corrupt any caller's state.
Ricardo Signes [Tue, 7 Feb 2012 12:32:57 +0000 (07:32 -0500)]
avoid making a new top-level package for one helper
Chris 'BinGOs' Williams [Tue, 7 Feb 2012 10:14:27 +0000 (10:14 +0000)]
Update CPAN-Meta-YAML to CPAN version 0.006
[DELTA]
0.006 2012-02-06 20:51:47 EST5EDT
- Generated from ADAMK/YAML-Tiny-1.50.tar.gz
- Set back configure_requires prerequisite for ExtUtils::MakeMaker
from 6.30 to 6.17 (per request from Andreas Koenig)
Zefram [Tue, 7 Feb 2012 02:45:19 +0000 (02:45 +0000)]
in Carp, fix circular dep on Perl 5.6
The circular dependency between Carp and warnings was causing trouble
with new versions of Carp against very old versions of warnings (versions
that were bundled with Perl 5.6). No functional effect on blead.
Ricardo Signes [Tue, 7 Feb 2012 03:11:04 +0000 (22:11 -0500)]
add an alias for marcg's other email
Marc Green [Wed, 25 Jan 2012 02:20:55 +0000 (21:20 -0500)]
Fix portability issue with Pod::Html test case
Marc Green [Tue, 17 Jan 2012 05:58:59 +0000 (00:58 -0500)]
Modernize the use of open() (copied from old code)
Marc Green [Tue, 17 Jan 2012 05:26:21 +0000 (00:26 -0500)]
Update MANIFEST
Marc Green [Fri, 13 Jan 2012 16:42:00 +0000 (11:42 -0500)]
Re-add cache feature
./installhtml was unacceptably slow without pod2html caching,
so this commit re-adds it.
Ricardo Signes [Mon, 6 Feb 2012 16:13:57 +0000 (11:13 -0500)]
note which releases have shipped
this is to make it easier to scan and find which release
is the next unscheduled, unshipped one, so we (I) can see
how far off we have scheduled and whether a next volutneer
needs to be volunteered
Rafael Garcia-Suarez [Mon, 6 Feb 2012 09:05:00 +0000 (10:05 +0100)]
Document the special meaning of "indir" on the shebang line
David Golden [Mon, 6 Feb 2012 03:20:03 +0000 (22:20 -0500)]
sync version.pm code with CPAN
Applied patch from John Peacock, but added whitespace fixes,
corrected pod link error and updated known Pod issues to reflect
a fix.
Nicholas Clark [Sun, 5 Feb 2012 21:16:58 +0000 (22:16 +0100)]
In t/porting/authors.t, correct a typo in the git log format string.
The format added in commit
3ea0c581844689ab had a typo - %cn (committer
name) used instead of %ce (committer e-mail).
Nicholas Clark [Sun, 5 Feb 2012 14:52:16 +0000 (15:52 +0100)]
In bisect-runner.pl, don't set $defines{cc} if printing usage messages.
ccache isn't needed to print a usage message. The current heuristic for
detecting ccache is noisy if it fails, which interferes with any usage or
error output.
Nicholas Clark [Sun, 5 Feb 2012 14:35:37 +0000 (15:35 +0100)]
bisect.pl --help should show the full documentation, not just the usage.
The initial paragraphs of documentation are actually more useful than most of
the options documentation. So show them too.
Suggested by David Leadbeater.
David Golden [Sun, 5 Feb 2012 01:28:05 +0000 (20:28 -0500)]
warnings.pm docs: clarify categories are in perllexwarn
Karl Williamson [Tue, 31 Jan 2012 18:21:22 +0000 (11:21 -0700)]
Unicode::UCD move =item in pod
This merely moves a whole=item to another place, in preparation for
future commits
Karl Williamson [Sat, 4 Feb 2012 23:04:34 +0000 (16:04 -0700)]
perldelta for Unicode::UCD::prop_invmap() changes
Karl Williamson [Tue, 31 Jan 2012 16:55:44 +0000 (09:55 -0700)]
Unicode::UCD::prop_invmap() compress digit results
This changes the output of prop_invmap() for the Perl_Decimal_Digit
property to use code point deltas, similar to other properties. This
causes the output to be 1/10 what it used to be.
Karl Williamson [Tue, 31 Jan 2012 17:25:46 +0000 (10:25 -0700)]
UCD.t: White space only
Indent properly to account for these being in a newly formed block
Karl Williamson [Tue, 31 Jan 2012 01:17:11 +0000 (18:17 -0700)]
Unicode::UCD::prop_invmap(): Make the NFKCCF property return deltas
The file for this property is stored in the old-style format for
backward compatibility with any applications that might be reading it
directly. But the values should be returned through the Unicode::UCD
API as deltas for consistency with other, similar properties.
Karl Williamson [Sun, 29 Jan 2012 00:39:10 +0000 (17:39 -0700)]
Unicode::UCD::prop_invmap(): Return deltas for the 'dm' property
Earlier commits caused the return of prop_invmap() for certain
properties to return deltas from code points instead of the code points
themselves, for compactness of storage and speed of searching. This
causes the same for the 'dm' property, for consistency with the others,
even though the space savings is not large for this one; essentially the
same code can be used for the two types now; instead of an application
having to have special cases.
Karl Williamson [Sat, 28 Jan 2012 19:44:29 +0000 (12:44 -0700)]
utf8.c: white-space only
This adds an indent now that the code is in a newly created block
Karl Williamson [Sat, 28 Jan 2012 17:47:25 +0000 (10:47 -0700)]
utf8.c: Use the new compact case mapping tables
This changes the Perl core when looking up the
upper/lower/title/fold-case of a code point to use the newly created
more compact tables. Currently the look-up is done by a linear search,
and the new tables are 54-61% of the size of the old ones, so that on
average searches are that much shorter
Karl Williamson [Sat, 28 Jan 2012 16:51:58 +0000 (09:51 -0700)]
mktables: Generate some delta tables
This commit has the effect of changing the non-legacy tables for the lc,
uc, tc, and fc properties to use maps of deltas from the code points
instead of the code points themselves, thus shortening them
significantly, and hence the time required to search through them.
Note that these tables are new, and currently used only by Unicode::UCD.
A future commit will change the Perl core to use them.
Karl Williamson [Sat, 28 Jan 2012 16:26:29 +0000 (09:26 -0700)]
mktables: Change generated file comment
All the files that mktables generates that are for external-to-core use
have now been changed so that the code requests explicitly for each that
they have the comment that says they are for external use, but it is
deprecated to use them. That means that any files that haven't been so
explicitly set should have the comment instead that says they are for
internal use only.
Karl Williamson [Fri, 27 Jan 2012 18:33:51 +0000 (11:33 -0700)]
mktables: Preserve old format in some tables
Future commits will cause tables that map to code points to, in general,
use deltas instead. This ensures that files that contain tables and
have been mentioned publicly in the past continue to have their current
contents and format, so that applications that read them (such as
Unicode::Normalize) are unaffected.
Karl Williamson [Fri, 27 Jan 2012 18:26:03 +0000 (11:26 -0700)]
Unicode::UCD: pod and comment nits
Karl Williamson [Fri, 27 Jan 2012 17:50:47 +0000 (10:50 -0700)]
mktables: Allow generation of delta tables
Delta tables are those in which the mapping is not stored as-is, but is
modified to be the delta between the actual mapping and the code point
it is for. This allows for smaller tables that are faster to search and
require less memory to store.
For example, consider the lower case mapping of A=>a, B=b, ... Z=>z.
Prior to this patch, this requires 26 entries in the table; now it
requires just one. This is because A=65 and a=97. We store 97-65=32.
And 32 is the same delta for each of A-Z, so we can store these as a
single range each with the same value, 32.
The delta tables tend to be half as large as the non-ones, or even
smaller.
This just enables the feature. No tables currently use it. For that,
changes in other Unicode::UCD need to be coordinated.
Karl Williamson [Fri, 27 Jan 2012 04:27:30 +0000 (21:27 -0700)]
mktables: White-space, comments only
A previous commit has added two nested blocks surrounding the affected
code. This looks like a big change, but it is in fact only white space
plus reflowing things to fit in an 80 column window, plus slight changes
to comments.
I verified that there were no code changes by using a diff command that
can ignore leading white space changes, and hence gave a more accurate
difference listing
Karl Williamson [Fri, 27 Jan 2012 04:01:33 +0000 (21:01 -0700)]
mktables: Refactor if-else series
This is a slight refactoring to avoid using 'next' in the loop, and to
surround things with a bare block. Future commits will want to
do common code at the bottom of the loop, including a redo of the bare
block.
Karl Williamson [Mon, 23 Jan 2012 19:43:42 +0000 (12:43 -0700)]
Unicode::UCD::prop_invmap(): Use regex to get trie
This should speed up this test slightly
Karl Williamson [Sun, 22 Jan 2012 15:50:24 +0000 (08:50 -0700)]
mktables: Don't generate no-longer used tables
Previous commits have removed all uses of these tables, so they are no
longer needed.
Karl Williamson [Sun, 22 Jan 2012 15:35:34 +0000 (08:35 -0700)]
Unicode::UCD: Rmv uses of no-longer needed tables
Previous commits have expanded whats in the full case mapping tables
to include the simple maps as well. Thus the specially constructed
tables need no longer be used, leading to simplification.
Karl Williamson [Sun, 22 Jan 2012 15:27:11 +0000 (08:27 -0700)]
UCD.t: white space only
outdent now that surrounding block is removed
Karl Williamson [Sun, 22 Jan 2012 03:04:51 +0000 (20:04 -0700)]
mktables: Include simple mappings in full tables
This changes the case change mapping tables to include the simple
mappings. This was done in 5.14 for the case folding table. The full
mappings are contained, as before, in a hash. Now the simple mappings
they override (when doing multi-char case changing) are added to the
main body of the table, to the already existing simple mappings that
aren't overridden.
If the caller wants to do full mapping, it should look first in the
hash, and only if not found, look in the main body. If the caller wants
only simple mapping, it ignores the hash.
This is already how the code in utf8.c that reads these tables is
constructed.
The .t is modified to take into account that these code points are now
in the main table body.