Daniel Dragan [Tue, 14 Aug 2012 12:50:25 +0000 (13:50 +0100)]
fix RT#88840, don't terminate a child fork psuedo process in DLL Loader Lock
TerminateThread will terminate a thread but leaks all resources
of that thread, and all locks will never be released, as documented in MSDN.
There is no alternative to locks not being released that I see, but atleast
-e "if ($pid=fork){kill(9,$pid)} else {sleep 5}"
in fork.t won't deadlock with this patch since win32_start_child be reached before
TerminateThread happens. The 5 ms timeout can be increased if problems
arise in the future. The HWND of the child is delivered by win32_start_child
very early, before any perl bytecode is executed, therefore the delay is
keeping in spirit with a kill 9. In any case, if the child thread fails
to schedule, (a DllMain in DLL_THREAD_ATTACH of some DLL in the process
deadlocks or does very long (over 5 ms right now) sync IO), the parent interp
will bail out.
Nicholas Clark [Mon, 13 Aug 2012 14:11:35 +0000 (16:11 +0200)]
Document that newCONSTSUB{,_flags} takes ownership of a reference to the SV.
Also note the collusion between op_const_cv() and cv_clone(), whereby the
former returns a fresh copy of the SV to the latter, which is then immediately
passed to newCONSTSUB.
Nicholas Clark [Mon, 13 Aug 2012 13:11:41 +0000 (15:11 +0200)]
XS::APItest::newCONSTSUB was not handling SV reference counts correctly.
newCONSTSUB() and newCONSTSUB_flags() take ownership of (one reference to)
the passed-in SV. As the XS wrapper is passing in a SV taken from the stack,
it needs to up the reference count by one in order to avoid later bugs.
Nicholas Clark [Mon, 13 Aug 2012 11:38:03 +0000 (13:38 +0200)]
Use ALIAS to provide XS::APItest::newCONSTSUB and newCONSTSUB_flags
Previously both C routines were wrapped with newCONSTSUB_type, which used a
"type" parameter to determine which C code to call. Use an ALIAS to bind the
code to two names, and eliminate the "type" parameter.
This makes the test code clearer. It's not perfect, as the XS wrapper
XS::APItest::newCONSTSUB has a flags parameter whereas the underlying C code
does not, but fixing this would require considerably more XS hackery.
Steve Hay [Tue, 14 Aug 2012 00:07:29 +0000 (01:07 +0100)]
Clean up temporary test file in Search-Dict test
Need to untie before unlink on Windows.
Steve Hay [Mon, 13 Aug 2012 23:44:32 +0000 (00:44 +0100)]
Regenerate test data for porting/customized.t
Steve Hay [Mon, 13 Aug 2012 23:23:00 +0000 (00:23 +0100)]
More tweaks to Maintainers.pl
- One CGI test is currently customized
- Nine Text::Balanced tests are currently customized
- One Test::Harness test is currently customized and another is renamed,
requiring a MAP; plus all the module version numbers are bumped
- Getopt::Long doesn't require a MAP
I have requested new CPAN versions of various CPAN distros that are
currently customized with changes which should be merged back upstream
with a view to getting them back in sync as they are intended to be.
Namely: CGI, Sys-Syslog, Test-Harness, Text-Balanced and Text-Tabs+Wrap.
Steve Hay [Mon, 13 Aug 2012 13:05:15 +0000 (14:05 +0100)]
Insert missing words into SvPV_force* documentation
Steve Hay [Mon, 13 Aug 2012 08:46:46 +0000 (09:46 +0100)]
Fix broken links in perldelta
Clearly, I need to be more vigilant about running porting/*.t before
pushing things which I'm sure couldn't possibly break anything...
Steve Hay [Mon, 13 Aug 2012 08:18:45 +0000 (09:18 +0100)]
Minor perldelta changes
including wrapping lines to 79 columns in the template pod.
Steve Hay [Mon, 13 Aug 2012 07:55:31 +0000 (08:55 +0100)]
Begin editing perldelta for 5.17.3
Add list of modules upgraded so far, plus a few tidy-ups.
Steve Hay [Mon, 13 Aug 2012 07:54:27 +0000 (08:54 +0100)]
Bump $Win32CORE::VERSION
The .c file was changed by
2e94d7323a.
Steve Hay [Mon, 13 Aug 2012 00:39:21 +0000 (01:39 +0100)]
Mark missing CPAN distribution files as EXCLUDED from the core
All tests and test data are already excluded, and I don't think
cpan-mirrors belongs in the perl core either.
jkeenan [Sun, 12 Aug 2012 18:45:08 +0000 (14:45 -0400)]
Better description for Pod::Find.
David Mitchell [Sun, 12 Aug 2012 17:27:31 +0000 (18:27 +0100)]
skip failing leak test under -Dmad
a test for a leaky regex introduced by
d97935e0 leaks under MAD:
because it uses eval, and eval leaks under MAD.
So skip it in this case.
Chris 'BinGOs' Williams [Sun, 12 Aug 2012 08:20:59 +0000 (09:20 +0100)]
Update Encode to CPAN version 2.46
[DELTA]
$Revision: 2.46 $ $Date: 2012/08/12 05:49:30 $
! Encode.pm
Fixed: RT#78917 for I18N-Charset: Fails with Encode 2.45
To be more exact, 2.45 broke Encode->encodings(':all')
https://rt.cpan.org/Ticket/Bug/Display.html?id=78917
Chris 'BinGOs' Williams [Sun, 12 Aug 2012 08:18:30 +0000 (09:18 +0100)]
Update Module-Load-Conditional to CPAN version 0.54
[DELTA]
Changes for 0.54 Sun Aug 12 09:10:13 BST 2012
=================================================
* VMS test fixes from Craig Berry
Chris 'BinGOs' Williams [Sun, 12 Aug 2012 08:09:29 +0000 (09:09 +0100)]
Sync Module-CoreList version with CPAN
Karl Williamson [Sat, 11 Aug 2012 20:56:55 +0000 (14:56 -0600)]
mktables: Rebuild if local Makefile has changed
Normally, mktables is called from the Makefile at the base level. But
during development, it may manually be called from the directory (and
hence that directory's Makefile). This patch causes it to rebuild if
that Makefile changes.
Karl Williamson [Sat, 11 Aug 2012 20:30:02 +0000 (14:30 -0600)]
perlre: Nits
This fixes some grammar ("either" legally should refer to only a
dual-valued option set) and removes unnecessary distracting detail.
Karl Williamson [Sat, 11 Aug 2012 20:19:45 +0000 (14:19 -0600)]
regcomp.c: Optimization not valid for Latin Sharp S
The regex optimizer optimizes some quantifier expressions into simpler
versions. It turns out that these optimizations don't work on a
quantified, folded LATIN SMALL LETTER SHARP S under /d. This is due to
the size differential of the fold from the source.
This commit omits the optimization if this circumstance occurs anywhere
in the regex prior to the determination of whether to optimize or not.
I tried adding a parameter to study_chunk() to indicate more locally if
the optimization should be excluded or not; but my first attempt did not
fix the bug, and I chose to not pursue that line. This character is so
abnormal that it's probably best anyway to be overly cautious when
confronted with it.
Karl Williamson [Sat, 11 Aug 2012 20:10:05 +0000 (14:10 -0600)]
regcomp.c: Extract duplicate code to common function
Comments warned about keeping the two code sections in sync; this commit
takes the portions that are identical and makes a common function out of
them, so the synchronization becomes automatic.
Karl Williamson [Fri, 10 Aug 2012 18:16:45 +0000 (12:16 -0600)]
regcomp.c: Make sure counter same in passes 1 and 2
The number of elements was not being incremented in pass 1, whereas that
number is needed later on in pass 1. This did not cause a
bug, as currently, in pass 1 we care only if the count is 1 or not, and
this occurred only in a case where it would get incremented properly to
more than 1 anyway. But this is a potential bug that should be
squelched before it happens.
Karl Williamson [Fri, 10 Aug 2012 17:58:49 +0000 (11:58 -0600)]
regcomp.c: Comments only
The diffs will show more than this, as a block of comments was moved and
revised
Karl Williamson [Fri, 10 Aug 2012 17:53:20 +0000 (11:53 -0600)]
regcomp.c: Use old paradigm in dealing with flags recursively
In a recursive call to reg(), instead of passing our flags pointer, pass
a new one and upon return or in that result with the existing one. I
can see why this should be done, as you don't want to lose what you
already have, as reg() will start by resetting it to 0. I don't know
why one ands it with the known flags, but I'm presuming there is a
reason, and so am copying the paradigm. I searched the commit messages
and didn't find anything. No tests failed, and I didn't figure out a
test that would fail.
Karl Williamson [Fri, 10 Aug 2012 15:11:11 +0000 (09:11 -0600)]
regcomp.c: Create NOTHING node when would have been 0 length EXACT
It's peculiar circumstances indeed that would get to this point in the
code with an EXACT node to be created, but nothing to populate it with.
Perhaps it is impossible; I'm not sure. But commit
5f820f894e71b6970a5aa0fd763a84b647fd628a changed the behavior, which I
discovered in later re-reading the code. Probably the node would be
populated with a single NUL. Just in case it is possible to get here
under these peculiar circumstances, this commit adds code to handle the
case, with a NOTHING node instead of a 0 length EXACT.
Karl Williamson [Thu, 9 Aug 2012 20:38:03 +0000 (14:38 -0600)]
regcomp.c: Set flags when optimizing a [char class]
A bracketed character class containing a single Latin1-range character
has long been optimized into an EXACT node. Also, flags are set to
include SIMPLE. However, EXACT nodes containing code points that are
different when encoded under UTF-8 versus not UTF-8 should not be marked
simple.
To fix this, the address of the flags parameter is now passed to
regclass(), the function that parses bracketed character classes, which
now sets it appropriately. The unconditional setting of SIMPLE that was
always done in the code after calling regclass() has been removed.
In addition, the setting of the flags for EXACT nodes has been pushed
into the common function that populates them.
regclass() will also now increment the naughtiness count if optimized to
a node that normally does that. I do not understand this heuristic
behavior very well, and could not come up with a test case for it;
experimentation revealed that there are no test cases in our test suite
for which naughtiness makes any difference at all.
Karl Williamson [Wed, 8 Aug 2012 03:06:06 +0000 (21:06 -0600)]
regcomp.c: change pattern to utf8 if needed in \N{}
This patch is in preparation for future patches that will no longer
always make any pattern that contains \N{} be encoded in UTF-8. Thus
this patch doesn't actually change anything, but enables future ones.
Karl Williamson [Mon, 6 Aug 2012 22:42:27 +0000 (16:42 -0600)]
re/re_tests: Correct Todo test
This test was not doing what it purported to test. It should show that
a /[s\xDF]/i would not match 'ss', because the 's' is seen in the class,
and not the \xDF (which matches 'ss' under /i) in the appropriate
strings
Karl Williamson [Sat, 4 Aug 2012 17:02:16 +0000 (11:02 -0600)]
re.pm: Nits in pod
This has clarifications, grammar changes, and reflowing to fit into 79
columns
Karl Williamson [Thu, 2 Aug 2012 16:50:00 +0000 (10:50 -0600)]
Add some tests for [\N{}]
Steve Hay [Sat, 11 Aug 2012 11:11:47 +0000 (12:11 +0100)]
Upgrade Compress-Raw-Zlib to 2.056
Craig A. Berry [Sat, 11 Aug 2012 02:25:09 +0000 (21:25 -0500)]
pp_require thread safety for VMS.
When we translate path names of required modules into Unix format,
we haven't (recently) been using the optional second argument to
the translation routines,[1] an argument that supplies a buffer for
the translation. That causes them to use a static buffer. Which
means that if two or more different threads are doing a require
operation at the same time, they will be blindly sharing the same
buffer.
So allocate buffers as we need them and make them mortal so they
will go away at the next state transition.
[1] Use of an automatic variable for the buffer was removed way
back in
46fc3d4c69a0ad.
Father Chrysostomos [Fri, 10 Aug 2012 20:35:42 +0000 (13:35 -0700)]
OptreeCheck.pm: typo
Father Chrysostomos [Fri, 10 Aug 2012 20:31:53 +0000 (13:31 -0700)]
coreamp: Fix test; correct test name
Father Chrysostomos [Thu, 9 Aug 2012 06:18:22 +0000 (23:18 -0700)]
toke.c: Remove unnecessary assignment
S_scan_formline never needs to do PL_bufptr = s, because it returns s,
and all the code paths that follow the one spot that calls it them-
selves assign s to PL_bufptr.
Father Chrysostomos [Thu, 9 Aug 2012 06:12:59 +0000 (23:12 -0700)]
toke.c: Set PL_lex_state less when scanning formats
This (in yylex) is the only code that calls scan_formline:
case LEX_FORMLINE:
PL_lex_state = PL_parser->form_lex_state;
s = scan_formline(PL_bufptr);
if (!PL_lex_formbrack)
{
formbrack = 1;
goto rightbracket;
}
PL_bufptr = s;
return yylex();
}
It is only reached when PL_lex_state is LEX_FORMLINE. scan_formline
itself does not even look at PL_lex_state. It does set it, though,
unless it has reached the end of the format (setting PL_lex_formbrack
to 0) or the end of input.
This means we end up flipping it back and forth between two values.
We don’t have to set PL_lex_state before scan_formline() at all. Hav-
ing scan_formline only set it when it does not need to be LEX_FORMLINE
simplifies things, resulting in less code and fewer assignments.
Father Chrysostomos [Thu, 9 Aug 2012 05:49:17 +0000 (22:49 -0700)]
toke.c: One less token for missing format args
In commit
705fe0e5f8a, when I made the parser understand format syntax
itself, I had to add special handling for a terminating dot where for-
mat arguments were expected:
format =
@
.
The parser expects every format argument line to look like this:
formarg : /* NULL */
{ $$ = NULL; }
| FORMLBRACK stmtseq FORMRBRACK
{ $$ = op_unscope($2); }
;
When the line break is encountered after the @, the FORMLBRACK token
is emitted, and the lexer switches into ‘normal’ (as opposed to for-
mat picture) mode. When the final dot is encountered, since the
FORMLBRACK has already been emitted, the lexer has to conjure up a
FORMRBRACK as well, to avoid a syntax error.
I had it producing a semicolon before the FORMRBRACK, but that is not
necessary, because stmtseq can be null. So this commit removes it.
Father Chrysostomos [Wed, 8 Aug 2012 20:59:03 +0000 (13:59 -0700)]
Deparse multiple stmts in format line correctly
They were being output on separate lines, resulting in the second
statement being treated as a format picture, etc. This affected
cases like this:
format =
@
$x; $y
.
Father Chrysostomos [Wed, 8 Aug 2012 19:50:03 +0000 (12:50 -0700)]
B::Concise: Two unused vars
One was made unnecessary by
35f7559499. The other was added by
35f7559499 and never used. I forgot to clean up once I had things
working.
Steve Hay [Fri, 10 Aug 2012 16:54:51 +0000 (17:54 +0100)]
Fix command-line argument quoting in Porting/cmpVERSION.pl
Windows, NetWare and VMS want double-quotes, other OSes want single quotes.
Ricardo Signes [Fri, 10 Aug 2012 16:35:01 +0000 (12:35 -0400)]
Florian Ragwitz is scheduled for 5.17.4, Sep 20
Tony Cook [Fri, 10 Aug 2012 13:57:10 +0000 (15:57 +0200)]
bump $VERSION for Porting/Maintainers.pm to placate cmp_version.t
7f58bf11 is the first modification of Porting/Maintainers.pm since the
last release, hence cmp_version.t complains.
Steve Hay [Fri, 10 Aug 2012 08:43:17 +0000 (09:43 +0100)]
Fix interpretation of empty list of EXCLUDED files in Maintainers.pl
All files in a distro were wrongly reported as CPAN only if EXCLUDED was
given as []. Fixed that, but really there is no need to specify EXCLUDED
as [] anyway.
Steve Hay [Fri, 10 Aug 2012 08:10:35 +0000 (09:10 +0100)]
Blead no longer customizes Test-Harness's t/testargs.t
The customization added by
6359c64336 was incorporated into Test-Harness
3.24.
Steve Hay [Fri, 10 Aug 2012 08:03:20 +0000 (09:03 +0100)]
pod/perlpodstyle.pod is not customized for blead
The file lives in pod/ rather than cpan/podlators/pod/ but that is already
catered for by the MAP entry. The file contents are identical so there is
no need for a CUSTOMIZED entry, the presence of which causes core-cpan-diff
to complain that blead customizations are missing.
Steve Hay [Fri, 10 Aug 2012 07:42:09 +0000 (08:42 +0100)]
Blead no longer customizes autodie's t/open.t
The customization added by
4b074b7191 was incorporated into autodie 2.11.
Steve Hay [Fri, 10 Aug 2012 01:09:56 +0000 (02:09 +0100)]
Newlines in a runperl() prog cause trouble so use progfile instead
This fixes "Format not terminated at -e line 2, at end of line" errors
on Windows (at least) coming from the new tests added by commit
35f7559499.
Steve Hay [Thu, 9 Aug 2012 17:02:55 +0000 (18:02 +0100)]
Remove spurious instruction to bump the perl version number from the RMG
Commit
30f926b5f1 moved version bump instructions from the start to the end
of the BLEAD-POINT release instructions, but it also left behind an
instruction in-between ("...if you didn't do it earlier...") which it
should have deleted, leaving the document looking like two version bumps
are required for a monthly blead release.
Ricardo Signes [Thu, 9 Aug 2012 14:10:10 +0000 (10:10 -0400)]
buildtoc is no longer the thing
Following these instructions failed, so I checked with Nicholas, whom
I knew had done some work on this stuff in the last year. He suggested
this change, which worked out.
Ricardo Signes [Thu, 9 Aug 2012 13:36:04 +0000 (09:36 -0400)]
add Module::CoreList data for 5.16.1
This does not include a version bump, since one will be made for the
5.17.3 release anyway, and for being a big rewrite of the data backend,
and probably for other reasons.
Ricardo Signes [Thu, 9 Aug 2012 13:18:41 +0000 (09:18 -0400)]
add perl-5.16.1 to perlhist
Ricardo Signes [Thu, 9 Aug 2012 13:16:27 +0000 (09:16 -0400)]
add the 5.16.1 perldelta
Ricardo Signes [Thu, 9 Aug 2012 13:03:21 +0000 (09:03 -0400)]
add epigraphs for 5.16.1
and fix some whitespace on 5.16.0
Steve Hay [Thu, 9 Aug 2012 12:54:24 +0000 (13:54 +0100)]
Don't write Windows EOLs in CoreList.(pm|pod) when updating them on Windows
Steve Hay [Thu, 9 Aug 2012 12:54:10 +0000 (13:54 +0100)]
Don't write Windows EOLs in MANIFEST when sorting it on Windows
Steve Hay [Thu, 9 Aug 2012 12:41:46 +0000 (13:41 +0100)]
Upgrade Module-Build to 0.4002
Steve Hay [Thu, 9 Aug 2012 11:51:05 +0000 (12:51 +0100)]
Upgrade Encode to 2.45
Steve Hay [Thu, 9 Aug 2012 07:31:43 +0000 (08:31 +0100)]
Upgrade to Time-Local 1.2300
Files are already up to date, but Maintainers.pl is not.
Steve Hay [Thu, 9 Aug 2012 07:21:59 +0000 (08:21 +0100)]
Upgrade Module-Metadata to 1.000010
Steve Hay [Thu, 9 Aug 2012 07:18:27 +0000 (08:18 +0100)]
Upgrade Module-Load-Conditional to 0.52
Father Chrysostomos [Wed, 8 Aug 2012 17:00:52 +0000 (10:00 -0700)]
[perl #114018] Let eval close over stale vars in active sub
See also commit
cae5dbbe30.
These two lines should never produce different values:
print $x, "\n";
print eval '$x', "\n";
But they were producing different values if $x happened to have the
tale flag set. Even if my in false conditional is not supported (this
was the cause of the bug report), it should still work; and it is
not the only way to get a stale lexical in an active sub (just the
easiest way).
As long as the sub containing the eval is active, the eval should be
able to see the same variables, stale or not.
However, this does get a bit tricky in cases like this, which legiti-
mately warn (from t/lib/warnings/pad):
{
my $x = 1;
$y = \$x; # force abandonment rather than clear-in-place at scope exit
sub f2 { eval '$x' }
}
f2();
In this case the f2 sub does not explicitly close over the $x, so by
the time the eval is reached the ‘right’ $x is gone.
It is only in those cases where the sub containing the eval has
the stale variable in its own pad that we can safely ignore the
stale flag.
Father Chrysostomos [Wed, 8 Aug 2012 13:15:24 +0000 (06:15 -0700)]
B::Concise: Document formats
Father Chrysostomos [Wed, 8 Aug 2012 07:36:57 +0000 (00:36 -0700)]
Don’t let format arguments ‘leak out’ of formline
When parsing formats, the lexer invents tokens to feed to the parser.
So when the lexer dissects this:
format =
@<<<< @>>>>
$foo, $bar, $baz
.
The parser actually sees this (the parser knows that = . is like { }):
format =
; formline "@<<<< @>>>>\n", $foo, $bar, $baz;
.
The lexer makes no effort to make sure that the argument line is con-
tained within formline’s arguments. To make
{ do_stuff; $foo, bar }
work, the lexer supplies a ‘do’ before the block, if it is
inside a format.
This means that
$a, $b; $c, $d
feeds ($a, $b) to formline, wheras
{ $a, $b; $c, $d }
feeds ($c, $d) to formline. It also has various other
strange effects:
This script prints "# 0" as I would expect:
print "# ";
format =
@
(0 and die)
.
write
This one, locking parentheses, dies because ‘and’ has low precedence:
print "# ";
format =
@
0 and die
.
write
This does not work:
my $day = "Wed";
format =
@<<<<<<<<<<
({qw[ Sun 0 Mon 1 Tue 2 Wed 3 Thu 4 Fri 5 Sat 6 ]}->{$day})
.
write
You have to do this:
my $day = "Wed";
format =
@<<<<<<<<<<
({my %d = qw[ Sun 0 Mon 1 Tue 2 Wed 3 Thu 4 Fri 5 Sat 6 ]; \%d}->{$day})
.
write
which is very strange and shouldn’t even be valid syntax.
This does not work, because ‘no’ is not allowed in an expression:
use strict;
$::foo = "bar"
format =
@<<<<<<<<<<<
no strict; $foo
.
write;
Putting a block around it makes it work. Putting a semicolon before
‘no’ stop it from being a syntax error, but it silently does the
wrong thing.
I thought I could fix all these by putting an implicit do { ... }
around the argument line and removing the special-casing for an open-
ing brace, allowing anonymous hashrefs to work in formats, such
that this:
format =
@<<<< @>>>>
$foo, $bar, $baz
.
would turn into this:
format =
; formline "@<<<< @>>>>\n", do { $foo, $bar, $baz; };
.
But that will lead to madness like this ‘working’:
format =
@
}+do{
.
It would also stop lexicals declared in one format line from being
visible in another.
So instead this commit starts being honest with the parser. We still
have some ‘invented’ tokens, to indicate the start and end of a format
line, but now it is the parser itself that understands a sequence of
format lines, instead of being fed generated code.
So the example above is now presented to the parser like this:
format = ; FORMRBRACK
"@<<<< @>>>>\n" FORMLBRACK $foo, $bar, $baz ; FORMRBRACK
; .
Note about the semicolons: The parser expects to see a semicolon at
the end of each statement. So the lexer has to supply one before
FORMRBRACK. The final dot goes through the same code that handles
closing braces, which generates a semicolon for the same reason. It’s
easier to make the parser expect a semicolon before the final dot than
to change the } code in the lexer. We use the } code for . because it
handles the internal variables that keep track of how many nested lev-
els there, what kind, etc.
The extra ;FORMRBRACK after the = is there also to keep the lexer sim-
ple (ahem). When a newline is encountered during ‘normal’ (as opposed
to format picture) parsing inside a format, that’s when the semicolon
and FORMRBRACK are emitted. (There was already a semicolon there
before this commit. I have just added FORMRBRACK in the same spot.)
Father Chrysostomos [Wed, 8 Aug 2012 06:16:46 +0000 (23:16 -0700)]
B::Concise: Dump formats upon request
Father Chrysostomos [Wed, 8 Aug 2012 05:50:19 +0000 (22:50 -0700)]
B::Concise: Fix -nobanner
It was only working when B::Concise was passed a code ref.
Father Chrysostomos [Wed, 8 Aug 2012 05:45:08 +0000 (22:45 -0700)]
Increase $B::Concise::VERSION to 0.92
Father Chrysostomos [Tue, 7 Aug 2012 20:34:45 +0000 (13:34 -0700)]
toke.c: Remove unnecessary assignment
This was made unnecessary in commit
64a408986cf3d, which changed the
enclosing if block’s protasis to PL_expect == XBLOCK (and also made
sure that PL_expect is set to XBLOCK when a format’s special = delim-
iter is expected).
Father Chrysostomos [Tue, 7 Aug 2012 20:25:24 +0000 (13:25 -0700)]
Prevent double frees/crashes with format syntax errs
This was brought up in ticket #43425.
The new slab allocator for ops (
8be227ab5e) makes a CV responsible for
cleaning up its ops if it is freed prematurely (before the root is
attached).
Certain syntax errors involving formats can cause the parser to free
the CV owning an op when that op is on the PL_nextval stack.
This happens in these cases:
format =
@
use; format
strict
.
format =
@
;use
strict
.
format foo require bar
In the first two cases it is the line containing ‘strict’ that is
being interpreted as a format picture line and being fed through
force_next by S_scan_formline.
Then the error condition kicks in after the force, causing a
LEAVE_SCOPE in the parser (perly.c) which frees the sub owning the
const op for the format picture. Then a token with a freed op is fed
to the parser.
To make this clearer, when the second case above is parsed, the tokens
produced are as follows:
format =
[;] [formline] "@\n" [,]
; use [<word>]
[;] [formline] <freed>
[;] .
Notice how there is an implicit semicolon before each (implicit)
formline. Notice also how there is an implicit semicolon before the
final dot.
The <freed> thing represents the "strict\n" constant after it has been
freed. (-DT displays it as THING(opval=op_freed).)
When the implicit semicolon is emitted before a formline, the next two
tokens (formline and the string constant for this format picture line)
are put on to the PL_nextval stack via force_next.
It is when the implicit semicolon before "strict\n" is emitted that
the parser sees the error (there is only one path through the gram-
mar that uses the USE token, and it must have two WORDs following it;
therefore a semicolon after one WORD is an immediate error), calling
LEAVE_SCOPE, which frees the sub created by ‘use’, which owns the
const op on the PL_nextval stack containing the word "strict" and con-
sequently frees it.
I thought I could fix this by putting an implicit do { ... } around
the argument line. (This would fix another bug, whereby the argument
line in a format can ‘leak out’ of the formline(...).) But this does
not solve anything, as we end up with four tokens ( } ; formline
const ) on the PL_nextval stack when we emit the implicit semicolon
after ‘use’, instead of two.
format=
@
;use
strict
.
will turn into
format =
[;] [formline] "@\n" [,]
[do] [{] ; use [<word>] [;] [}]
[;] [formline] "strict\n"/<freed>
[;] .
It is when the lexer reaches "strict" that it will emit the semicolon
after the use. So we will be in the same situation as before.
So fixing the fact that the argument line can ‘leak out’ of the
formline and start a new statement won’t solve this particu-
lar problem.
I tried eliminating the LEAVE_SCOPE. (See
<https://rt.perl.org/rt3/Ticket/Display.html?id=43425#txn-273447>
where Dave Mitchell explains that the LEAVE_SCOPE is not strictly nec-
essary, but it is ‘still good to ensure that the savestack gets cor-
rectly popped during error recovery’.) That does not help, because
the lexer itself does ENTER/LEAVE to make sure form_lex_state and
lex_formbrack get restored properly after the lexer exits the format
(see
583c9d5cccf and
64a408986cf).
So when the final dot is reached, the ‘use’ CV is freed. Then an op
tree that includes the now-freed "strict\n" const op is passed to
newFORM, which tries to do op_free(block) (as of 3 commits ago; before
that the errors were more catastrophic), and ends up freeing an op
belonging to a freed slab.
Removing LEAVE_SCOPE did actually fix ‘format foo require bar’,
because there is no ENTER/LEAVE involved there, as the = (ENTER) has
not been reached yet. It was failing because ‘require bar’ would call
force_next for "bar", and then feed a REQUIRE token to the parser,
which would immediately see the error and call LEAVE_SCOPE (free-
ing the format), with the "bar" (belonging to the format’s slab)
still pending.
The final solution I came up with was to reuse an mechanism I came up
with earlier. Since the savestack may cause ops to outlive their CVs
due to SAVEFREEOP, opslab_force_free (called when an incomplete CV is
freed prematurely) will skip any op with o->op_savestack set. The
nextval stack can use the same flag. To make sure nothing goes awry
(we don’t want the same op on the nextval stack and the savestack at
the same time), I added a few assertions.
Father Chrysostomos [Tue, 7 Aug 2012 12:51:35 +0000 (05:51 -0700)]
toke.c: Remove TOKENTYPE_GVVAL
It has been unused since it was added in
bbf60fe6.
Father Chrysostomos [Tue, 7 Aug 2012 06:20:14 +0000 (23:20 -0700)]
toke.c: Add missing debug tokens
Father Chrysostomos [Tue, 7 Aug 2012 04:59:17 +0000 (21:59 -0700)]
Don’t create formats after compilation errors
Otherwise you can end up with a format that has nonsensical op tree,
but it gets attached and you can call it anyway.
Father Chrysostomos [Tue, 7 Aug 2012 00:52:10 +0000 (17:52 -0700)]
Add freed ops to PL_op_(name|desc)
This is useful for debugging, especially with -DT.
Father Chrysostomos [Mon, 6 Aug 2012 21:34:25 +0000 (14:34 -0700)]
toke.c: Fix confused interp-in-format parsing
The lexer keeps track of how many levels of brackets it is currently
in (including the = and . delimiting formats, which count as brackets)
in PL_lex_brackets.
When parsing a format, the lexer sets lex_formbrack to the form’s
bracket number. Whitespace parsing takes note of this, and avoids
reading beyond the end of the current line inside a format, unless it
is in an inner bracketed construct. So this:
format =
@
$a
+ $b
.
treats ‘ + $b’ as a literal string, which becomes the second line
of output.
But this:
format =
@
{ $a
+ $b }
.
treats the whole {...} block as the argument for the first format pic-
ture line.
For interpolating (i.e., quoted) constructs, the lexer temporarily
resets the bracket count. This is so that when it reaches the final
brace in "${...}foo()" it can parse the rest (foo()) as a constant,
rather than a sub call.
When the bracket count was reset, lex_formbrack was left as it was,
resulting in confusion:
# ok
format =
@
${; use
strict}
.
# also ok
format =
@
"${; {use
strict} }"
.
# syntax error (or crash)
format =
@
"${; use
strict }"
.
# crash
format =
@
"${ use
strict }"
.
When the bracket count is localised and reset at the start of a quoted
construct, we now do the same with lex_formbrack.
Steve Hay [Wed, 8 Aug 2012 13:15:41 +0000 (14:15 +0100)]
Upgrade IO-Compress to 2.055
Steve Hay [Wed, 8 Aug 2012 13:09:53 +0000 (14:09 +0100)]
Upgrade Compress-Raw-Bzip2 and Compress-Raw-Zlib to 2.055
Paul [Sat, 4 Aug 2012 15:53:10 +0000 (16:53 +0100)]
MAgic for Zlib & Bzip2
Tony Cook [Wed, 8 Aug 2012 11:29:29 +0000 (13:29 +0200)]
ignore PERL_XMLDUMP when tainting
In theory this is a security issue, but from discussion on the
security list that the system perl (or the perl used for anything
critical) is wildly unlikely to have been built with -Dmad.
Steve Hay [Wed, 8 Aug 2012 08:22:40 +0000 (09:22 +0100)]
Don't try to set perms in makerel on files which no longer exist
Steve Hay [Wed, 8 Aug 2012 08:12:00 +0000 (09:12 +0100)]
Add /Module/CoreList/TieHashDelta.pm to lib/.gitignore
Steve Hay [Wed, 8 Aug 2012 07:47:51 +0000 (08:47 +0100)]
Allow .tgz as a recognized suffix in core-cpan-diff
otherwise Memoize-1.03.tgz gets skipped.
Steve Hay [Wed, 8 Aug 2012 07:44:59 +0000 (08:44 +0100)]
Note that core-cpan-diff always requires diff(1), even without --diff
Steve Hay [Tue, 7 Aug 2012 07:53:42 +0000 (08:53 +0100)]
Use _strtoi64, _strtoui64 and _atoi64 for WIN64 VC++ builds
VC++ has QUADKIND == QUAD_IS___INT64 so the "secret handshakes" were only
getting used for WIN64 GCC builds (where QUADKIND == QUAD_IS_LONG_LONG).
Steve Hay [Tue, 7 Aug 2012 07:49:09 +0000 (08:49 +0100)]
VC++ has QUADKIND == QUAD_IS___INT64 so we might as well make use of it
- Use I64/UI64 suffixes rather than I64TYPE/U64TYPE casts for
INT64_C/UINT64_C, not just when _WIN64 is defined
- Use UI64 suffix rather than UL for U64_CONST
Steffen Mueller [Tue, 7 Aug 2012 06:48:08 +0000 (08:48 +0200)]
Data::Dumper: Updated changelog
Steffen Mueller [Tue, 7 Aug 2012 06:45:56 +0000 (08:45 +0200)]
Data::Dumper: Fix tests for pure-Perl implementation
Father Chrysostomos fixed vstring handling in both XS and pure-Perl
implementations of Data::Dumper in
de5ef703c7d8db6517e7d56d9c018d3ad03f210e.
He also updated the tests for the default XS implementation, but it seems
that he missed the test changes necessary for the pure-Perl implementation
which now also does the right thing.
Steffen Mueller [Mon, 6 Aug 2012 18:14:47 +0000 (20:14 +0200)]
Delta entry for Data::Dumper optimization
Karl Williamson [Mon, 6 Aug 2012 17:27:53 +0000 (11:27 -0600)]
Data::Dumper: Fix to use with earlier Perls
Commit
4b88fb76efce8c436e63b907c9842345d4fa77c7 broke Data::Dumper when
it is used on Perl versions earlier than v5.16.
Father Chrysostomos [Mon, 6 Aug 2012 21:03:31 +0000 (14:03 -0700)]
cv.h: macro parentheses
I forgot about -- having higher precedence than unary *. And then
I forgot to test it, convincing myself I had. :-(
Father Chrysostomos [Mon, 6 Aug 2012 16:48:07 +0000 (09:48 -0700)]
Nested formats
Are nested formats a good idea? Probably not. But the only rea-
son they don’t work is that the parser becomes confused and loses
track of where it is.
And it would be nice to have some consistency. I can put sub defini-
tions inside a format:
format =
@
;sub foo {
bar
}
.
and:
format =
@
{
sub foo {
bar
}
}
.
so why not these?
format foo =
@
;format bar =
@
.
.
format foo =
@
{
format bar =
@
.
}
.
In perl 5.17.2 and earlier, you can nest formats, but, due to the
parser being confused, the outer format must be terminated with }
instead of a dot. That stopped working with commit
7c70caa5333.
format =
@<<<<<<<<<<<<<<<
"Just another"; format STDERR =
@<<<<<<<<<<<<<<<
"Perl hacker"
.
}
write; select STDERR;
write;
Father Chrysostomos [Mon, 6 Aug 2012 15:41:07 +0000 (08:41 -0700)]
toke.c: move leftbracket label
The only code path that goesto leftbracket was precede by s--.
The first thing leftbracket did was s++.
We can simplify the code slighty by moving the label down one
statement and not s--ing before goingto leftbracket.
Father Chrysostomos [Mon, 6 Aug 2012 15:38:28 +0000 (08:38 -0700)]
[perl #114040] Parse formats in interpolating constructs
For re-evals, this is something that broke recently, post-5.16 (the
jumbo fix). For other interpolating constructs, this has never
worked, as far as I can tell.
The lexer was losing track of PL_lex_state (aka PL_parser->lex_state)
when parsing formats. Usually, the state alternates between
LEX_FORMLINE (a picture line) and LEX_NORMAL (an argument line), but
the LEX_NORMAL should actually be whatever the state was before the
format started.
This commit adds a new parser member to track the ‘normal’ state when
parsing a format.
It also tweaks S_scan_formline to handle multi-line buffers outside of
string eval (such as happens in interpolating constructs).
That bufend assignment that is removed as a result is not necessary as
of
a0d0e21ea6ea (perl 5.000). That very commit added a bufend assign-
ment after the sv_gets (later filter_gets; later lex_next_chunk) fur-
ther down in the loop in scan_formline.
Steve Hay [Mon, 6 Aug 2012 17:10:37 +0000 (18:10 +0100)]
Update notes on which Cygwin tools are needed to make a release on Windows
(corelist.pl no longer uses curl, diffstat is no longer required to produce
the list of acknowledgements and 7z should be used to create smaller
tarballs. Also, core-cpan-diff no longer uses wget.)
Steve Hay [Mon, 6 Aug 2012 08:02:50 +0000 (09:02 +0100)]
Restore parts of config_H.gc lost by regen_config_h
Steve Hay [Mon, 6 Aug 2012 07:58:07 +0000 (08:58 +0100)]
Run regen_config_h for GCC
A few new symbols are added but no existing ones are changed.
Steve Hay [Mon, 6 Aug 2012 07:57:44 +0000 (08:57 +0100)]
Run regen_config_h for VC++
A few new symbols are added but no existing ones are changed.
Steffen Mueller [Fri, 3 Aug 2012 05:52:21 +0000 (07:52 +0200)]
Data::Dumper: Import more change history
Father Chrysostomos [Mon, 6 Aug 2012 05:24:09 +0000 (22:24 -0700)]
Forbid braces as format delimiters
As long the argument line is not missing, braces can be used
for a format:
# valid
format foo {
@<<<
$a
}
but if the argument line is missing, you get a syntax error:
# invalid
format foo {
@<<<
}
and also if the name is missing:
# invalid
format {
@<<<
$a
}
If you use = then you can use a closing brace to terminate the format,
but only if the argument line is missing:
# valid, but useless
format =
@<<<
}
# invalid
format =
@<<<
$a
}
In commit
79072805 (perl 5.0 alpha 20), the lexer was changed to lie
to the parser about formats’ = and . delimiters, pretending they are
actually { and }, because the bracket-handling code takes care of all
the scoping issues (well, most of them).
Unfortunately, this implementation detail can leak through, but it is
far from consistent, as shown above.
Fixing this makes it easier to fix other bugs.
We do this by recording the fact that we are dealing with a format
token before jumping to the bracket code. And we feed format-specific
tokens to the parser in that case, so it can’t get fake brackets and
real brackets mixed up.
Father Chrysostomos [Mon, 6 Aug 2012 00:22:29 +0000 (17:22 -0700)]
assert_(...)
This new macro expands to ‘assert(...),’ (with a trailing comma) under
debugging builds; the empty string otherwise.
It allows for the removal of some #ifdef DEBUGGINGs, which could not be
avoided otherwise.
Dave Rolsky [Mon, 6 Aug 2012 02:00:49 +0000 (21:00 -0500)]
Fix a typo in perlootut