From: Karl Williamson Date: Mon, 13 Feb 2012 16:54:22 +0000 (-0700) Subject: mktables: viacode() return unparenthesized names for 4 controls X-Git-Tag: accepted/trunk/20130322.191538~688 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=898b2fa7ca685c63900bc063ed519d98af0db541;p=platform%2Fupstream%2Fperl.git mktables: viacode() return unparenthesized names for 4 controls This commit changes the viacode() returned name for four control characters, as follows: Code point Old Name New Name U+000A LINE FEED (LF) LINE FEED U+000C FORM FEED (FF) FORM FEED U+000D CARRIAGE RETURN (CR) CARRIAGE RETURN U+0085 NEXT LINE (NEL) NEXT LINE Only the return from viacode is affected. All the names are accepted as input, as they always have been. Unicode 6.1 now has official names for all the controls, and the new names match those. The old names were the ones that were recommended by TR18 prior to 6.1, and still are, sort of. This change uses the official names in preference to the TR18 ones. We probably wouldn't bother except that the old names were problematic--the only names in the whole universe of names containing parentheses, and not matching traditional usage. The new names have always been accepted as inputs by Perl. I actually doubt that Unicode ever grokked that they were recommending these ugly names. and they haven't paid much attention to TR18 anyway, breaking it in version 6.0 by encoding one of the recommended names (BELL) as an official name for another code point, and without realizing it. TR18 now is in limbo, still wrongly recommending BELL, with a rewrite being promised for many months now. It's unclear what will happen with it. It was agreed on p5p to go with the cleaner, now official names, instead of the older, likely obsolete, TR18 names. I did a search of CPAN; it was unclear if this change, (which again is only for viacode()) mattered to any code there or not. There were a few instances of the old names, but none of those were apparently associated with viacode(). --- diff --git a/lib/charnames.t b/lib/charnames.t index 9d37daa..09a4314 100644 --- a/lib/charnames.t +++ b/lib/charnames.t @@ -313,6 +313,19 @@ is("\N{BOM}", chr(0xFEFF), 'Verify "\N{BOM}" is correct'); is(charnames::viacode(0xFEFF), "ZERO WIDTH NO-BREAK SPACE", 'Verify viacode(0xFEFF) is correct'); +# These test that the changes to these in 6.1 are recognized. (The double +# test of using viacode and vianame is less than optimal as two errors could +# cancel each other out, but later each is tested individually, and this +# sidesteps and EBCDIC issues. +is(charnames::viacode(charnames::vianame("CR")), "CARRIAGE RETURN", + 'Verify viacode(vianame("CR")) is "CARRIAGE RETURN"'); +is(charnames::viacode(charnames::vianame("LF")), "LINE FEED", + 'Verify viacode(vianame("LF")) is "LINE FEED"'); +is(charnames::viacode(charnames::vianame("FF")), "FORM FEED", + 'Verify viacode(vianame("FF")) is "FORM FEED"'); +is(charnames::viacode(charnames::vianame("NEL")), "NEXT LINE", + 'Verify viacode(vianame("NEL")) is "NEXT LINE"'); + { use warnings; cmp_ok(ord("\N{BOM}"), '==', 0xFEFF, 'Verify \N{BOM} is correct'); diff --git a/lib/unicore/mktables b/lib/unicore/mktables index 64f7522..6fea440 100644 --- a/lib/unicore/mktables +++ b/lib/unicore/mktables @@ -10334,9 +10334,8 @@ END # Some code points in this file have the pseudo-name # '', but the official name for such ones is the null - # string. For charnames.pm, we use the Unicode version 1 name - $fields[$NAME] = ""; - $fields[$CHARNAME] = $fields[$UNICODE_1_NAME]; + # string. + $fields[$NAME] = $fields[$CHARNAME] = ""; # We had better not be in between range lines. if ($in_range) { @@ -12511,7 +12510,6 @@ sub compile_perl() { if (defined $alias) { push @composition, 'Name_Alias'; $perl_charname->set_proxy_for('Name_Alias'); - my $unicode_1 = property_ref('Unicode_1_Name'); my %abbreviations; # Add each entry in Name_Alias to Perl_Charnames. Where these go with @@ -12548,8 +12546,6 @@ sub compile_perl() { next; } elsif ($type eq 'control') { - my $unicode_1_value = $unicode_1->value_of($code_point); - next if $unicode_1_value eq $value; $replace_type = $MULTIPLE_AFTER; } else { @@ -12561,6 +12557,32 @@ sub compile_perl() { $perl_charname->add_duplicate($code_point, $value, Replace => $replace_type); } + # Now add the Unicode_1 names for the controls. These come after the + # official names, as they are only recommended (by TR18; unclear as of + # this writing if that recommendation will be withdrawn, but if it is, + # we want to add them anyway for backwards compatibility). Only a few + # differ from the official names. + foreach my $range (property_ref('Unicode_1_Name')->ranges) { + my $code_point = $range->start; + my $unicode_1_value = $range->value; + next if $unicode_1_value eq ""; # Skip if name doesn't exist. + + if ($code_point != $range->end) { + Carp::my_carp_bug("Bad News. Expecting only one code point in the range $range. Just to keep going, using only the first code point;"); + } + + # To handle EBCDIC, we don't hard code in the code points of the + # controls; instead realizing that all of them are below 256. + last if $code_point > 255; + + # We only add in the controls. + next if $gc->value_of($code_point) ne 'Cc'; + + # This won't add an exact duplicate. + $perl_charname->add_duplicate($code_point, $unicode_1_value, + Replace => $MULTIPLE_AFTER); + } + # Now that have everything added, add in abbreviations after # everything else. foreach my $value (keys %abbreviations) { diff --git a/pod/perldelta.pod b/pod/perldelta.pod index 07a684a..9a8f9cf 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -86,10 +86,25 @@ New aliases (synonyms) have been defined for many property values; these, along with the previously existing ones, are all cross indexed in L. -The return value of C is affected by other changes. -One of these is that the preferred name (which is what C -returns) for the character at U+2118 has been changed from SCRIPT CAPITAL P -to WEIERSTRASS ELLIPTIC FUNCTION. But most of these changes are the +The return value of C is affected by other +changes: + + Code point Old Name New Name + U+000A LINE FEED (LF) LINE FEED + U+000C FORM FEED (FF) FORM FEED + U+000D CARRIAGE RETURN (CR) CARRIAGE RETURN + U+0085 NEXT LINE (NEL) NEXT LINE + U+008E SINGLE-SHIFT 2 SINGLE-SHIFT-2 + U+008F SINGLE-SHIFT 3 SINGLE-SHIFT-3 + U+0091 PRIVATE USE 1 PRIVATE USE-1 + U+0092 PRIVATE USE 2 PRIVATE USE-2 + U+2118 SCRIPT CAPITAL P WEIERSTRASS ELLIPTIC FUNCTION + +Perl will accept any of these names as input, but +C now returns the new name of each pair. The +change for U+2118 is considered by Unicode to be a correction, that is +the original name was a mistake (but again, it will remain forever valid +to use it to refer to U+2118). But most of these changes are the fallout of the mistake Unicode 6.0 made in naming a character used in Japanese cell phones to be "BELL", which conflicts with the long standing industry use of (and Unicode's recommendation to use) that name @@ -105,21 +120,23 @@ this character, and not U+0007. Unicode has taken steps to make sure that this sort of mistake does not happen again. The Standard now includes all the generally accepted names and abbreviations for control characters, whereas previously it -didn't. This means that all the names that Perl had previously -deprecated (except BELL) are no longer deprecated, such as FILE -SEPARATOR. Also, the names for four rarely used characters are subtly -different (a hyphen instead of a space) than before: - - Code point Old Name New Name - U+008E SINGLE-SHIFT 2 SINGLE-SHIFT-2 - U+008F SINGLE-SHIFT 3 SINGLE-SHIFT-3 - U+0091 PRIVATE USE 1 PRIVATE USE-1 - U+0092 PRIVATE USE 2 PRIVATE USE-2 - -Perl will accept either name as input, but C now -returns the new name. - -Additional name abbreviations are accepted: +didn't (though there were recommended names for most of them, which Perl +used). This means that most of those recommended names are now +officially in the Standard. Unicode did not recommend names for the +four code points listed above between U+008E and U+008F, and in +standardizing them Unicode subtly changed the names that Perl had +previously given them, by replacing the final blank in each name by a +hyphen. Unicode also officially accepts names that Perl had deprecated, +such as FILE SEPARATOR. Now the only deprecated name is BELL. +Finally, Perl now uses the new official names instead of the old +recommended names for the first four code points in the list above (the +ones which have the parentheses in them). + +Now that the names have been placed in the Unicode standard, these kinds +of changes should not happen again, though corrections, such as to +U+2118, are still possible. + +Unicode also added some name abbreviations, which Perl now accepts: SP for SPACE; TAB for CHARACTER TABULATION; NEW LINE, END OF LINE, NL, and EOL for LINE FEED;