platform/upstream/coreutils.git
13 years agosort: document --compress reaper fixes
Paul Eggert [Tue, 14 Dec 2010 18:21:49 +0000 (10:21 -0800)]
sort: document --compress reaper fixes

* NEWS: Document the --compress reaper fixes installed yesterday.

13 years agotests: default to /tmp as the temporary directory
Paul Eggert [Tue, 14 Dec 2010 18:07:36 +0000 (10:07 -0800)]
tests: default to /tmp as the temporary directory

* tests/check.mk (TESTS_ENVIRONMENT): Default TMPDIR to /tmp,
rather than to the working directory; this is more common in
practice, which makes the tests more real-worldish; and it is
often faster.  Also, it avoids some problems with NFS cleanups.
* tests/misc/sort-compress: Remove unnecessary code setting TMPDIR.
* tests/misc/sort-compress-proc: Likewise.  Do the final sleep
only if TMPDIR is relative, which should be rarely given the
change to TESTS_ENVIRONMENT.

13 years agosort: fix some --compress reaper bugs
Paul Eggert [Tue, 14 Dec 2010 07:23:17 +0000 (23:23 -0800)]
sort: fix some --compress reaper bugs

* src/sort.c (uintptr): New type.
(enum procstate, struct procnode, update_proc): Remove.
(proctab_hasher, proctab_comparator, register_proc, wait_proc):
(reap_some): The proctab is now simply a hash of process-IDs
rather than of pointers to objects with reference counts and
states; this is smaller and faster and easier to understand.
(nprocs): Now pid_t, not size_t, since one cannot have more than
PID_MAX children.
(reap): If the argument is -1, wait; if 0 (a new value), do not.
Delete pid from proctab as needed.  Ignore children that are not
in proctab, as they are from the program that exec'ed us and are
irrelevant to our success or failure.
(delete_proc, reap_all): New functions.
(open_temp): Register the child.
(sort): Clean up all children afterwards; without this patch,
'sort' sometimes missed failures in children due to race conditions.
* tests/Makefile.am (TESTS): Add misc/sort-compress-proc.
* tests/misc/sort-compress-proc: New file, to test for the
bugs fixed above.

13 years agotests: typo fix
Paul Eggert [Mon, 13 Dec 2010 18:02:06 +0000 (10:02 -0800)]
tests: typo fix

* tests/misc/sort-stale-thread-mem: Fix typo in comment.

13 years agotests: tweak basic-1 to use warn_ rather than literal "exit 77"
Jim Meyering [Mon, 13 Dec 2010 07:19:12 +0000 (08:19 +0100)]
tests: tweak basic-1 to use warn_ rather than literal "exit 77"

* tests/install/basic-1 (just_built_dd): Use warn_, rather than
cat and exit 77.

13 years agotests: mark new test as very expensive
Jim Meyering [Mon, 13 Dec 2010 07:07:25 +0000 (08:07 +0100)]
tests: mark new test as very expensive

* tests/misc/sort-stale-thread-mem: Don't initialize fail=0 here;
that is done in init.sh.  This avoids a syntax-check failure.
Invoke "Exit $fail" at end, too.
Mark as a very expensive test.

13 years agotests: test for access to stale thread memory
Paul Eggert [Sun, 12 Dec 2010 21:38:19 +0000 (13:38 -0800)]
tests: test for access to stale thread memory

* tests/misc/sort-stale-thread-mem: New tests.
* tests/Makefile.am (TESTS): Add it.

13 years agotests: avoid FP failure when run under valgrind
Jim Meyering [Sat, 11 Dec 2010 17:44:57 +0000 (18:44 +0100)]
tests: avoid FP failure when run under valgrind

* tests/misc/printenv: Filter out LD_PRELOAD, as the comment
said, not LD_LIBRARY.

13 years agosort: avoid segfault when using two or more threads
Jim Meyering [Sat, 11 Dec 2010 10:38:21 +0000 (11:38 +0100)]
sort: avoid segfault when using two or more threads

This change does not fix the actual bug.  That was done by commit
c9db0ac6, "sort: preallocate merge tree nodes to heap".  The fix
was to store each "node" structure on the heap, not on the stack.
Otherwise, a node from one thread's stack could be used in another
thread after the first thread had expired (via pthread_join).
This bug was very hard to trigger when using spinlocks, but
easier once we began using mutexes.
* NEWS (Bug fixes): Mention it.
For details, see http://debbugs.gnu.org/7597.

13 years agosort: syntax cleanup
Jim Meyering [Sat, 11 Dec 2010 10:29:38 +0000 (11:29 +0100)]
sort: syntax cleanup

* src/sort.c (xfopen, debug_key, sortlines, sort, main): Adjust
formatting: fix misplaced braces, use consistent spacing,
split a 2-stmt line.

13 years agosort: integer overflow checks in thread counts, etc.
Paul Eggert [Sat, 11 Dec 2010 08:27:05 +0000 (00:27 -0800)]
sort: integer overflow checks in thread counts, etc.

* src/sort.c (specify_nthreads, merge_tree_init, init_node):
(queue_init, sortlines, struct thread_args, sort, main):
Use size_t, not unsigned long int, for thread counts, since thread
counts are now used to compute sizes.
(specify_nthreads): Check for size_t overflow.
(merge_tree_init, sort): Shorten name of local variable, for
readability.
(merge_tree_init): Move constants next to each other in product,
so that the constant folding is easier to see.
(init_node): Now static.  Add 'restrict' only where it might
be helpful for compiler optimization.
(queue_init): 2nd arg is now nthreads, not "reserve", which is
a bit harder to follow.  All uses changed.
(struct thread_args): Rename lo_child to is_lo_child, so that
it's obvious to the reader when we're talking about this boolean
as opposed to the new lo_child member of the other structure.
All uses changed.
(sort): Remove unused local variable end_node.
(main): Don't allow large thread counts to cause undefined behavior
later, due to integer overflow.

13 years agosort: preallocate merge tree nodes to heap.
Chen Guo [Fri, 10 Dec 2010 21:13:36 +0000 (13:13 -0800)]
sort: preallocate merge tree nodes to heap.

* src/sort.c: (merge_tree_init) New function. Allocates memory for
merge tree nodes.
(merge_tree_destory) New function.
(init_node) New function.
(sortlines) Refactor node creation code to init_node. Remove now
superfluous arguments. All callers changed.
(sort) Initialize/destory merge tree. Refactor root node creation
to merge_tree_init.

13 years agosort: comment fix
Paul Eggert [Sat, 11 Dec 2010 04:52:04 +0000 (20:52 -0800)]
sort: comment fix

* src/sort.c: Comment fix re spin locks.

13 years agosort: use mutexes, not spinlocks (avoid busy loop on blocked output)
Chen Guo [Mon, 6 Dec 2010 08:15:42 +0000 (00:15 -0800)]
sort: use mutexes, not spinlocks (avoid busy loop on blocked output)

Running a command like this on a multi-core system
  sort < big-file | less
would peg all processors at near 100% utilization.
* src/sort.c: (struct merge_node) Change member lock to mutex.
All uses changed.
* tests/Makefile.am (XFAIL_TESTS): Remove definition, now that
this test passes once again.  I.e., the sort-spinlock-abuse test
no longer fails.
* NEWS (Bug reports): Mention this.
Reported by DJ Lucas in http://debbugs.gnu.org/7489.

13 years agosplit: fix a case where --elide-empty causes invalid chunking
Pádraig Brady [Wed, 8 Dec 2010 08:33:15 +0000 (08:33 +0000)]
split: fix a case where --elide-empty causes invalid chunking

When -n l/N is used and long lines are present that both
span partitions and multiple buffers, one would get
inconsistent chunk sizes.

* src/split.c (main): Add a new undocumented ---io-blksize option
to support full testing with varied buffer sizes.
(cwrite): Refactor most handling of --elide-empty to here.
(bytes_split): Remove handling of --elide-empty.
(lines_chunk_split): Likewise.  The specific issue here
was the first handling of elide_empty_files interfered
with the replenishing of the input buffer.
* test/misc/split-lchunk: Add -e and the new ---io-blksize
combinations to the test.

13 years agotests: remove useless definition of $SORT in sort-compress
Jim Meyering [Sun, 5 Dec 2010 16:07:35 +0000 (17:07 +0100)]
tests: remove useless definition of $SORT in sort-compress

* tests/misc/sort-compress (SORT): Remove unused definition.

13 years agotests: make it harder to lose a race in spinlock-abuse
Jim Meyering [Sat, 4 Dec 2010 10:52:21 +0000 (11:52 +0100)]
tests: make it harder to lose a race in spinlock-abuse

* tests/misc/sort-spinlock-abuse: On a busy system, with only 12
pauses of length 0.1 seconds, the buggy (busy-spinlock blocked)
sort would fail to accumulate 1 second of CPU time, and hence
would mistakenly pass.  Increase from 12 to 50.

13 years agosort: merge_queue -> queue
Paul Eggert [Fri, 3 Dec 2010 23:39:50 +0000 (15:39 -0800)]
sort: merge_queue -> queue

* src/sort.c (struct thread_args, sortlines_thread, sortlines, sort):
Rename "merge_queue" to "queue", for consistency with other functions
that just use the name "queue" for these things.

13 years agosort: clarify queue_check_insert
Paul Eggert [Fri, 3 Dec 2010 23:23:43 +0000 (15:23 -0800)]
sort: clarify queue_check_insert

* src/sort.c (queue_check_insert): Clarify body a bit, and remove
no-longer-needed comment.

13 years agosort: fix problems with merge node dest pointer
Paul Eggert [Fri, 3 Dec 2010 23:11:46 +0000 (15:11 -0800)]
sort: fix problems with merge node dest pointer

* src/sort.c (mergelines_node): Return void, not size_t.  All
callers changed.  Change *node->dest here, not in caller.
Do not change node->dest: it's not needed and could cause problems
on (mostly theoretical) hosts that do not allow adding integers to
null pointers.
(queue_check_insert_parent): Omit MERGED parameter; no longer needed.
All callers changed.

13 years agosort: simplify write_unique
Paul Eggert [Fri, 3 Dec 2010 23:04:31 +0000 (15:04 -0800)]
sort: simplify write_unique

* src/sort.c (write_unique): Simplify slightly so that there is
just one call to write_line, not two.

13 years agosort: put queue arg first
Paul Eggert [Fri, 3 Dec 2010 23:01:21 +0000 (15:01 -0800)]
sort: put queue arg first

* src/sort.c (queue_check_insert, queue_check_insert_parent): Make
the queue arg first, for consistency with other functions such as
queue_insert that put the queue arg first.  Rename from
check_insert and update_parent, respectively.  All callers
changed.

13 years agosort: tune struct_merge_node slightly
Paul Eggert [Fri, 3 Dec 2010 22:39:23 +0000 (14:39 -0800)]
sort: tune struct_merge_node slightly

* src/sort.c (struct merge_node): 'lock' is now the actual lock,
not a pointer to the lock; there's no need for indirection here.
Make 'level' unsigned int instead of size_t, since it is a
bit-shift count; also, move it next to a bool so that it's more
likely to take less space.  All uses changed.
(sortlines, sort): Spell out initialization instead of using an
initializer.  This makes the initializer a bit easier to understand,
and avoids unnecessary stores into the spin lock.

13 years agosort: Clarify comments
Paul Eggert [Fri, 3 Dec 2010 22:27:02 +0000 (14:27 -0800)]
sort: Clarify comments

* src/sort.c: Improve comments a bit.

13 years agotests: cleanup rm -rf fails under NFS
Paul Eggert [Fri, 3 Dec 2010 19:08:48 +0000 (11:08 -0800)]
tests: cleanup rm -rf fails under NFS

This problem was observed on RHEL 5.5 x86-64 when running as a
client of a NetApp FAS2050.
* tests/cp/cp-mv-backup: Don't leave a file descriptor open to
a file in a directory that will be cleaned up with "rm -rf".
Under NFS, when the rm unlinks that file, it is instead renamed
to .nfsXXXX and then rm cannot remove the parent directory,
and the test fails.
* tests/cp/same-file: Likewise.

13 years agocp: fix bug with fine-grained src to nearby coarse-grained dest
Paul Eggert [Fri, 3 Dec 2010 17:55:48 +0000 (09:55 -0800)]
cp: fix bug with fine-grained src to nearby coarse-grained dest

The actual fix is in gnulib's lib/utimecmp.c.
* NEWS: Document fix.

13 years agobuild: update gnulib submodule to latest
Paul Eggert [Fri, 3 Dec 2010 17:52:44 +0000 (09:52 -0800)]
build: update gnulib submodule to latest

13 years agosort: fix bug on 64-bit hosts with at least 32768 processors
Paul Eggert [Thu, 2 Dec 2010 05:50:00 +0000 (21:50 -0800)]
sort: fix bug on 64-bit hosts with at least 32768 processors

* src/sort.c (MAX_MERGE): Avoid integer overflow when on a machine
with (say) 32-bit int and 64-bit size_t and when level == 15.
Without this fix, on such a machine with 32768 or more processors,
the level computation could overflow on large input, and this
would result in division by zero.

13 years agotests: add test for parallel sort -u segfault bug
Jim Meyering [Sun, 28 Nov 2010 11:59:38 +0000 (12:59 +0100)]
tests: add test for parallel sort -u segfault bug

* tests/misc/sort-unique-segv: New file.
* tests/Makefile.am (TESTS): Add it.

13 years agosort -u: fix a thread-race pointer corruption bug
Paul Eggert [Tue, 30 Nov 2010 21:30:12 +0000 (22:30 +0100)]
sort -u: fix a thread-race pointer corruption bug

* src/sort.c (write_unique): Save the entire "struct line", not
just a pointer to one.  Otherwise, with a multi-thread run,
sometimes, with some inputs, fillbuf would would win a race
and clobber a "saved->text" pointer in one thread just before
it was dereferenced in a comparison in another thread.
* NEWS (Bug fixes): Mention it.

13 years agotests: don't let the OMP_NUM_THREADS envvar affect sort tests
Jim Meyering [Sat, 27 Nov 2010 14:58:51 +0000 (15:58 +0100)]
tests: don't let the OMP_NUM_THREADS envvar affect sort tests

* tests/envvar-check (vars): Add OMP_NUM_THREADS.

13 years agotests: test for parallel sort spinlock abuse
Jim Meyering [Sat, 27 Nov 2010 14:50:43 +0000 (15:50 +0100)]
tests: test for parallel sort spinlock abuse

* tests/misc/sort-spinlock-abuse: New file.
* tests/Makefile.am (TESTS): Add it.
(XFAIL_TESTS): Mark this as an expected-to-fail (for now) test.

13 years agotsort: suppress a valgrind memory leak warning
Pádraig Brady [Thu, 25 Nov 2010 19:16:00 +0000 (19:16 +0000)]
tsort: suppress a valgrind memory leak warning

* src/tsort.c (tsort): Unconditionally invoking the free()
doesn't increase scalability, so do it only with -Dlint

13 years agomaint: update the valgrind support script
Pádraig Brady [Wed, 24 Nov 2010 07:50:01 +0000 (07:50 +0000)]
maint: update the valgrind support script

* README-valgrind: Include the "noinst" programs in
those wrapped by valgrind.  Update $PATH in check.mk
rather than Makefile.am.  Make wrapper scripts
work when suppressions not setup.  Keep lines < 80 chars.

13 years agocp: give a better diagnostic for nonexistent dest/
Paul Eggert [Mon, 22 Nov 2010 02:50:41 +0000 (18:50 -0800)]
cp: give a better diagnostic for nonexistent dest/

This patch was written by Jim Meyering and myself.
* src/copy.c (copy_reg): Turn EISDIR to ENOTDIR to improve the
quality of diagnostics for commands like "cp a nosuch/".  Reported
by Марк Коренберг and Alan Curry in the thread starting at:
http://lists.gnu.org/archive/html/bug-coreutils/2010-11/msg00178.html
* THANKS: Update.
* tests/mv/trailing-slash: Add a test.

13 years agosplit: add --number to generate a particular number of files
Chen Guo [Fri, 8 Jan 2010 11:42:27 +0000 (03:42 -0800)]
split: add --number to generate a particular number of files

* src/split.c (usage, long_options, main): New options --number,
--unbuffered, --elide-empty-files.
(set_suffix_length): New function to auto increase suffix length
to handle a specified number of files.
(create): New function.  Refactored from cwrite() and ofile_open().
(bytes_split): Add max_files argument to support byte chunking.
(lines_chunk_split): New function.  Split file into chunks of lines.
(bytes_chunk_extract): New function.  Extract a chunk of file.
(of_info): New struct.  Used by functions lines_rr and ofile_open
to keep track of file descriptors associated with output files.
(ofile_open): New function.  Shuffle file descriptors when there
are more output files than available file descriptors.
(lines_rr): New function to distribute lines round-robin to files.
(chunk_parse): New function.  Parses K/N syntax.
* tests/misc/split-bchunk: New test for byte chunking.
* tests/misc/split-lchunk: New test for line delimited chunking.
* tests/misc/split-rchunk: New test for round-robin chunking.
* tests/Makefile.am: Reference new tests.
* tests/misc/split-fail: Add failure scenarios for new options.
* tests/misc/split-l: Fix a typo. s/ln/split/.
* doc/coreutils.texi (split invocation): Document --number.
* NEWS: Mention the new feature.
* .mailmap: Map new email address for shortlog.

Signed-off-by: Pádraig Brady <P@draigBrady.com>
13 years agobuild: update gnulib to fix a syntax error in a test
Jim Meyering [Fri, 19 Nov 2010 13:29:32 +0000 (14:29 +0100)]
build: update gnulib to fix a syntax error in a test

13 years agomaint: avoid NEWS-related syntax-check failure
Jim Meyering [Thu, 18 Nov 2010 20:16:27 +0000 (21:16 +0100)]
maint: avoid NEWS-related syntax-check failure

* NEWS: Add 2nd blank line to separate latest changes from
those of 8.7, to avoid syntax-check failure.

13 years agood: fix bugs in displaying floating-point values
Paul Eggert [Thu, 18 Nov 2010 17:50:49 +0000 (09:50 -0800)]
od: fix bugs in displaying floating-point values

* NEWS: Describe patch.
* bootstrap.conf (gnulib_modules): Add ftoastr.
* src/od.c: Include ftoastr.h, not float.h.
(FLT_DIG, DBL_DIG): Remove.  No need to verify LDBL_DIG.
(FMT_BYTES_ALLOCATED): No need to worry about floating point now,
since this format is no longer used for floating point.
(PRINT_FIELDS): New macro, with most of the guts of the old PRINT_TYPE.
(PRINT_TYPE): Rewrite to use PRINT_FIELDS.
(PRINT_FLOATTYPE): New macro.  This uses the new functions from
ftoastr.
(print_float, print_double, print_long_double): Reimplement
using PRINT_FLOATTYPE.
(decode_one_format): Calculate field widths based on ftoastr-supplied
macros.
* tests/Makefile.am (TESTS): Add misc/od-float.
* tests/misc/od-float: New file.

13 years agobuild: update gnulib submodule to latest
Paul Eggert [Thu, 18 Nov 2010 17:33:26 +0000 (09:33 -0800)]
build: update gnulib submodule to latest

13 years agomaint: syntax-check: prevent new $VERBOSE/--version tests
Jim Meyering [Thu, 18 Nov 2010 05:29:54 +0000 (06:29 +0100)]
maint: syntax-check: prevent new $VERBOSE/--version tests

* cfg.mk (sc_prohibit_verbose_version): New rule.

13 years agotests: convert tests/misc/selinux manually
Jim Meyering [Tue, 16 Nov 2010 07:32:00 +0000 (08:32 +0100)]
tests: convert tests/misc/selinux manually

13 years agotests: convert the multi-prog $VERBOSE/--version uses
Jim Meyering [Wed, 17 Nov 2010 20:38:38 +0000 (21:38 +0100)]
tests: convert the multi-prog $VERBOSE/--version uses

E.g.,

-test "$VERBOSE" = yes && { env -- pwd --version; readlink --version; }
+print_ver_ pwd readlink

-test "$VERBOSE" = yes && { stdbuf --version; mv --version; }
+print_ver_ stdbuf mv

Use this command:
git grep -l 'VERBOSE.*--version'|xargs perl -ni \
  -e '/^test "\$VERBOSE" = yes && { .*--version/ or print,next;' \
  -e 's/env -- //g;' \
  -e 's/test "\$VERBOSE" = yes && { /print_ver_ /;' \
  -e ' s/(\w+) --version;/$1/g; s/ *}$//; print'

13 years agotests: convert "... env -- prog --version" uses
Jim Meyering [Wed, 17 Nov 2010 20:42:13 +0000 (21:42 +0100)]
tests: convert "... env -- prog --version" uses

E.g.,

-test "$VERBOSE" = yes && env -- pwd --version
+print_ver_ pwd

git grep -l 'VERBOSE.*--version'|xargs perl -pi -e \
  's/test "\$VERBOSE" = yes && env -- (\w+) --version/print_ver_ $1/'

13 years agotests: substitute the single-program $VERBOSE/--version uses
Jim Meyering [Wed, 17 Nov 2010 20:35:31 +0000 (21:35 +0100)]
tests: substitute the single-program $VERBOSE/--version uses

Automatically make all of the changes like this:

  -test "$VERBOSE" = yes && chgrp --version
  +print_ver_ chgrp

git grep -l 'VERBOSE.*--version'|xargs perl -pi -e \
  's/test "\$VERBOSE" = yes && (\w+) --version/print_ver_ $1/'

13 years agotests: factor out VERBOSE-only --version-printing code
Jim Meyering [Tue, 16 Nov 2010 07:05:11 +0000 (08:05 +0100)]
tests: factor out VERBOSE-only --version-printing code

* tests/init.cfg (print_ver_): New function.

13 years agotests: remove test-lib.sh; now all tests use gnulib's init.sh
Jim Meyering [Sun, 14 Nov 2010 11:13:58 +0000 (12:13 +0100)]
tests: remove test-lib.sh; now all tests use gnulib's init.sh

* tests/test-lib.sh: Remove file.  No longer used.
* tests/Makefile.am (EXTRA_DIST): Remove it here, too.
* tests/sample-test: Correct a comment.

13 years agotests: convert remaining uses of test-lib.sh to init.sh
Jim Meyering [Sun, 14 Nov 2010 11:07:57 +0000 (12:07 +0100)]
tests: convert remaining uses of test-lib.sh to init.sh

RHS='. "\${srcdir=.}/init.sh"; path_prepend_ ../src'
git grep -l test-lib.sh \
  | xargs perl -pi -e 's,^\. \$srcdir/test-lib\.sh$,'"$RHS",

13 years agotests: convert 'if test "$VERBOSE" = yes; then' to test ... &&
Jim Meyering [Sun, 14 Nov 2010 11:02:39 +0000 (12:02 +0100)]
tests: convert 'if test "$VERBOSE" = yes; then' to test ... &&

13 years agotests: convert first batch of tests from test-lib.sh to init.sh
Jim Meyering [Sun, 14 Nov 2010 10:05:05 +0000 (11:05 +0100)]
tests: convert first batch of tests from test-lib.sh to init.sh

Initially, I did this,
git grep -l srcdir/test-lib.sh|xargs perl -p0i -e '~180-byte script'
but that line would have been much longer than the maximum permitted
by coreutils' commit hook, and wasn't readable besides, so here's a
more readable version:

lhs=$(printf '%s\\n' \
'if test "$VERBOSE" = yes; then' \
'  set -x' \
'  touch --version' \
'fi' \
'' \
'. $srcdir/test-lib.sh' \
  | sed 's/\$/\\\$/g;s/touch/(\\w+)/')

rhs=$(printf '%s\\n' \
'. "${srcdir=.}/init.sh"; path_prepend_ ../src' \
'test "$VERBOSE" = yes && FIXME --version' \
  | sed 's/\$/\\\$/g;s/FIXME/\$1/')

git grep -l srcdir/test-lib.sh|xargs perl -p0i -e "s,$lhs,$rhs,"

13 years agotests: move a comment so automated conversion works
Jim Meyering [Sun, 14 Nov 2010 10:01:25 +0000 (11:01 +0100)]
tests: move a comment so automated conversion works

13 years agomaint: add a NEWS-checking syntax-check rule
Jim Meyering [Sun, 14 Nov 2010 09:16:27 +0000 (10:16 +0100)]
maint: add a NEWS-checking syntax-check rule

* cfg.mk (sc_NEWS_two_empty_lines): New syntax-check rule.

13 years agobuild: add `patch` as a bootstrap dependency
Pádraig Brady [Tue, 16 Nov 2010 07:32:32 +0000 (07:32 +0000)]
build: add `patch` as a bootstrap dependency

* bootstrap.conf (buildreq): require `patch` as it's used
by gnulib-tool to apply local diffs to gnulib modules

13 years agosplit: fail immediately if impossible to create a large file
Pádraig Brady [Sun, 7 Nov 2010 03:09:38 +0000 (03:09 +0000)]
split: fail immediately if impossible to create a large file

* src/split.c (main): Error if -[bC] value > OFF_T_MAX
* tests/misc/split-fail: Adjust for the new lower limits

13 years agotruncate: fix a very unlikely case for undiagnosed errors
Pádraig Brady [Sun, 24 Oct 2010 21:44:52 +0000 (22:44 +0100)]
truncate: fix a very unlikely case for undiagnosed errors

src/truncate.c (main): Use a bool to store if an error occurred,
rather than an int, to protect against overflow.
(do_ftruncate): Likewise.   Also change 0/false to mean failure
rather than success.

13 years agomaint: fix a new -Wpointer-sign gcc warning
Pádraig Brady [Tue, 16 Nov 2010 00:31:05 +0000 (00:31 +0000)]
maint: fix a new -Wpointer-sign gcc warning

* src/csplit.c (max_out): Fix a new warning introduced with
commit 6568b173, 2010-11-10, "csplit: do not rely on..."

13 years agobuild: enable the -Wpointer-sign gcc warning
Pádraig Brady [Mon, 25 Oct 2010 10:16:37 +0000 (11:16 +0100)]
build: enable the -Wpointer-sign gcc warning

... when configured with the --enable-gcc-warnings option.
This follows on from commit 34ef0a01, 2010-10-14,
"sort: fix unportable cast of unsigned char * -> char *"

* configure.ac: -Wall implicitly enables this warning
so remove the explicit disabling.

13 years agomaint: add a missed fadvise-tests module
Pádraig Brady [Mon, 15 Nov 2010 09:36:16 +0000 (09:36 +0000)]
maint: add a missed fadvise-tests module

* gl/modules/fadvise-tests: Add the module previously missed
in commit 63b5e816, 2010-07-14, "fadvise: new module ...".
* gl/tests/test-fadvise.c: Add a comment as to why we don't
check return values.

13 years agodoc: tweak NEWS and coreutils.texi
Paul Eggert [Sat, 13 Nov 2010 21:02:29 +0000 (22:02 +0100)]
doc: tweak NEWS and coreutils.texi

* doc/coreutils.texi (stat invocation): Add sentence-ending period.
* NEWS: Correct stat change description: s/floating point //.
* cfg.mk (old_NEWS_hash): Update, to match this NEWS change.

13 years agopost-release administrivia
Jim Meyering [Sat, 13 Nov 2010 16:37:18 +0000 (17:37 +0100)]
post-release administrivia

* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.

13 years agoversion 8.7 v8.7
Jim Meyering [Sat, 13 Nov 2010 16:21:08 +0000 (17:21 +0100)]
version 8.7

* NEWS: Record release date.

13 years agobuild: update gnulib...
Jim Meyering [Sat, 13 Nov 2010 12:28:24 +0000 (13:28 +0100)]
build: update gnulib...

...to fix the failing update-copyright test,
and for the openat portability fix.

13 years agobuild: update gnulib for linux 2.6.9 nanosleep workaround
Jim Meyering [Sat, 13 Nov 2010 09:12:49 +0000 (10:12 +0100)]
build: update gnulib for linux 2.6.9 nanosleep workaround

Also, do the following to avoid "make syntax-check" failure
induced by new rules.
* .x-sc_bindtextdomain: Exempt files with an #ifdef'd "main".
* Makefile.am: Add this file.

13 years agomaint: remove fstimeprec (variable precision time stamp) support
Jim Meyering [Sat, 13 Nov 2010 07:04:13 +0000 (08:04 +0100)]
maint: remove fstimeprec (variable precision time stamp) support

The implementation of variable-precision time stamps relied
on heuristics that made the output subtly nondeterministic,
or at least hard to reproduce:
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/21531/focus=21538
So, for now at least, we're removing that feature.
* bootstrap.conf (gnulib_modules): Remove fstimeprec.
* gl/lib/fstimeprec.c, gl/lib/fstimeprec.h: Remove files.
* gl/modules/fstimeprec, gl/modules/fstimeprec-tests: Likewise.
* gl/tests/test-fstimeprec.c: Remove file.

13 years agostat: do not provide variable precision time stamps
Jim Meyering [Sat, 13 Nov 2010 07:02:02 +0000 (08:02 +0100)]
stat: do not provide variable precision time stamps

* src/stat.c: Don't include fstimeprec.c.
(out_epoch_sec): Don't call fstimeprec.
* NEWS: Update description.
* doc/coreutils.texi: Likewise.

13 years agocsplit: do not rely on undefined behavior in printf formats
Paul Eggert [Thu, 11 Nov 2010 04:34:52 +0000 (20:34 -0800)]
csplit: do not rely on undefined behavior in printf formats

* doc/coreutils.texi (csplit invocation): Say that %d and %i are
aliases for %u.
* src/csplit.c (FLAG_THOUSANDS, FLAG_ALTERNATIVE): New constants.
(get_format_flags): Now take char const * and int * and return
size_t.  It now stores info about the flags instead of merely
scanning them.  Also, it handles '0' correctly.  Drop support for
the undocumented '+' and ' ' flags since the value is unsigned.
Add support for the (undocumented) "'" flag.  All uses changed.
(get_format_width, get_format_prec): Remove.
(check_format_conv_type): Renamed from get_format_conv_type, with
a different signature.  It now converts the format to one that is
compatible with unsigned int, and checks flags.  All uses changed.
(max_out): Have snprintf compute the number of bytes needed rather
than attempting to do it ourselves (which doesn't work portably
with outlandish formats such as %4294967296d).
(check_format_conv_type, main): Check for overflow in size
calculations.  Don't assume size_t fits in unsigned int.
* tests/misc/csplit: Check for proper handling of flags, with
%0#6.3x.  Coreutils 8.6 mishandles this somewhat-weird example.

13 years agocsplit: fix a memory leak per input buffer
Pádraig Brady [Wed, 10 Nov 2010 14:35:17 +0000 (14:35 +0000)]
csplit: fix a memory leak per input buffer

* src/csplit.c (free_buffer): Also free the line offsets buffers
(remove_line): Also free the containing structure
* tests/misc/csplit-heap: A new test to trigger with leaks of
this magnitude.
* tests/Makefile.am: Reference the new test
* NEWS: Mention the fix
Reported by David Hofstee

13 years agotests: fix comments and --version invocation in new test
Jim Meyering [Wed, 10 Nov 2010 20:54:57 +0000 (21:54 +0100)]
tests: fix comments and --version invocation in new test

* tests/misc/csplit-1000: Fix comments and --version invocation.

13 years agocsplit: avoid buffer overrun when writing more than 999 files
Jim Meyering [Wed, 10 Nov 2010 12:53:38 +0000 (13:53 +0100)]
csplit: avoid buffer overrun when writing more than 999 files

Without this fix, seq 1000 | csplit - /./ '{*}' would write
the NUL-terminated file name, xx1000, into a buffer of size 6.
* src/csplit.c (main): Use properly sized file name buffer.
* NEWS (Bug fixes): Mention it.
* tests/misc/csplit-1000: New test to trigger the bug.
* tests/Makefile.am (TESTS): Add misc/csplit-1000.

13 years agotests: avoid portability problem with dash vs. symlinked ttys
Pádraig Brady [Tue, 9 Nov 2010 12:43:20 +0000 (12:43 +0000)]
tests: avoid portability problem with dash vs. symlinked ttys

* tests/mv/i-3: Adjust so that the symlink is resolved
before redirecting to the background command, as otherwise
the stdin descriptor passed to the command will fail the
isatty() or ttyname() test.

13 years agotests: avoid portability problem with dash vs. local v=$splittable_val
Pádraig Brady [Mon, 8 Nov 2010 10:26:08 +0000 (11:26 +0100)]
tests: avoid portability problem with dash vs. local v=$splittable_val

* tests/init.cfg (retry_delay_): Adjust awk script to print a single
floating point value, rather than many.

13 years agotests: don't disqualify FreeBSD 8.1's /bin/sh just yet,
Jim Meyering [Sun, 7 Nov 2010 13:23:32 +0000 (14:23 +0100)]
tests: don't disqualify FreeBSD 8.1's /bin/sh just yet,

because dash-0.5.6-2.fc14.x86_64 would also be disqualified.
This reverts part of yesterday's commit 6c058b2d, "tests: avoid
failure due to bug in FreeBSD 8.1's /bin/sh".  Note that the offending
aspect of those shells is not officially a bug, since "local" is not
specified by POSIX.  However, it is very unintuitive that prepending
"local" to an existing, standards-compliant assignment would evoke
such a fundamental change in semantics.
* tests/init.sh: Remove snippet requiring sane "local" support.
Upstream dash bug report:
http://thread.gmane.org/gmane.comp.shells.dash/419

13 years agostat: do not rely on undefined behavior in printf formats
Paul Eggert [Sat, 6 Nov 2010 20:57:08 +0000 (13:57 -0700)]
stat: do not rely on undefined behavior in printf formats

* src/stat.c (digits, printf_flags): New static vars.
(make_format): New function.
(out_string, out_int, out_uint, out_uint_o, out_uint_x):
(out_minus_zero): Use it to avoid undefined behavior when invoking
printf.
(print_it): Check for invalid conversion specifications such as
%..X and %1-X, which would otherwise rely on undefined behavior
when invoking printf.
* tests/misc/stat-nanoseconds: Check that the "I" printf flag
doesn't mess up in the C locale, as it formerly did on non-GNU
hosts.

13 years agotests: avoid failure due to bug in FreeBSD 8.1's /bin/sh
Jim Meyering [Sat, 6 Nov 2010 14:20:06 +0000 (15:20 +0100)]
tests: avoid failure due to bug in FreeBSD 8.1's /bin/sh

* tests/init.sh: Arrange not to accept FreeBSD 8.1's /bin/sh, since
it fails this test: /bin/sh -c 'f(){ local s=$IFS; test -n "$s"; }; f'
* tests/init.cfg (sanitize_path_): Stop-gap measure to work around
a bug in FreeBSD 8.1's /bin/sh.  We'll un-do this change once all
300+ tests use init.sh.

13 years agobuild: update gnulib to latest for strtod and icc fixes
Jim Meyering [Sat, 6 Nov 2010 07:57:44 +0000 (08:57 +0100)]
build: update gnulib to latest for strtod and icc fixes

13 years agostat: use e.g. %.3X instead of %X.%3:X for sub-second precision
Paul Eggert [Sat, 6 Nov 2010 02:35:12 +0000 (19:35 -0700)]
stat: use e.g. %.3X instead of %X.%3:X for sub-second precision

* NEWS: Document this.
* doc/coreutils.texi (stat invocation): Likewise.
* gl/lib/fstimeprec.c, gl/lib/fstimeprec.h, gl/modules/fstimeprec:
* gl/modules/fstimeprec-tests, gl/tests/test-fstimeprec.c:
New files.
* bootstrap.conf (gnulib_modules): Add fstimeprec.
* src/stat.c: Include fstimeprec.h.  Don't include xstrtol.h.
(decimal_point, decimal_point_len): New static vars.
(main): Initialize them.
(epoch_sec, out_ns): Remove.
(out_int, out_uint): Now returns whatever printf returned.
(out_minus_zero, out_epoch_secs): New functions.
(print_stat): Use out_epoch_sec instead of out_ns and epoch_sec.
(print_stat, print_it, usage): Remove the %:X-style formats.
* tests/misc/stat-nanoseconds: Set TZ=UTC0 to avoid problems
with weird time zones.  Use a time stamp near the Epoch so that we
don't have to worry about leap seconds.  Redo test cases to match
new behavior.
* tests/touch/60-seconds: Change %Y.%:Y to %.9Y, to adjust to
new behavior.

13 years agocp: NEWS entry for previous fix
Eric Blake [Thu, 4 Nov 2010 16:49:21 +0000 (10:49 -0600)]
cp: NEWS entry for previous fix

Resolves bug#7324

* NEWS: Document the extent of the Solaris crash.

13 years agocp, mv, touch: fix file time manipulation on Solaris 10
Eric Blake [Thu, 4 Nov 2010 14:16:04 +0000 (08:16 -0600)]
cp, mv, touch: fix file time manipulation on Solaris 10

* gnulib: Update to latest, for futimens fix.
* .gitignore: Ignore another recent gnulib-generated file.

13 years agobuild: update gnulib to latest
Jim Meyering [Wed, 3 Nov 2010 12:11:21 +0000 (13:11 +0100)]
build: update gnulib to latest

13 years agostat: revert %X-%Y-%Z change; use e.g., %:X to print fractional seconds
Jim Meyering [Thu, 21 Oct 2010 16:41:24 +0000 (18:41 +0200)]
stat: revert %X-%Y-%Z change; use e.g., %:X to print fractional seconds

This reverts part of the recent commit 9069af45,
"stat: print timestamps to full resolution", which made %X, %Y, %Z
print floating point numbers.  We prefer to retain portability of
%X, %Y and %Z uses, while still providing access to full-resolution
time stamps via modified format strings.  Also make the new
%W consistent.
* src/stat.c: Include "xstrtol.h".
(print_it): Accept a new %...:[XYZ] format directive,
e.g., %:X, to print the nanoseconds portion of the corresponding time.
For example, %3.3:Y prints the zero-padded, truncated, milliseconds
part of the time of last modification.
(print_it): Update print_func signature to match.
(neg_to_zero): New helper function.
(epoch_time): Remove function; replace with...
(epoch_sec): New function; use timetostr.
(out_ns): New function.  Use "09" only when no other modifier
is specified.
(print_statfs): Change type of "m" to unsigned int,
now that it must accommodate values larger than 255.
(print_stat): Likewise.
Map :X to a code of 'X' + 256.  Likewise for Y, Z and W.
(usage): Update.
* tests/touch/60-seconds: Use %Y.%:Y in place of %Y.
* tests/misc/stat-nanoseconds: New file.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Changes in behavior): Mention this.
With improvements by Pádraig Brady.
Thanks to Andreas Schwab for raising the issue.

13 years agotests: fix a false positive for 32 bit on 64 bit hosts
Nix [Mon, 1 Nov 2010 15:46:01 +0000 (15:46 +0000)]
tests: fix a false positive for 32 bit on 64 bit hosts

* tests/ls/stat-free-symlinks: strace currently outputs an
informational message about arch difference to stdout.
Therefore we need to strip that before comparison.

13 years agomaint: remove an unnecessary FIXME comment
Patrick W. Plusnick II [Thu, 28 Oct 2010 12:54:53 +0000 (14:54 +0200)]
maint: remove an unnecessary FIXME comment

* src/seq.c (terminator): This does not need to be specifiable
via an option.  Remove the FIXME comment.

13 years agocp: make --attributes-only override --reflink completely
Pádraig Brady [Tue, 26 Oct 2010 16:55:10 +0000 (17:55 +0100)]
cp: make --attributes-only override --reflink completely

* doc/coreutils.texi (cp invocation): Change the description slightly
so as users might not immediately discount using this option.
Mention that --reflink is overridden by the other linking options and
--attributes-only, and give an example where this might be useful.
* src/copy.c (copy_internal): Bypass the reflink if
--attributes-only is specifed.
* tests/cp/reflink-perm: Ensure both --reflink modes are
overridden by --attributes-only.
* NEWS: Mention the change in behavior.
Reported by Jim Meyering.

13 years agotests: mention that btrfs also causes failure of misc/ls-time
Jim Meyering [Tue, 26 Oct 2010 14:34:51 +0000 (16:34 +0200)]
tests: mention that btrfs also causes failure of misc/ls-time

* tests/misc/ls-time: Mention that this test also fails on btrfs,
at least until http://bugzilla.redhat.com/591068 is addressed.

13 years agomaint: update THANKS
Eric Blake [Mon, 25 Oct 2010 17:51:45 +0000 (11:51 -0600)]
maint: update THANKS

* THANKS: Update.

13 years agodate: correct typos in date --help
Tobias Quathamer [Thu, 21 Oct 2010 07:15:00 +0000 (09:15 +0200)]
date: correct typos in date --help

* src/date.c (usage): Use "e.g." correctly.

13 years agotail: support rechecking currently missing remote dirs
Pádraig Brady [Sun, 24 Oct 2010 13:02:12 +0000 (14:02 +0100)]
tail: support rechecking currently missing remote dirs

src/tail.c (main): As an optimization, don't bother checking
for stdin or remote files, when ---disable-inotify is specified.
To improve the fix in commit 61b77891, set the disable_inotify
flag when we fall back to polling, so that we recheck remote files.
NEWS: Mention the fix

13 years agodu: don't print junk when diagnosing out-of-range time stamps
Paul Eggert [Sun, 24 Oct 2010 00:20:01 +0000 (17:20 -0700)]
du: don't print junk when diagnosing out-of-range time stamps

* src/du.c (show_date): Fix call to fputs with a buffer that
contains some uninitialized data.
* tests/Makefile.am (TESTS): Add du/big-timestamp.
* tests/du/bigtime: New file, which checks for the bug.

13 years agodoc: add a cross reference from cat's man page to "tac"
Rodrigo Campos [Tue, 19 Oct 2010 22:48:46 +0000 (19:48 -0300)]
doc: add a cross reference from cat's man page to "tac"

* man/cat.x: See also "tac".

13 years agotests: trigger and test for md5sum's new diagnostics
Jim Meyering [Tue, 19 Oct 2010 07:19:59 +0000 (09:19 +0200)]
tests: trigger and test for md5sum's new diagnostics

* tests/misc/md5sum: Test for new diagnostics.

13 years agomd5sum: print a summary warning for improperly formatted lines
Benno Schulenberg [Mon, 18 Oct 2010 20:35:39 +0000 (22:35 +0200)]
md5sum: print a summary warning for improperly formatted lines

And remove the now-superfluous totals from the other two warnings,
so the plurals will also work in other languages than English.

* src/md5sum.c (digest_check): Change as above.
* tests/misc/md5sum (check-quiet2): Adjust accordingly.

13 years agobuild: enable -Werror for gnulib-tests, too
Jim Meyering [Sun, 17 Oct 2010 17:08:44 +0000 (19:08 +0200)]
build: enable -Werror for gnulib-tests, too

* gnulib: Update to latest for warning-free tests.
* gnulib-tests/Makefile.am (AM_CFLAGS): Uncomment $(WERROR_CFLAGS),
now that these tests are warning-free.

13 years agomaint: update bootstrap from gnulib
Jim Meyering [Sun, 17 Oct 2010 13:32:03 +0000 (15:32 +0200)]
maint: update bootstrap from gnulib

* bootstrap: Update from gnulib.

13 years ago.gitignore: anchor patterns
Jim Meyering [Sun, 17 Oct 2010 10:47:38 +0000 (12:47 +0200)]
.gitignore: anchor patterns

* .gitignore: Anchor most patterns.  Remove .tarball-version.
* doc/.gitignore: Anchor all patterns.
Remove getdate.texi and get_date.texi.

13 years agotests: sort-float: avoid spurious test failure on ppc/ppc64
Jim Meyering [Sat, 16 Oct 2010 18:18:19 +0000 (20:18 +0200)]
tests: sort-float: avoid spurious test failure on ppc/ppc64

* tests/misc/sort-float: On systems with DBL_MIN < LDBL_MIN,
this test would fail because the expected output was not sorted.
Detect that case, and if needed, reverse those two values.

13 years agofold: fix fadvise hint
Andreas Schwab [Sat, 16 Oct 2010 11:32:04 +0000 (13:32 +0200)]
fold: fix fadvise hint

* src/fold.c (fold_file): Apply fadvise to istream, not stdin.
This bug would have inhibited the fadvise optimization when not
reading from standard input.

13 years agopost-release administrivia
Jim Meyering [Fri, 15 Oct 2010 15:06:56 +0000 (17:06 +0200)]
post-release administrivia

* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.

13 years agoversion 8.6 v8.6
Jim Meyering [Fri, 15 Oct 2010 14:50:15 +0000 (16:50 +0200)]
version 8.6

* NEWS: Record release date.

13 years agobuild: update gnulib for vasnprintf fix
Jim Meyering [Fri, 15 Oct 2010 05:49:54 +0000 (07:49 +0200)]
build: update gnulib for vasnprintf fix

13 years agobug#7213: [PATCH] sort: fix buffer overrun on 32-bit hosts when warning re obsolete...
Paul Eggert [Thu, 14 Oct 2010 07:12:23 +0000 (00:12 -0700)]
bug#7213: [PATCH] sort: fix buffer overrun on 32-bit hosts when warning re obsolete keys

* src/sort.c (key_warnings): Local buffer should be of size
INT_BUFSIZE_BOUND (uintmax_t), not INT_BUFSIZE_BOUND (sword).
This bug was discovered by running 'make check' on a 32-bit
Solaris 8 sparc host, using Sun cc.  I saw several other instances
of invoking umaxtostr on a buffer declared to be of size
INT_BUFSIZE_BOUND (VAR), and these instances should at some point
be replaced by INT_BUFSIZE_BOUND (uintmax_t) too, as that's a
less error-prone style.