Jim Meyering [Fri, 17 Apr 2009 16:44:18 +0000 (18:44 +0200)]
cp: work around linux kernel bug: short-read != EOF on /proc
Remove the optimization that avoided up to 50% of cp's read syscalls.
Do not assume that a short read on a regular file indicates EOF.
When reading from a file in /proc on linux [at least 2.6.9 - 2.6.29]
into a 4k-byte buffer or larger, a short read does not
always indicate EOF. For example, "cp /proc/slabinfo /tmp"
copies only 4068 of the total 7493 bytes. This optimization
(
25719a33154f0c62ea9881f0c79ae312dd4cec7a, Improve performance a bit
by optimizing away; 2005-11-24) appears to have been worth less than
a 2% speed-up (and usually much less), so the impact of removing it
is negligible.
* src/copy.c (copy_reg): Don't exit the loop early.
* tests/cp/proc-short-read: New test, lightly based on a suggestion
from Mike Frysinger, to exercise this fix.
* tests/Makefile.am (TESTS): Add cp/proc-short-read.
* NEWS (Improve robustness): Mention this change.
Jim Meyering [Wed, 22 Apr 2009 19:41:10 +0000 (21:41 +0200)]
tests: avoid new "make distcheck" failure due to newer File::Temp
With newer perl (e.g., Fedora 10's 4:5.10.0-68.fc10), tests/CuTmpdir.pm
stopped removing its temporary directories, with diagnostics like this:
cannot remove path when cwd is /c/coreutils/tests/misc/seq.tmp-e2up \
for /c/coreutils/tests/misc/seq.tmp-e2up: at \
/usr/lib/perl5/5.10.0/File/Temp.pm line 902
Chdir out of the target directory before that code runs:
* tests/CuTmpdir.pm (END): chdir '..'.
(chmod_tree): Remove explicit "chdir $dir".
Jim Meyering [Sat, 18 Apr 2009 07:17:04 +0000 (09:17 +0200)]
doc: update README
* README: (Reporting bugs): List the bug-reporting address here, too,
not just in the following more test-oriented paragraph.
Reported by Tim Mooney.
All changes are no longer listed in version-controlled ChangeLog
files, so note that contributions are attributed in the commit logs.
Mention bootstrap.conf, now that it's the authoritative source of
minimal prerequisite program/version# pairs.
Aurelien Jarno [Fri, 17 Apr 2009 16:10:04 +0000 (18:10 +0200)]
tests: avoid failure of install-C test on FreeBSD UFS file system
* tests/install/install-C: Use 2755 (set-gid), not 1755 (sticky),
to test install -C with non-permission mode bits set. At least on
FreeBSD with a UFS file system, a non-root user may not set the
sticky bit on a non-directory.
Jim Meyering [Sun, 19 Apr 2009 19:49:14 +0000 (21:49 +0200)]
comment touch up: insert a space between "#" and text of comment
* bootstrap: It's easier to read that way.
Ralf Wildenhues [Sat, 18 Apr 2009 17:37:29 +0000 (19:37 +0200)]
fix comment typos
* bootstrap: Fix comment typos.
* src/pr.c: Likewise.
Jim Meyering [Fri, 17 Apr 2009 07:10:57 +0000 (09:10 +0200)]
maint: bootstrap: sync a tiny change from gnulib
* bootstrap: rename variable to sync with gnulib: s/\$depth/$shallow/
Jim Meyering [Thu, 16 Apr 2009 05:42:20 +0000 (07:42 +0200)]
* gnulib: Update submodule to latest.
Jim Meyering [Thu, 16 Apr 2009 05:03:59 +0000 (07:03 +0200)]
build: use '.'-relative name, gnulib_dir
* cfg.mk (gnulib_dir): Use "gnulib", not "/gnulib", now that
the former is a git submodule, and guaranteed to exist.
Jim Meyering [Wed, 15 Apr 2009 18:18:24 +0000 (20:18 +0200)]
maint: clean up skipped syntax-check rules
* cfg.mk (local-checks-to-skip): Remove patch-check and changelog-check.
* maint.mk (patch-check): Remove rule.
(local-checks-available): Remove patch-check.
(changelog-check): Remove rule.
Ralf Wildenhues [Sat, 11 Apr 2009 12:48:49 +0000 (14:48 +0200)]
build: avoid parallel distcheck failure
* Makefile.am (ALL_RECURSIVE_TARGETS): Initialize here, too,
to please automake. Add install-root, check-root, distcheck-hook.
* cfg.mk (ALL_RECURSIVE_TARGETS): Add sc_tight_scope.
* maint.mk (ALL_RECURSIVE_TARGETS): Add patch-check,
check-AUTHORS, maintainer-distcheck, vc-dist, taint-distcheck,
my-distcheck, alpha, beta, major.
Jim Meyering [Fri, 10 Apr 2009 10:50:14 +0000 (12:50 +0200)]
doc: fix a typo: s/is a is a/is a/
* doc/coreutils.texi (Putting the tools together): Fix typo.
Jim Meyering [Fri, 10 Apr 2009 09:35:10 +0000 (11:35 +0200)]
build: tight-scope test: don't build excluded programs
* src/Makefile.am (sc_tight_scope): Depend on $(bin_PROGRAMS),
not $(all_programs)
Jim Meyering [Fri, 10 Apr 2009 08:12:10 +0000 (10:12 +0200)]
gnulib: update to latest: fix ls -v & sort -V
* NEWS (Bug fixes): Mention it.
Jim Meyering [Fri, 10 Apr 2009 06:42:10 +0000 (08:42 +0200)]
maint: remove obsolete utime module
* bootstrap.conf (obsolete_gnulib_modules): Remove utime.
Jim Meyering [Fri, 10 Apr 2009 06:20:13 +0000 (08:20 +0200)]
maint: put each module name on its own line; sort
* bootstrap.conf (gnulib_modules): List them one per line.
Jim Meyering [Thu, 9 Apr 2009 13:29:59 +0000 (15:29 +0200)]
tests: misc/ls-misc: avoid shadowing local decl of $e
* tests/misc/ls-misc (make_j_d): Rename latter $e to $env
Pádraig Brady [Wed, 8 Apr 2009 09:43:15 +0000 (10:43 +0100)]
id: fix infinite loop on some systems
Steven Parkes reported that `id -G $USER` went into an infinite loop
on Darwin systems for users in more than 10 groups:
http://bugs.gentoo.org/show_bug.cgi?id=264007
* gl/lib/mgetgroups.c (mgetgroups): Work around buggy getgrouplist
implementations that don't update the required size correctly,
by doubling the result buffer and retrying. Also return the
parameter updated by getgrouplist rather than its return value,
as the documentation doesn't actually state the number of groups
stored is returned by getgrouplist.
* tests/misc/id-groups: Add test to exercise this logic
* tests/Makefile.am: Reference new test
* NEWS: Mention the fix
* THANKS: Update
Jim Meyering [Tue, 7 Apr 2009 18:16:00 +0000 (20:16 +0200)]
maint: avoid "make syntax-check" failure
* src/install.c (main): Tweak syntax of new diagnostic.
Pádraig Brady [Tue, 7 Apr 2009 13:14:58 +0000 (14:14 +0100)]
install: deprecate the --preserve_context option
src/install.c: Print a warning when --preserve_context
rather than --preserve-context is used as per FIXME.
Pádraig Brady [Mon, 6 Apr 2009 11:23:26 +0000 (12:23 +0100)]
maint: remove compiler warnings from randint and randread modules
* gl/lib/randint.c: Remove unused MAX macro
* gl/lib/randread.c (randread_error): Add __attribute__((__noreturn__))
Pádraig Brady [Mon, 6 Apr 2009 07:42:15 +0000 (08:42 +0100)]
shred,sort,shuf: don't use /dev/urandom by default
Suggestion from Steven Schveighoffer at:
http://savannah.gnu.org/patch/?6797
to greatly speed up the random passes done by shred.
* gl/lib/randread.c: Default to using the internal
pseudorandom generator, rather than reading /dev/urandom
* src/shred.c (usage): remove mention of /dev/urandom
* src/shuf.c (usage); ditto
* src/sort.c (usage): ditto
* doc/coreutils.text: Document the new behaviour
for aquiring random data.
Pádraig Brady [Mon, 6 Apr 2009 06:43:49 +0000 (07:43 +0100)]
maint: Clarify ambiguous refs to Linux kernels or GNU/Linux systems
* README-prereq: s_linux_GNU/Linux_ or s_linux_Linux kernel_
* README-valgrind: ditto
* src/chown-core.c: ditto
* src/dd.c: ditto
* src/df.c: ditto
* src/ls.c: ditto
* src/mv.c: ditto
* src/pwd.c: ditto
* src/remove.c: ditto
* src/shred.c: ditto
* src/stat.c: ditto
* src/su.c: ditto
* src/system.h: ditto
* src/timeout.c: ditto
* src/truncate.c: ditto
Pádraig Brady [Mon, 6 Apr 2009 18:21:51 +0000 (19:21 +0100)]
doc: add missing documentation for some SELinux options
* doc/coreutils.texi (ls invocation): Describe the --context (-Z) option
(install invocation): Describe the --preserve-context and -Z options
(id invocation): Describe the --context (-Z) option
(mkdir invocation): ditto
(mknod invocation): ditto
(mkfifo invocation): ditto
* TODO: remove the todo item
Jim Meyering [Mon, 6 Apr 2009 05:59:52 +0000 (07:59 +0200)]
* doc/code-vs-command: Remove long-unused, never-distributed file.
Jim Meyering [Sat, 4 Apr 2009 20:52:22 +0000 (22:52 +0200)]
gnulib: update to latest: includes Solaris 8 portability fixes
Jim Meyering [Sat, 4 Apr 2009 10:39:44 +0000 (12:39 +0200)]
maint: clean up Makefile.am, rename remaining ChangeLog files
* Makefile.am (changelog_etc, syntax_check_exceptions): Define.
(EXTRA_DIST): Use. Remove names that are now automatically
included: build-aux/git-version-gen, build-aux/vc-list-files.
* build-aux/ChangeLog-2007: Rename from */ChangeLog.
* doc/ChangeLog-2007: Likewise.
* lib/ChangeLog-2007: Likewise.
* m4/ChangeLog-2007: Likewise.
* po/ChangeLog-2007: Likewise.
Jim Meyering [Thu, 2 Apr 2009 14:51:11 +0000 (16:51 +0200)]
tests: tweak po_check syntax check
* maint.mk (sc_po_check): Don't let dangling lib/*.[ch] links cause
spurious "grep: lib/file.h: No such file or directory" warnings.
Jim Meyering [Thu, 2 Apr 2009 14:44:44 +0000 (16:44 +0200)]
tests: improve ChangeLog syntax check
* maint.mk (sc_changelog): Use $(VC_LIST_EXCEPT), not find.
Don't hang when there are no ChangeLog files.
(sc_require_config_h): For the file-name-filter, use grep -l
to be slightly more efficient.
(sc_require_config_h_first, _header_without_use, sc_program_name):
Likewise.
Ralf Wildenhues [Sat, 4 Apr 2009 08:25:18 +0000 (10:25 +0200)]
tests: make syntax-checks more robust
* maint.mk (_prohibit_regexp): Add ';' at end of ':'-introduced
comment so that the comment doesn't gobble up the following test
for a missing regexp definition.
(_header_without_use): Escape all '.'s in a header file name,
not just the first one.
Jim Meyering [Fri, 3 Apr 2009 20:05:10 +0000 (22:05 +0200)]
* cfg.mk (old_NEWS_hash): Regenerate once more.
Jim Meyering [Fri, 3 Apr 2009 19:52:16 +0000 (21:52 +0200)]
doc: adjust 7.2 "cat,cp,install,mv,split speed-up" NEWS item
* NEWS: Reword an entry from 7.2 and change "linux" to "GNU/Linux".
The latter was requested by Richard Stallman.
* cfg.mk (old_NEWS_hash): Regenerate.
Pádraig Brady [Tue, 24 Mar 2009 14:29:21 +0000 (14:29 +0000)]
ls: fix alignment when month names have varying widths
Reported by Samuel Thibault and Stéphane Raimbault, as the glibc fr_FR
locale has recently changed to use the official but variable width
abbreviated month names. Other glibc locales also have variable widths.
http://sourceware.org/ml/libc-locales/2008-q1/msg00035.html
http://sourceware.org/bugzilla/show_bug.cgi?id=9859
* NEWS: Mention the fix
* gl/lib/mbsalign.c: A new module to align and truncate a
string in a specified number of screen cells, while handling
multi-byte characters appropriately.
* gl/lib/mbsalign.h: Ditto
* gl/modules/mbsalign: Ditto
* bootstrap.conf: Reference the new module
* src/ls.c (abmon_init): New function, precompute the abbreviated
months aligned left in a minimum width column <= 5 screen cells.
(align_nstrftime): New function, replace the first %b in the
format specification to strftime with the precomputed month string.
Note using the cached month strings speeds up `ls -lU` by around 17%
on glibc-2.7-2 on linux at least. Also if we implement this function
using heap storage rather than automatic storage, and use snprintf
instead of strcpy, ls will slow down by 2% and 1% respectively
(i.e. a net gain of 14% rather than 17%).
* tests/ls/abmon-align: A new test to test ls alignment for
various formats and locales
* tests/Makefile.am: Reference the new test
Pádraig Brady [Wed, 1 Apr 2009 17:59:54 +0000 (18:59 +0100)]
doc: Clarify the help for version sorting in ls and sort
* src/ls.c: Update the -v help description
* src/sort.c: Update the -V help description
Jim Meyering [Thu, 2 Apr 2009 11:44:09 +0000 (13:44 +0200)]
doc: adjust README-prereq instructions
* README-prereq: Reverse order of -b/--track options.
Pádraig Brady mentioned that is required with at least git-1.5.3.6.
Invoke automake's own bootstrap script.
C de-Avillez [Wed, 1 Apr 2009 14:35:28 +0000 (09:35 -0500)]
build: require automake-1.10b or newer
* bootstrap.conf: Require at least automake-1.10b.
* README-prereq: Mention 1.10b, not 1.10a.
Jim Meyering [Wed, 1 Apr 2009 14:59:54 +0000 (16:59 +0200)]
tests: skip mv/i-3 if /dev/stdin is unreadable
* tests/mv/i-3: Skip if /dev/stdin is unreadable.
Reported by Sergei Steshenko.
Jim Meyering [Tue, 31 Mar 2009 18:43:35 +0000 (20:43 +0200)]
build: set automake's silent-rules option via configure.ac
* bootstrap: Revert commit
9f39fa855, 2009-03-28,
"build: use automake's --silent-rules option when possible".
* configure.ac (AM_INIT_AUTOMAKE): Instead, set it here.
Require the just-released version of automake: 1.10b.
Jim Meyering [Mon, 30 Mar 2009 07:47:07 +0000 (09:47 +0200)]
tests: enable automake's color-tests and parallel-tests via configure.ac
* gnulib-tests/Makefile.am (AUTOMAKE_OPTIONS): Don't define here.
* tests/Makefile.am (AUTOMAKE_OPTIONS): Likewise.
* configure.ac (AM_INIT_AUTOMAKE): Instead, add color-tests and
parallel-tests here.
Ralf Wildenhues [Sat, 14 Mar 2009 11:15:40 +0000 (12:15 +0100)]
build: use Automake's new parallel-tests feature
* build-aux/check.mk: Remove, not needed any more.
* tests/Makefile.am (AUTOMAKE_OPTIONS): Add parallel-tests and
color-tests.
* tests/check.mk: Do not include build-aux/check.mk any more.
(SUFFIXES, TEST_LOGS): Remove.
(TESTS_ENVIRONMENT): Use $$f rather than $$tst in $PERL invocation.
* gnulib-tests/Makefile.am: Do not include build-aux/check.mk.
(AUTOMAKE_OPTIONS): New macro, add parallel-test and color-tests.
(TEST_LOGS): Remove.
Jim Meyering [Tue, 31 Mar 2009 18:28:41 +0000 (20:28 +0200)]
* README-release (savannah news): Also mention the Subject: to use.
Jim Meyering [Tue, 31 Mar 2009 12:48:50 +0000 (14:48 +0200)]
post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
Jim Meyering [Tue, 31 Mar 2009 12:11:57 +0000 (14:11 +0200)]
version 7.2
* NEWS: Record release date.
Jim Meyering [Sun, 29 Mar 2009 17:54:14 +0000 (19:54 +0200)]
tests: don't use lang-default from individual tests
* tests/sample-test: Don't recommend using lang-default here.
It is now run for each test automatically, via TESTS_ENVIRONMENT.
* tests/dd/reblock: Don't source lang-default here.
* tests/misc/truncate-fail-diag: Likewise.
Jim Meyering [Sun, 29 Mar 2009 17:42:34 +0000 (19:42 +0200)]
tests: other-fs-tmpdir: remove just-added duplicate /usr/tmp
* tests/other-fs-tmpdir (CANDIDATE_TMP_DIRS): Remove 2nd /usr/tmp.
Jim Meyering [Sun, 29 Mar 2009 07:59:23 +0000 (09:59 +0200)]
tests: other-fs-tmpdir: avoid bogus "skipped **********..." diagnostic
* tests/other-fs-tmpdir (CANDIDATE_TMP_DIRS): Add /dev/shm to the list.
Don't use a line of asterisks as the first line of the "why skipped"
diagnostic, now that that line is printed on 'make check's stderr.
Jim Meyering [Sat, 28 Mar 2009 11:48:24 +0000 (12:48 +0100)]
build: use automake's --silent-rules option when possible
* bootstrap: Use automake's --silent-rules option.
Jim Meyering [Fri, 27 Mar 2009 16:33:26 +0000 (17:33 +0100)]
tests: test-argv-iter.c: avoid declaration-after-stmt
* gl/tests/test-argv-iter.c: Move declaration "up", so as not
to ruffle feathers of crufty compilers. We accommodate such
compilers here solely because this module may migrate to gnulib.
Tiny patch by Matthew Woehlke.
Eric Blake [Thu, 26 Mar 2009 12:39:14 +0000 (06:39 -0600)]
tests: fix 'make sc_tight_scope' on cygwin
* src/Makefile.am (sc_tight_scope): Ignore leading underscores in symbols.
Jim Meyering [Fri, 27 Mar 2009 08:26:03 +0000 (09:26 +0100)]
tests: avoid spurious test failure in set-GID build dir
* tests/install/install-C-root: Skip this test if in set-gid directory.
Reported by Sven Joachim and C de-Avillez.
Jim Meyering [Fri, 27 Mar 2009 08:23:35 +0000 (09:23 +0100)]
tests: migrate setgid-check into test-lib.sh
* tests/test-lib.sh (skip_if_setgid_): New function.
* tests/setgid-check: Remove file.
* tests/Makefile.am (EXTRA_DIST): Remove setgid-check.
* tests/chmod/c-option: Use the new function rather than sourcing
the separate file.
* tests/cp/cp-parents: Likewise.
* tests/install/install-C: Likewise.
* tests/mkdir/parents: Likewise.
* tests/mkdir/perm: Likewise.
Jim Meyering [Thu, 26 Mar 2009 09:33:37 +0000 (10:33 +0100)]
tests: mark the rm/ext3-perf test as "very" expensive
* tests/rm/ext3-perf: Relegate this test to the very_expensive
category, since it is failing too often for me, when run in
parallel (-j4) with other disk-hammering tests. This test fails
when it takes too long, so the fact that it fails under pressure
is an inconvenience more than a real problem.
Jim Meyering [Thu, 26 Mar 2009 09:20:43 +0000 (10:20 +0100)]
* gnulib: Update submodule to latest.
Paul Eggert [Thu, 26 Mar 2009 03:50:17 +0000 (20:50 -0700)]
df: port the known-value fix to AIX as well
* src/df.c (known_value): New function, which also works on AIX
/proc file systems.
(df_readable, show_dev): Use it instead of hardcoding comparison to
UINTMAX_MAX. Suggested by Jim Meyering and Matthew Woehlke.
Paul Eggert [Wed, 25 Mar 2009 21:16:46 +0000 (14:16 -0700)]
df: fix a bug when totaling unknown values
* src/df.c (show_dev): Don't add UINTMAX_MAX to grand totals, as that
value indicates that the true value is unknown; adding it effectively
subtracts 1 from the total, whereas we want to leave the total alone.
Jim Meyering [Wed, 25 Mar 2009 21:49:09 +0000 (22:49 +0100)]
tests: don't fail "make syntax-check" on cygwin
* maint.mk (NEWS_hash): Remove trailing " -" or " *-".
* cfg.mk (old_NEWS_hash): Regenerate, removing " -".
Reported by Eric Blake.
Eric Blake [Mon, 23 Mar 2009 20:48:19 +0000 (14:48 -0600)]
pwd: support -L and -P
* src/pwd.c (longopts): New variable.
(logical_getcwd): New function.
(main): Use it.
(usage): Document new options.
* doc/coreutils.texi (pwd invocation): Likewise.
* NEWS: Likewise.
* TODO (pwd): Mark it done.
* tests/misc/pwd-option: New file.
* tests/Makefile.am (TESTS): Add test.
* THANKS: Update.
Reported by Paul D. Smith, in savannah bug 24949.
Ondřej Vašík [Wed, 25 Mar 2009 11:04:11 +0000 (12:04 +0100)]
doc: update suffix regex in version sort
* doc/coreutils.texi: Reflect current filevercmp regex.
Ralf Wildenhues [Wed, 25 Mar 2009 09:15:46 +0000 (10:15 +0100)]
tests: CuTmpdir.pm: avoid use of undefined variable in warning
* tests/CuTmpdir.pm (chmod_tree): Do not warn if $dir is undefined.
Jim Meyering [Tue, 24 Mar 2009 21:33:47 +0000 (22:33 +0100)]
tests: ls -v: exercise yet another change in gnulib's new filevercmp
* tests/misc/ls-misc: Add names with ~ and ~.1~ suffixes.
Jim Meyering [Tue, 24 Mar 2009 20:49:03 +0000 (21:49 +0100)]
* gnulib: Update submodule to latest.
Kamil Dudka [Tue, 24 Mar 2009 11:16:20 +0000 (12:16 +0100)]
ls: make it possible to disable coloring of hard links
* src/ls.c (print_color_indicator): Don't color hard links if disabled
(when there is an empty hl= in the LS_COLORS environment variable).
* tests/ls/hardlink: Add test case for disabled hard link highlighting.
* NEWS: Mention the change.
Jim Meyering [Sat, 21 Mar 2009 08:43:56 +0000 (09:43 +0100)]
tests: search only files that are under version control
* maint.mk (sc_error_exit_success): Search only files
that are under version control.
(sc_makefile_path_separator_check): Likewise.
Check *.mk files, too, not just Makefile.am files.
Jim Meyering [Sat, 21 Mar 2009 08:52:41 +0000 (09:52 +0100)]
tests: follow own advice: use $(PATH_SEPARATOR), not ":"
* maint.mk (coreutils-path-check): Use the variable, not the literal.
Jim Meyering [Sat, 21 Mar 2009 08:56:00 +0000 (09:56 +0100)]
tests: modernize: use $(var) makefile notation, not @var@
* maint.mk (sc_makefile_path_separator_check): Recommend
the use of $(PATH_SEPARATOR), not @PATH_SEPARATOR@.
Remove an obsolete comment.
Jim Meyering [Fri, 20 Mar 2009 14:20:09 +0000 (15:20 +0100)]
tests: teach sc_dd_O_FLAGS about new syntax in dd.c
* cfg.mk (sc_dd_O_FLAGS): Adapt to new syntax.
Eric Blake [Thu, 19 Mar 2009 19:14:26 +0000 (20:14 +0100)]
dd: use a more portable definition of O_FULLBLOCK
* src/dd.c (O_FULLBLOCK): Compute its value without using a 180KB
macro. This avoids triggering a compilation failure with HP-UX's cc.
Reported by Matthew Woehlke.
Jim Meyering [Thu, 19 Mar 2009 18:04:19 +0000 (19:04 +0100)]
tests: avoid spurious mkdir/selinux test failure on HP-UX 11.23
* tests/mkdir/selinux: Accept yet another diagnostic.
Reported by Matthew Woehlke.
Jim Meyering [Thu, 19 Mar 2009 17:48:46 +0000 (18:48 +0100)]
tests: df/total-verify: avoid test failure with older Perl
* tests/df/total-verify: Don't use perl's -f option.
Reported by Matthew Woehlke.
Jim Meyering [Thu, 19 Mar 2009 06:55:03 +0000 (07:55 +0100)]
* gnulib: Update submodule to latest.
Jim Meyering [Tue, 17 Mar 2009 20:23:51 +0000 (21:23 +0100)]
tests: adjust sort-continue not to fail under valgrind
* tests/misc/sort-continue: Don't run cat inside fd-limited shell.
If sort fails to run in an fd-limited shell, skip the test.
Paul Eggert [Mon, 9 Mar 2009 21:56:13 +0000 (14:56 -0700)]
tests: add another sort/nmerge test
* tests/Makefile.am (TESTS): Add sort-merge-fdlimit.
* tests/misc/sort-merge-fdlimit: New file.
* doc/coreutils.texi (sort invocation): Document that we now silently
lower nmerge if necessary.
Patch by Paul Eggert, Nima Nikzad, Max Chang, Alexander Nguyen,
Sahil Amoli, and Nick Graham.
Paul Eggert [Fri, 13 Mar 2009 22:48:30 +0000 (15:48 -0700)]
sort: handle fd exhaustion better when merging
This is an alternative to my 9 March patch labeled "Silently lower
nmerge; don't (sometimes incorrectly) range-check"
<http://lists.gnu.org/archive/html/bug-coreutils/2009-03/msg00070.html>.
It differs by not using 'dup' to probe for extra file descriptors;
instead, it simply calls 'open' (and 'pipe') to open files and pipes,
until one of these calls fails due to file descriptor exhaustion; it
then backs off by 1, does a merge with the files that it has opened,
and then retries with the (now-smaller) number of files.
This patch requires quite a few more changes to the source code than
the earlier patch, but it is in some sense "better" because it doesn't
need to call "dup" ahead of time in order to decide whether "open" or
"pipe" will fail. Also, it's more robust in the case where "open" or
"pipe" fails with errno==EMFILE because some system-wide limit is
exhausted.
* src/sort.c (create_temp_file): New arg SURVIVE_FD_EXHAUSTION.
(stream_open): New function, containing guts of xfopen.
(xfopen): Use it.
(pipe_fork): Set errno on failure.
(maybe_create_temp): New function, containing guts of create_temp.
(create_temp): Use it.
(open_temp): Distinguish failures due to file descriptor exhaustion
from other failures, and on fd exhaustion return a notice to caller
rather than dying. Don't test execlp's return value; when it returns,
it *always* returns -1.
(open_input_files): New function.
(mergefps): New arg FPS. It's now the caller's responsibility to open
the input and output files. All callers changed.
(mergefiles): New function.
(avoid_trashing_input, merge): Handle the case where a single merge
can't merge as much as we wanted due to file descriptor exhaustion, by
merging as much as we can and then retrying.
* tests/Makefile.am (TESTS): Add misc/sort-continue.
* tests/misc/sort-continue: New file.
* THANKS: Add Glen Lenker and Matt Pham who coauthored this patch.
Jim Meyering [Wed, 18 Mar 2009 11:20:32 +0000 (12:20 +0100)]
maint: normalize leading-TAB indentation in Makefiles
* maint.mk (sc_makefile_TAB_only_indentation): New rule.
Replace each TAB+8-space sequence with two TABs.
* man/Makefile.am: Likewise.
* build-aux/check.mk: Likewise.
I used this command (run it more than once, if needed):
t=$'\t'; git grep -l -E "$t {8}"|grep -E 'Makefile|\.mk$' \
| xargs perl -pi -e 's/\t {8}/\t\t/'
Jim Meyering [Sun, 15 Mar 2009 20:40:54 +0000 (21:40 +0100)]
maint: remove long-unused Makefile
* tests/sort-time/Makefile: Remove long-unused file.
Jim Meyering [Sat, 14 Mar 2009 11:48:44 +0000 (12:48 +0100)]
maint: remove unused include and definitions
* gl/lib/randread.c (__attribute__): Remove unused definition.
(ATTRIBUTE_UNUSED): Likewise.
* gl/lib/selinux-at.c (_): Remove unused definition.
Don't include <gettext.h>. No longer needed.
Jim Meyering [Sun, 15 Mar 2009 07:09:53 +0000 (08:09 +0100)]
* gnulib: Update submodule to latest.
Jim Meyering [Sat, 14 Mar 2009 20:29:33 +0000 (21:29 +0100)]
system.h: correct compilation error: MAX not defined
* src/system.h (io_blksize): Move definition down, so it follows that
of MAX -- to avoid compilation failure on OpenSolaris. Reported by
David Bartley. Details in
http://lists.gnu.org/archive/html/bug-coreutils/2009-03/msg00190.html
Jim Meyering [Sat, 14 Mar 2009 10:49:56 +0000 (11:49 +0100)]
* gnulib: Update to pull in select/interix work-around.
Kamil Dudka [Wed, 25 Feb 2009 09:10:36 +0000 (10:10 +0100)]
doc: update "version sort" documentation
* doc/coreutils.texi: Reflect current filevercmp behavior.
Jim Meyering [Thu, 12 Mar 2009 13:41:38 +0000 (14:41 +0100)]
tests: groups-version: don't require that id be built
* tests/misc/groups-version: groups is now independent of id.
Jim Meyering [Thu, 12 Mar 2009 13:40:45 +0000 (14:40 +0100)]
tests: skip the groups-dash test when "groups" is not built
* tests/misc/groups-dash: Skip if groups is not being installed.
Suggested by Mike Frysinger.
Also fail if groups exits with nonzero status.
Pádraig Brady [Thu, 12 Mar 2009 14:16:54 +0000 (14:16 +0000)]
maint: mention Autoconf version required to build Automake
* README-prereq: coreutils currently requires an unreleased
version of Automake, so mention the minimum version of Autoconf
required to build that. Suggestion from James Youngman.
Jim Meyering [Thu, 12 Mar 2009 09:46:31 +0000 (10:46 +0100)]
* gnulib: Update submodule to latest.
Ondřej Vašík [Wed, 11 Mar 2009 15:08:20 +0000 (16:08 +0100)]
cp: make -a option preserve xattrs, but with reduced diagnostics
* copy.c (copy_attr_by_fd): Reduce xattr diagnostics for 'cp -a'.
(copy_attr_by_name): Likewise.
* cp.c (main): Preserve xattrs with -a option, when possible.
* doc/coreutils.texi: Document that xattrs are preserved with
cp -a, with no added diagnostics.
* NEWS: Mention the change.
* tests/misc/xattr: Add tests for 'cp --preserve=all' and 'cp -a'.
Pádraig Brady [Wed, 11 Mar 2009 12:09:08 +0000 (12:09 +0000)]
maint: Document how to best to enable gcc warnings for developers
* README-hacking: Mention --enable-gcc-warnings configure option
* configure.ac: Don't scare developers away from using gcc warnings
Jim Meyering [Tue, 10 Mar 2009 09:04:23 +0000 (10:04 +0100)]
maint: remove an obsolete FIXME comment
* src/Makefile.am (../AUTHORS): Remove obsolete FIXME comment.
There are no longer any installed *.sh programs.
Pádraig Brady [Wed, 11 Mar 2009 11:39:33 +0000 (11:39 +0000)]
cat: Fix immediate output of processed data
Introduced by commit
790892db, 2006-06-08 "Ensure that cat works ...".
* NEWS: Mention the bugfix.
* src/cat.c (cat): Fix the typo which stopped the writing
of processed data before a blocking read() is done.
* tests/misc/cat-buf: Add to ensure processed data is not buffered.
* tests/Makefile.am: Reference the new test.
Pádraig Brady [Fri, 6 Mar 2009 22:30:55 +0000 (22:30 +0000)]
cat,cp,mv,install,split: Set the minimum IO block size used to 32KiB
This is following on from this change:
[
02c3dc9d 2008-03-06 cat: use larger buffer sizes ...]
which increased the IO block size used by cat by 8 times,
but also capped it at 32KiB.
* NEWS: Mention the change in behavior.
* src/system.h: Add a new io_blksize() function that
returns the max of ST_BLKSIZE or 32KiB, as this was
seen as a good value for a minimum block size to use
to get good performance while minimizing system call overhead.
* src/cat.c: Use it.
* src/copy.c: ditto
* src/split.c: ditto
Jim Meyering [Sun, 8 Mar 2009 17:37:08 +0000 (18:37 +0100)]
tests: add a test for newly-fixed bug in comm --check-order
* tests/misc/comm (ooo-prefix): Add a test for today's fix.
* NEWS (Bug fixes): Mention it.
Bruno Haible [Sun, 8 Mar 2009 17:25:29 +0000 (18:25 +0100)]
comm: fix a bug in its new --check-order option
* src/comm.c: Include memcmp2.h.
(check_order): Use memcmp2 instead of memcmp.
* bootstrap.conf (gnulib_modules): Add memcmp2.
Jim Meyering [Sat, 7 Mar 2009 19:41:00 +0000 (20:41 +0100)]
tests: ls -v: exercise the bug fixed by gnulib's new filevercmp
* tests/misc/ls-misc (version-sort): New test.
(mk_file): New function.
Reported by Josh Triplett in <http://bugs.debian.org/517558>.
Details in http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/16902
* NEWS (Bug fixes): Mention it.
Jim Meyering [Sun, 8 Mar 2009 20:16:30 +0000 (21:16 +0100)]
gnulib: sync submodule to latest
Jim Meyering [Fri, 6 Mar 2009 19:03:26 +0000 (20:03 +0100)]
pr: fix a bug whereby --indent=N (-o) did not indent header lines
* src/pr.c (print_header): Honor chars_per_margin also for the header.
Inspired by a patch from C de-Avillez.
This change also adjusts two other %*s directives to be %*.*s instead,
(for the padding on either side of "file_text") to correct a bug that
could make pr print one space of padding instead of none.
* NEWS (Bug fixes): Mention it.
* tests/pr/o3Jml24f-lm-lo: Adjust for new expected output.
* tests/pr/o3a3Sl24f-tn: Likewise.
* tests/pr/o3a3Snl24f-tn: Likewise.
* tests/pr/o3a3l24f-tn: Likewise.
* tests/pr/o3b3Sl24f-tn: Likewise.
* tests/pr/o3b3Snl24f-tn: Likewise.
* tests/pr/o3b3l24f-tn: Likewise.
* tests/pr/o3mSl24f-bl-tn: Likewise.
* tests/pr/o3mSnl24fbltn: Likewise.
* tests/pr/o3ml24f-bl-tn: Likewise.
Jim Meyering [Sat, 7 Mar 2009 15:20:55 +0000 (16:20 +0100)]
bootstrap: add to XGETTEXT_OPTIONS
* bootstrap.conf (XGETTEXT_OPTIONS): Add c-format --flag options
for verror and verror_at_line; alphabetize.
Jim Meyering [Sat, 7 Mar 2009 09:19:46 +0000 (10:19 +0100)]
maint: use a git submodule for gnulib
* .gitmodules: New file, to track gnulib.
* gnulib: New file, created by running this:
git submodule add git://git.sv.gnu.org/gnulib.git gnulib
Jim Meyering [Sat, 7 Mar 2009 15:11:54 +0000 (16:11 +0100)]
build: allow ./bootstrap --srcdir=... to work with a git submodule
* bootstrap (git_modules_config): Use GIT_CONFIG, not GIT_CONFIG_LOCAL.
The latter was removed in v1.5.6.1-109-gdc87183.
Jim Meyering [Sat, 7 Mar 2009 12:27:29 +0000 (13:27 +0100)]
build: avoid "multiply defined..." warning from automake
* Makefile.am (AM_CFLAGS): Add $(WARN_CFLAGS) via +=, not =, to
avoid "multiply defined..." warning from automake.
Jim Meyering [Fri, 6 Mar 2009 09:27:43 +0000 (10:27 +0100)]
cat: use larger buffer sizes to reduce read/write-syscall overhead
* src/cat.c (max): Remove definition. Use MAX from system.h instead.
(compute_buffer_size): New function to compute the input and output
buffer sizes, which are now set at 8 times st_blksize with a maximum
of 32KiB. Previously the typical block sizes used were 1KiB for pipes
and 4KiB for files, and now will be 8KiB and 32KiB respectively.
(main): Use it.
This change can double throughput on modern systems. For timings,
see http://article.gmane.org/gmane.comp.gnu.core-utils.bugs/16040
Suggestion from Tzvi Rotshtein.
Jim Meyering [Fri, 6 Mar 2009 09:49:18 +0000 (10:49 +0100)]
maint: avoid new unused-macro warnings from gcc
* src/ls.c (ASSERT_MATCHING_DEV_INO): Comment-out unused definition,
to match commented-out use.
* src/install.c (INITIAL_HASH_MODULE, INITIAL_ENTRY_TAB_SIZE):
(READ_SIZE): Remove definitions of unused macros.
* src/mv.c (INITIAL_HASH_MODULE, INITIAL_ENTRY_TAB_SIZE): Likewise.
* src/dd.c (max): Likewise.
* src/expand.c (OUTPUT_BLOCK): Likewise.
* src/csplit.c (ALLOC_SIZE): Likewise.
* src/pr.c (NULLCOL): Likewise.
* src/ptx.c (Sword): Likewise.
* src/unexpand.c (OUTPUT_BLOCK): Likewise.
* src/factor.c (NDEBUG): Likewise.