Jim Meyering [Tue, 22 Mar 2011 05:09:06 +0000 (06:09 +0100)]
build: update gnulib submodule to latest
Jim Meyering [Tue, 22 Mar 2011 05:07:35 +0000 (06:07 +0100)]
tests: exercise tests new "==" operator
* tests/misc/test: Exercise the new operator.
* NEWS (Changes in behavior): Mention it.
David A. Wheeler [Tue, 22 Mar 2011 05:03:55 +0000 (06:03 +0100)]
test: accept "==" as a synonym for "="
Make GNU coreutils' test recognize "==" as a synonym for "=".
This is already the case in GNU coreutils' expr, bash, ksh,
busybox ash, FreeBSD-current /bin/sh and /bin/test, and
OpenBSD's /bin/sh.
Before, env test a '==' a would fail with this diagnostic:
"test: ==: binary operator expected". Now, it succeeds.
* src/test.c: Accept "==" as a synonym for "=".
* doc/coreutils.texi (String tests): Document it.
Reported as http://debbugs.gnu.org/8263
Also see http://austingroupbugs.net/view.php?id=375
Jim Meyering [Sun, 20 Mar 2011 07:56:06 +0000 (08:56 +0100)]
tests: fix a bug in the cp/preserve-gid test
* tests/cp/preserve-gid: Ensure that every process under test uses
the cp binary we've just built. Before this fix, with a restrictive
umask or build-dir permissions, the UID-changing tests would end up
using whatever cp happened to be available through $PATH
Analysis by arbogast.cedric@gmail.com in http://debbugs.gnu.org/8292.
Paul Eggert [Sun, 20 Mar 2011 19:24:57 +0000 (12:24 -0700)]
bootstrap: do not exclude m4/message.m4
* bootstrap.conf (excluded_files): Don't exclude m4/lcmessage.m4,
as it's needed with the latest gnulib.
Jim Meyering [Sat, 19 Mar 2011 15:55:35 +0000 (16:55 +0100)]
maint: remove a name from THANKS.in that is derived from git log
The names in THANKS are generated from two sources: the hard-coded
list, THANKS.in, and the names of committers from the git log.
When a contributor on the hard-coded list commits a change,
we remove their now-redundant name from THANKS.in.
* THANKS.in: Remove a now-duplicate name.
Pádraig Brady [Sat, 19 Mar 2011 23:58:49 +0000 (23:58 +0000)]
tests: fix the sparse-fiemap test
* tests/filefrag-extent-compare: Merge adjacent extents in
each list before processing, so we correctly account for
split extents in either list.
* tests/cp/sparse-fiemap: Remove the explicit syncing,
which was only changing the way extents were arranged,
and thus working around the extent comparison issue
that was seen on ext4 loop back.
Jim Meyering [Wed, 16 Mar 2011 15:09:31 +0000 (16:09 +0100)]
sort: avoid memory pressure of 130MB/thread when reading from pipe
* src/sort.c (INPUT_FILE_SIZE_GUESS): Decrease initial allocation
factor used to size buffer used when reading a non-regular file.
For motivation, see discussion here:
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/878/focus=887
Jim Meyering [Mon, 14 Mar 2011 13:26:38 +0000 (14:26 +0100)]
maint: stop using .x-sc_* files to list syntax-check exemptions
Instead, use the brand new mechanism with which you merely use a
variable (derived from the rule name) defined in cfg.mk to an ERE
matching the exempted file names.
* gnulib: Update to latest, to get maint.mk that implements this.
* Makefile.am (syntax_check_exceptions): Remove variable.
(EXTRA_DIST): Remove use of the variable.
* cfg.mk (sc_x_sc_dist_check): Remove rule, no longer useful.
(exclude_file_name_regexp--sc_space_tab): Define variable.
(exclude_file_name_regexp--sc_bindtextdomain): Likewise.
(exclude_file_name_regexp--sc_unmarked_diagnostics): Likewise.
(exclude_file_name_regexp--sc_error_message_uppercase): Likewise.
(exclude_file_name_regexp--sc_trailing_blank): Likewise.
(exclude_file_name_regexp--sc_system_h_headers): Likewise.
(exclude_file_name_regexp--sc_require_config_h_first): Likewise.
(exclude_file_name_regexp--sc_require_config_h): Likewise.
(exclude_file_name_regexp--sc_po_check): Likewise.
(exclude_file_name_regexp--sc_prohibit_always-defined_macros): Likewise.
(exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF): Likewise.
(exclude_file_name_regexp--sc_program_name): Likewise.
(exclude_file_name_regexp--sc_file_system): Likewise.
(exclude_file_name_regexp--sc_prohibit_always_true_header_tests):
Likewise.
(exclude_file_name_regexp--sc_prohibit_fail_0): Likewise.
(exclude_file_name_regexp--sc_prohibit_atoi_atof): Likewise.
(exclude_file_name_regexp--sc_prohibit_tab_based_indentation): Likewise.
(exclude_file_name_regexp--sc_prohibit_stat_st_blocks): Likewise.
* configure.ac [whether localtime caches TZ]: Use return 0/1, not
exit (0/1) to avoid triggering a sc_prohibit_magic_number_exit failure.
* .x-sc_GPL_version: Remove file.
* .x-sc_bindtextdomain: Likewise.
* .x-sc_error_message_uppercase: Likewise.
* .x-sc_file_system: Likewise.
* .x-sc_obsolete_symbols: Likewise.
* .x-sc_po_check: Likewise.
* .x-sc_program_name: Likewise.
* .x-sc_prohibit_always-defined_macros: Likewise.
* .x-sc_prohibit_always_true_header_tests: Likewise.
* .x-sc_prohibit_atoi_atof: Likewise.
* .x-sc_prohibit_empty_lines_at_EOF: Likewise.
* .x-sc_prohibit_fail_0: Likewise.
* .x-sc_prohibit_magic_number_exit: Likewise.
* .x-sc_prohibit_stat_st_blocks: Likewise.
* .x-sc_prohibit_strcmp: Likewise.
* .x-sc_prohibit_tab_based_indentation: Likewise.
* .x-sc_require_config_h: Likewise.
* .x-sc_require_config_h_first: Likewise.
* .x-sc_space_tab (config): Likewise.
* .x-sc_sun_os_names: Likewise.
* .x-sc_system_h_headers: Likewise.
* .x-sc_trailing_blank: Likewise.
* .x-sc_unmarked_diagnostics: Likewise.
* .x-sc_useless_cpp_parens: Likewise.
Pádraig Brady [Sat, 12 Mar 2011 14:22:54 +0000 (14:22 +0000)]
maint: use wcswidth from gnulib
* gl/lib/mbsalign.c (rpl_wcswidth): Remove this in favor
of the equivalent wcswidth replacement in gnulib.
* bootstrap.conf: Depend on the wcswidth module.
Suggested by Bruno Haible.
Jim Meyering [Sun, 13 Mar 2011 12:20:54 +0000 (13:20 +0100)]
touch: update to latest gnulib to fix Solaris 10 touch segfault
* gnulib: Update to latest, to address http://debbugs.gnu.org/8230.
When built on Solaris 9 and run on Solaris 10, touch would segfault.
Reported by Ben Walton.
* bootstrap: Update from gnulib.
* tests/init.sh: Likewise.
* NEWS (Bug fixes): Mention this.
Jim Meyering [Thu, 10 Feb 2011 07:48:27 +0000 (08:48 +0100)]
sort: spawn fewer threads for small inputs
* src/sort.c (SUBTHREAD_LINES_HEURISTIC): Do not spawn a new thread
for every 4 lines. Increase this from 4 to 128K. 128K lines seems
appropriate for a 5-year-old dual-core laptop, but it is too low for
some common combinations of short lines and/or newer systems.
* NEWS (Bug fixes): Mention it.
Pádraig Brady [Mon, 7 Mar 2011 08:34:35 +0000 (08:34 +0000)]
copy: merge similar extents before processing
* src/extent-scan.c (extent_scan_read): Merge adjacent extents
that vary only in size, so that we may process them more efficiently.
This will be especially useful when we introduce fallocate()
so that we don't reproduce fragmentation in the destination.
Paul Eggert [Sat, 5 Mar 2011 08:14:25 +0000 (00:14 -0800)]
dd: avoid or diagnose some problems with short reads
* src/dd.c (warn_partial_read): New static var.
(iread): Diagnose partial reads if needed.
(iwrite): Don't diagnose them here; not needed any more.
(scanargs): Determine whether partial reads should be diagnosted.
Pádraig Brady [Sat, 5 Mar 2011 09:53:00 +0000 (09:53 +0000)]
maint: fix dd nocache test to be independent of current stdin
* tests/dd/nocache: Don't assume stdin is a pipe
Pádraig Brady [Tue, 22 Feb 2011 21:14:00 +0000 (21:14 +0000)]
dd: add a flag to discard cached data
* src/dd.c (FFS_MASK): A new macro (Find First Set) refactored
from the following enum as it's now used twice.
(usage): Mention the new 'nocache' flag.
(cache_round): A new function to help ignore requests
to drop cache, that are less than page_size.
(invalidate_cache): A new function to call posix_fadvise()
with the appropriate offset and length. Note we don't
use fdadvise() so we can detect errors when count=0.
(dd_copy): Call invalidate_cache() for the portions read.
(iwrite): Likewise for the portions written.
(main): Call invalidate_cache for page_size slop or
for full file when count=0.
* cfg.mk (sc_dd_O_FLAGS): Adjust to pass.
* doc/coreutils.texi (dd invocation): Describe the 'nocache' flag,
and give some examples of how it can be used.
* tests/dd/nocache: A new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature.
Pádraig Brady [Sat, 5 Mar 2011 02:21:45 +0000 (02:21 +0000)]
doc: clarify that `dd bs=` can immediately output short reads
* doc/coreutils.texi (dd invocation): Clarify that bs= can
cause parital reads to be immediately written to output.
* src/dd.c (usage): Hint that bs= can cause partial writes.
See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8171
Pádraig Brady [Fri, 25 Feb 2011 08:16:23 +0000 (08:16 +0000)]
doc: group dd conv= options that are actually flags
* src/dd.c (usage): Move 'sync' up with other data transformation
options. Having it alongside 'fsync' and 'fdatasync' is
particularly confusing. Also the double line description of
the 'sync' option, serves as a visual break from the "flag"
type options that follow.
* doc/coreutils.texi (dd invocation): Apply the same grouping
as above, by splitting the "conv=" table in two.
Pádraig Brady [Fri, 25 Feb 2011 12:27:25 +0000 (12:27 +0000)]
dd: warn when we disable oflag=direct not at EOF
An alternative to this is to auto enable iflag=fullblock
when oflag=direct and bs= is specified.
It was thought better though, to warn about the specific issue,
and give full control of dd's options to the user.
* src/dd.c (iwrite): Warn, when we write after having
disabled O_DIRECT.
See https://bugzilla.redhat.com/show_bug.cgi?id=614605
Jim Meyering [Wed, 2 Mar 2011 18:16:46 +0000 (19:16 +0100)]
du: don't infloop for --files0-from=DIR
* src/du.c (main): Fail on AI_ERR_READ error, rather than merely
diagnosing and continuing. Based on a patch by Stefan Vargyas.
Also move the handling of AI_ERR_EOF into the case stmt.
Do not report ferror/fclose(stdin) failure when we've
already diagnosed e.g., failure to read the DIR, above.
Bug introduced by 2008-11-24 commit
031e2fb5, "du: read and
process --files0-from= input a name at a time,".
* src/wc.c: Handle read failure as with du: do not exit
immediately, but rather go on to print any total and to clean-up.
As above, move the handling of AI_ERR_EOF into the case stmt.
* tests/du/files0-from-dir: New file, to test both du and wc.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
Jim Meyering [Wed, 2 Mar 2011 17:54:43 +0000 (18:54 +0100)]
wc: avoid NULL dereference on out-of-memory error
* src/wc.c (main): Diagnose failed argv_iter_init_* failure,
rather than falling through and dereferencing NULL.
Bug introduced by 2008-11-25 commit
c2e56e0d,
"wc: read and process --files0-from= input a name at a time,".
* NEWS (Bug fixes): Mention it.
Pádraig Brady [Thu, 24 Feb 2011 10:25:52 +0000 (10:25 +0000)]
tests: without filefrag, only skip part of sparse-fiemap
* tests/cp/sparse-fiemap: Move the PERL check to the top,
since we don't test anything without it. In the loop,
don't use skip_test_ as it exits the test completely.
Gilles Espinasse [Sat, 19 Feb 2011 23:09:09 +0000 (23:09 +0000)]
maint: replace spaces with tab in tests/Makefile.am
* tests/Makefile.am: Replace spaces with tabs for consistency
Jim Meyering [Fri, 18 Feb 2011 22:29:14 +0000 (23:29 +0100)]
stdbuf: avoid even the appearance of a possible use-after-free
There was an execution path by which "libstdbuf" could be used after
being freed, but that would happen only if there were no libstdbuf.so
alongside the stdbuf program and there had been an installation error
leading to absence of the file, PKGLIBDIR/libstdbuf.so.
* src/stdbuf.c (set_LD_PRELOAD): Rearrange loop to make it perfectly
clear that there is no possibility of use-after-free.
Steve Grubb reported this possible use-after-free of "libstdbuf".
Gilles Espinasse [Fri, 18 Feb 2011 12:52:21 +0000 (13:52 +0100)]
tests: correct racy sparse-to-pipe test
* tests/cp/sparse-to-pipe: Wait for backgrounded "cat" to complete
before comparing the results.
Paul Eggert [Tue, 15 Feb 2011 05:05:18 +0000 (21:05 -0800)]
uptime: omit unnecessary #if
* src/uptime.c (print_uptime): Omit unnecessary "#if defined
HAVE_GETLOADAVG || defined C_GETLOADAVG". This #if is always
true, and removing it will help us simplify the gnulib getloadavg
module.
Pádraig Brady [Sun, 13 Feb 2011 18:56:10 +0000 (18:56 +0000)]
tests: support more file systems in the cp fiemap tests
* tests/cp/sparse-fiemap: Check for fiemap support against a file
rather than a directory to enable tests on BTRFS for example.
Explicity disable the test on ext3 or file systems where we
can't determine the type.
* tests/cp/fiemap-perf: Likewise. Also disable the test on older
BTRFS (like in Fedora 14), where extents are returned for holes.
* tests/init.cfg: Comment that BTRFS only supports fiemap
for regular files.
Pádraig Brady [Thu, 10 Feb 2011 17:47:21 +0000 (17:47 +0000)]
copy: adjust fiemap handling of sparse files
Don't depend on heuristics to detect sparse files
if fiemap is available. Also don't scan for new
holes unless --sparse=always has been specified.
* src/copy.c (extent_copy): Pass the user specified
sparse mode, and handle as described above.
Pádraig Brady [Tue, 8 Feb 2011 19:16:55 +0000 (19:16 +0000)]
copy: suppress redundant lseeks when using fiemap
* src/copy.c (extent_copy): Suppress redundant lseek()s in both
the source and dest files, when there is no hole between extents.
Jim Meyering [Thu, 10 Feb 2011 09:01:23 +0000 (10:01 +0100)]
tests: fix bug in preceding check.mk change
* tests/check.mk (.built-programs): Run cd'd submake in a subshell
so the redirected output ends up in the current directory, not ../src.
Jim Meyering [Wed, 9 Feb 2011 07:29:38 +0000 (08:29 +0100)]
tests: print "python missing:..." diagnostic where more will see it
* tests/init.cfg (fiemap_capable_): Print with warn_, so that the
diagnostic shows up alongside the corresponding SKIP message.
Jim Meyering [Wed, 9 Feb 2011 16:08:58 +0000 (17:08 +0100)]
tests: avoid gross inefficiency in "make test"
Do not run a sub-make to set up the environment for each
and every test script. Instead, run it just once and store
the result in a file.
* tests/check.mk (built_programs): Remove definition.
(.built-programs): New rule to create the temporary file.
(CLEANFILES): Arrange to remove it.
(TESTS_ENVIRONMENT): Simply cat .built-programs, rather than
running the sub-make.
* .gitignore: Ignore it.
Pádraig Brady [Fri, 4 Feb 2011 22:05:20 +0000 (22:05 +0000)]
test: improve the cp fiemap tests
* tests/cp/fiemap-2: Enable the fiemap check for files, which
will enable the test for files on ext3.
* tests/cp/fiemap-perf: Comment why we're not enabling for ext3.
* tests/cp/sparse-fiemap: Ditto. Also sync the files before
doing a fiemap which was needed for ext4 loop back at least.
Add a comment that FIEMAP_FLAG_SYNC is ineffective, thus
requiring the explicit syncs.
* tests/fiemap-capable: A new python script to determine
if a specified path supports fiemap.
* tests/init.cfg (fiemap_capable_): Use the new python script.
* tests/Makefile.am (EXTRA_DIST): Include the new python script.
Jim Meyering [Tue, 8 Feb 2011 07:25:46 +0000 (08:25 +0100)]
tests: randread-tests: use macro.h not "#define ASSERT..."
* gl/modules/randread-tests (Files): Add tests/macros.h
* gl/tests/test-rand-isaac.c: Remove now-unneeded #include directives.
(ASSERT): Remove definition.
Instead, include "macros.h".
Prompted by suggestions from Bruno Haible.
Jim Meyering [Mon, 7 Feb 2011 15:24:10 +0000 (16:24 +0100)]
maint: move di-set and ino-map modules from ./gl to gnulib
* gl/lib/di-set.c: Remove file.
* gl/lib/di-set.h: Likewise.
* gl/lib/ino-map.c: Likewise.
* gl/lib/ino-map.h: Likewise.
* gl/modules/di-set: Likewise.
* gl/modules/di-set-tests: Likewise.
* gl/modules/ino-map: Likewise.
* gl/modules/ino-map-tests: Likewise.
* gl/tests/test-di-set.c: Likewise.
* gl/tests/test-ino-map.c: Likewise.
* gnulib: Update to latest, now that these two modules are there.
Jim Meyering [Mon, 7 Feb 2011 14:46:09 +0000 (15:46 +0100)]
di-set: provide a lookup method
This is required for patch, and hence is about to move to gnulib.
* gl/lib/di-set.c (di_set_lookup): New function.
* gl/lib/di-set.h: Declare it.
* gl/tests/test-di-set.c (main): Exercise it.
The bug was introduced on 2004-12-04 via commit
7380cf79.
Jim Meyering [Mon, 7 Feb 2011 07:29:33 +0000 (08:29 +0100)]
cut: don't segfault for large unbounded range
* src/cut.c (set_fields): When computing the maximum range endpoint,
take into consideration the start of any unbounded range, like "999-".
* NEWS (Bug fixes): Mention it.
* tests/misc/cut (big-unbounded-b,c,f): Add tests.
Reported by Paul Marinescu in http://debbugs.gnu.org/7993
The bug was introduced on 2004-12-04 via commit
7380cf79.
Jim Meyering [Sat, 5 Feb 2011 21:40:57 +0000 (22:40 +0100)]
copy: don't let a failed lseek go undiagnosed
Upon failed lseek, sparse_copy_finalize would mistakenly return true.
Admittedly, that is very unlikely, since that particular lseek
is attempted only if the preceding call to sparse_copy induced
a hole at EOF (via lseek on the destination FD). However, now
that sparse_copy has an output parameter, N_READ, there is no
longer any reason to call lseek (fd, 0, SEEK_CUR), so...
* src/copy.c (sparse_copy_finalize): Remove the function.
(copy_reg): Call ftruncate with n_read, rather than
sparse_copy_finalize with its now-unnecessary lseek.
Lasse Collin spotted the bug in sparse_copy_finalize.
Jim Meyering [Fri, 4 Feb 2011 17:19:36 +0000 (18:19 +0100)]
post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
Jim Meyering [Fri, 4 Feb 2011 17:08:12 +0000 (18:08 +0100)]
version 8.10
* NEWS: Record release date.
Jim Meyering [Fri, 4 Feb 2011 14:01:39 +0000 (15:01 +0100)]
cp: avoid spurious failure on any non-linux kernel
* src/extent-scan.c (extent_scan_read) [!linux]: Always set
scan->initial_scan_failed so caller knows not to report the failure.
Jim Meyering [Fri, 4 Feb 2011 13:59:03 +0000 (14:59 +0100)]
tests: skip mv/i-3 on FreeBSD to avoid spurious failure
* tests/mv/i-3: Skip when uname -s reports FreeBSD.
Pádraig Brady [Wed, 2 Feb 2011 23:08:42 +0000 (23:08 +0000)]
sort: fix --debug key highlighting when key start after key end
This case was overlooked in commit
bdde34f9, 2010-08-05,
"sort: tune and refactor --debug code, and fix minor underlining bug"
* src/sort.c (debug_key): Don't adjust the key end when
it's before the key start.
* tests/misc/sort-debug-keys: Add a test case.
Jim Meyering [Wed, 2 Feb 2011 11:02:22 +0000 (12:02 +0100)]
tests: correct part of fiemap-perf
* tests/cp/fiemap-perf: Correct erroneous added test.
Since nonexistent names were used, the final test ended up
being "test =", which would always "succeed".
Jim Meyering [Wed, 2 Feb 2011 08:40:55 +0000 (09:40 +0100)]
build: update gnulib submodule to latest
Pádraig Brady [Mon, 31 Jan 2011 22:04:35 +0000 (22:04 +0000)]
cp: fix the buffer size used when writing zeros
* src/copy.c (write_zeros): This bug caused 4 or 8 bytes to
be written at a time which is very inefficient. One could
trigger the issue with `cp --sparse=never sparse non-sparse`
on a file system that supports fiemap.
Jim Meyering [Mon, 31 Jan 2011 19:55:34 +0000 (20:55 +0100)]
cp: always initialize extent_copy's output parameter
* src/copy.c (extent_copy): Otherwise it would be used uninitialized.
Jim Meyering [Mon, 31 Jan 2011 16:18:13 +0000 (17:18 +0100)]
tests: exercise a rarely-used corner of copy.c
* tests/cp/sparse-to-pipe: New test.
* tests/Makefile.am (TESTS): Add it.
Jim Meyering [Mon, 31 Jan 2011 17:25:58 +0000 (18:25 +0100)]
cp: fix copying a sparse file to a pipe
The recent FIEMAP-related changes made it so the unusual case of
copying a sparse file to a non-regular destination (e.g., a pipe)
would erroneously write one byte too many to that destination.
That happened because extent_copy assumed that it could use lseek
to obtain the number of bytes written to the output file descriptor.
That was valid only for regular files.
* src/copy.c (sparse_copy): Add a parameter, to be used by extent_copy,
but not by reg_copy. Adjust callers.
(extent_copy): Maintain new local, dest_pos, using new arg, n_read.
Don't call lseek on dest_fd; use new var, dest_pos, instead.
(copy_reg): Add unused arg.
Jim Meyering [Mon, 31 Jan 2011 14:20:29 +0000 (15:20 +0100)]
tests: gfs2 also supports FIEMAP
* tests/init.cfg (fiemap_capable_): Add gfs2.
Clarify comment.
Jim Meyering [Mon, 31 Jan 2011 14:17:12 +0000 (15:17 +0100)]
maint: remove unused inclusion of "fiemap.h"
* src/copy.c: Don't include "fiemap.h".
Jim Meyering [Mon, 31 Jan 2011 13:44:15 +0000 (14:44 +0100)]
tests: factor fiemap-related predicate into init.cfg
* tests/init.cfg (fiemap_capable_): New function.
* tests/cp/fiemap-perf: Use it.
* tests/cp/sparse-fiemap: Likewise.
* tests/cp/fiemap-2: Likewise.
Jim Meyering [Mon, 31 Jan 2011 12:42:49 +0000 (13:42 +0100)]
maint: update copyright year lists in new files
* src/extent-scan.h: Update copyright year list.
* src/extent-scan.c: Likewise.
* tests/cp/sparse-fiemap: Likewise.
Jim Meyering [Mon, 31 Jan 2011 12:40:26 +0000 (13:40 +0100)]
tests: modernize sparse-fiemap test
* tests/cp/sparse-fiemap: Use print_ver_, not open-coded VERBOSE test.
Jim Meyering [Mon, 31 Jan 2011 12:35:55 +0000 (13:35 +0100)]
tests: remove duplicate fiemap-perf test
* tests/cp/fiemap-perf: Copy block-comparing code from sparse-fiemap.
* tests/cp/sparse-fiemap: The same test was here, alongside a much
more involved test. Remove it, now that it is in its own file.
Jim Meyering [Sun, 30 Jan 2011 20:27:12 +0000 (21:27 +0100)]
doc: NEWS: mention cp's improvement
* NEWS (New Features): cp now copies sparse files efficiently.
Jim Meyering [Sun, 30 Jan 2011 08:21:57 +0000 (09:21 +0100)]
build: update gnulib submodule to latest
Jim Meyering [Sun, 30 Jan 2011 15:12:56 +0000 (16:12 +0100)]
copy, tee: assume EINTR is always defined: remove #ifdefs
Don't use "#ifdef EINTR". dd.c has been doing that since 2004.
* src/copy.c (sparse_copy): Remove #ifdef...#endif around EINTR use.
* src/tee.c (tee_files): Remove #ifdef...#endif around EINTR use.
If we need it, add something like this in system.h:
/* When EINTR is not defined, define it to an improbable value
so that each use does not have to be #ifdef'd. */
#ifndef EINTR
# define EINTR 999988
#endif
Jim Meyering [Thu, 27 Jan 2011 14:17:42 +0000 (15:17 +0100)]
tests: cp/fiemap: exercise previously-failing parts
* tests/cp/fiemap-2: New test.
* tests/Makefile.am (TESTS): Add it.
Jim Meyering [Fri, 28 Jan 2011 20:19:50 +0000 (21:19 +0100)]
copy: make extent_copy use sparse_copy, rather than its own code
* src/copy.c (extent_copy): Before this change, extent_copy would fail
to create holes, thus breaking --sparse=auto and --sparse=always.
I.e., copying a large enough file of all zeros, cp --sparse=always
should introduce a hole, but with extent_copy, it would not.
Jim Meyering [Thu, 27 Jan 2011 20:01:07 +0000 (21:01 +0100)]
copy: remove obsolete comment
* src/copy.c (sparse_copy): Remove now-obsolete comment about
how we used to work around lack of ftruncate. Combine nested
if conditions into one.
Jim Meyering [Thu, 27 Jan 2011 19:57:17 +0000 (20:57 +0100)]
copy: factor sparse-copying code into its own function, because
we're going to have to use it from within extent_copy, too.
* src/copy.c (sparse_copy): New function, factored out of...
(copy_reg): ...here.
Remove now-unused locals.
Jim Meyering [Thu, 27 Jan 2011 16:49:04 +0000 (17:49 +0100)]
fiemap copy: avoid leak-on-error
* src/copy.c (extent_copy): Don't leak an extent_scan buffer on
failed lseek, read, or write.
Jim Meyering [Thu, 27 Jan 2011 16:30:08 +0000 (17:30 +0100)]
fiemap copy: avoid a performance hit due to very small buffer
* src/copy.c (extent_copy): Don't let what should have been a
temporary reduction of buf_size (to handle a short ext_len) become
permanent and thus impact the performance of all further iterations.
Jim Meyering [Thu, 27 Jan 2011 18:00:48 +0000 (19:00 +0100)]
fiemap copy: simplify post-loop logic; improve comments
* src/copy.c (extent_copy): Avoid duplication in post-loop
extend-to-desired-length code.
Jim Meyering [Thu, 27 Jan 2011 17:28:25 +0000 (18:28 +0100)]
fiemap copy: rename some locals
(extent_copy): Rename locals: s/*ext_logical/*ext_start/
Jim Meyering [Fri, 28 Jan 2011 21:31:23 +0000 (22:31 +0100)]
tests: ensure that FIEMAP-enabled cp copies a sparse file efficiently
* tests/cp/fiemap-perf: New file.
* tests/Makefile.am (TESTS): Add it.
Jim Meyering [Sat, 22 Jan 2011 12:09:08 +0000 (13:09 +0100)]
copy: don't allocate a separate buffer just for extent-based copy
* src/copy.c (copy_reg): Move use of extent_scan to just *after*
we allocate the main copying buffer, so we can...
(extent_scan): Take a new parameter, BUF, and use that rather
than allocating a private buffer. Update caller.
Jim Meyering [Sat, 22 Jan 2011 11:55:58 +0000 (12:55 +0100)]
copy: tweak variable name; improve a comment
* src/copy.c (copy_reg): Rename a variable to make more sense from
caller's perspective: s/require_normal_copy/normal_copy_required/.
This is an output-only variable, and the original name could make
it look like an input (or i&o) variable.
Jim Meyering [Sat, 22 Jan 2011 11:36:03 +0000 (12:36 +0100)]
copy: call extent_copy also when make_holes is false, ...
so that we benefit from using extents also when reading a sparse
input file with --sparse=never.
* src/copy.c (copy_reg): Remove erroneous test of "make_holes"
so that we call extent_copy also when make_holes is false.
Otherwise, what's the point of that parameter?
Jim Meyering [Sat, 22 Jan 2011 11:30:21 +0000 (12:30 +0100)]
copy: remove else-after-goto and adjust indentation
* src/copy.c (copy_reg): Remove useless else-after-goto.
Jim Meyering [Mon, 11 Oct 2010 09:19:02 +0000 (11:19 +0200)]
extent-scan: adjust naming and formatting
* src/extent-scan.h [struct extent_scan]: Rename member:
s/hit_last_extent/hit_final_extent/. "final" is clearer,
since "last" can be interpreted as "preceding".
Rename extent-scan functions to start with extent_scan_.
* src/Makefile.am (copy_sources): Also distribute extent-scan.h.
* src/extent-scan.c: Don't include error.h or quote.h. Neither is used.
* src/copy.c: shorten a comment to fit in 80 columns
* src/extent-scan.c, src/extent-scan.h: Correct formatting.
Jim Meyering [Mon, 11 Oct 2010 08:39:50 +0000 (10:39 +0200)]
fiemap copy: don't let write failure go unreported; adjust style, etc.
* src/copy.c (write_zeros): Add comments.
(extent_copy): Move decls of "ok" and "i" down to scope where used.
Adjust comments.
Rename local: s/holes_len/hole_size/
Print a diagnostic upon failure to write zeros.
jeff.liu [Wed, 29 Sep 2010 08:11:41 +0000 (16:11 +0800)]
fiemap copy: add extent-scan.[ch], avoid a double-free and reorganize
Changes:
========
1. fix write_zeros() per Jim's comments.
2. remove char const *fname from struct extent_scan.
3. change the signature of open_extent_scan() from
"void open_extent_scan(struct extent_scan **scan)" to
"void open_extent_scan(struct extent_scan *scan)" to avoid having
to malloc the extent_scan variable; instead save it on the stack.
4. move close_extent_scan() from a function defined in extent-scan.c
to extent-scan.h as a macro definition, but it does nothing for now,
since initial extent scan defined at stack.
5. add a macro "free_extents_info()" defined at extent-scan.h to
release the memory allocated to extent info which should be called
combine with get_extents_info(), it just one line, so IMHO, define
it as macro should be ok.
* src/extent-scan.c: New file; functions to read "extents".
* src/extent-scan.h: Header file of extent-scan.c.
* src/Makefile.am: Reference it and link it to copy_source.
* src/copy.c: Use the new functions and avoid double-free.
Jim Meyering [Sun, 13 Jun 2010 14:34:42 +0000 (16:34 +0200)]
build: distribute new test script, filefrag-extent-compare
* tests/Makefile.am (EXTRA_DIST): Add filefrag-extent-compare.
Jim Meyering [Sun, 13 Jun 2010 14:19:29 +0000 (16:19 +0200)]
build: distribute new file, fiemap.h
* src/Makefile.am (noinst_HEADERS): Add fiemap.h.
Jie Liu [Fri, 11 Jun 2010 08:29:02 +0000 (16:29 +0800)]
copy.c: add FIEMAP_FLAG_SYNC to fiemap ioctl
* src/copy.c (fiemap_copy): Force kernel to sync the source
file before mapping.
Jim Meyering [Wed, 9 Jun 2010 06:42:30 +0000 (08:42 +0200)]
fiemap.h: include <stdint.h>, not <linux/types.h>
* src/fiemap.h: Include stdint.h, not linux/types.h,
now that this file uses only portable type names.
Paul Eggert [Wed, 9 Jun 2010 06:15:07 +0000 (08:15 +0200)]
copy.c: ensure proper alignment of fiemap buffer
* src/copy.c (fiemap_copy): Ensure that our fiemap buffer
is large enough and well-aligned.
Replace "0LL" with equivalent "0" as 3rd argument to lseek.
Jim Meyering [Sat, 5 Jun 2010 08:17:48 +0000 (10:17 +0200)]
copy.c: adjust comments, tweak semantics
* src/copy.c (fiemap_copy): Rename from fiemap_copy_ok.
Add/improve comments.
Remove local, "fail".
(fiemap_copy): Do not require caller to set
"normal_copy_required" before calling fiemap_copy.
Report ioctl failure if it's the 2nd or subsequent call.
Jim Meyering [Sat, 29 May 2010 19:22:40 +0000 (21:22 +0200)]
tests: relax the root-tests cross-check
* cfg.mk (sc_root_tests): Allow spaces before "require_root_",
now that tests/cp/sparse-fiemap has a conditional use.
Jim Meyering [Fri, 11 Jun 2010 12:34:03 +0000 (14:34 +0200)]
tests: accommodate varying filefrag -v "flags" output
* tests/cp/sparse-fiemap: Accommodate values other than "eof"
in the "flags" column of filefrag -v output
Jim Meyering [Fri, 21 May 2010 16:28:42 +0000 (18:28 +0200)]
tests: exercise more of the new FIEMAP copying code
* tests/cp/sparse-fiemap: Ensure that a file with many extents
(more than fit in copy.c's internal buffer) is copied properly.
Don't require root access if current partition is btrfs or xfs.
Use init.sh, not test-lib.sh.
* tests/filefrag-extent-compare: New file.
Jie Liu [Thu, 13 May 2010 14:17:53 +0000 (22:17 +0800)]
tests: add a new test for FIEMAP-copy
* tests/cp/sparse-fiemap: Add a new test for FIEMAP-copy against a
loopbacked ext4 partition.
* tests/Makefile.am (sparse-fiemap): Reference the new test.
Jie Liu [Thu, 13 May 2010 14:09:30 +0000 (22:09 +0800)]
cp: copy sparse files efficiently using the FIEMAP ioctl
* src/fiemap.h: Add fiemap.h for fiemap ioctl(2) support. Copied
from linux's include/linux/fiemap.h, with minor formatting changes.
* src/copy.c (copy_reg): Now, when `cp' is invoked with --sparse=[WHEN],
we will try to do FIEMAP-copy if the underlying file system
supports it, and fall back to a normal copy if it fails.
Pádraig Brady [Wed, 24 Nov 2010 08:37:23 +0000 (08:37 +0000)]
doc: add alternatives for field processing not supported by cut
* doc/coreutils.texi (cut invocation): Remove the tr -s '[:blank:]'
example, as it doesn't handle leading and trailing blanks. Add `awk`
examples for common field processing operations often asked about.
Also document a `join` hack, to achieve the same thing. Note the
join options are ordered so as to be compatible with other systems.
Pádraig Brady [Thu, 27 Jan 2011 07:17:16 +0000 (07:17 +0000)]
join: don't report disorder against an empty file
This allows one to use join as a field extractor like:
join -a1 -o 1.3,1.1 - /dev/null
* src/join.c (join): Don't flag unpairable lines when
one of the files is empty.
* tests/misc/join: Add a new test for empty input, and adjust
a previous test that was only checking against empty input.
* doc/coreutils.texi (join invocation): Document the change.
* NEWS: Likewise.
Pádraig Brady [Fri, 14 Jan 2011 08:46:21 +0000 (08:46 +0000)]
join: ensure --header skips the order check with empty files
* src/join.c: Skip the header even if one of the files is empty.
* tests/misc/join: Add a test case.
* NEWS: Mention the fix
Pádraig Brady [Wed, 5 Jan 2011 11:52:54 +0000 (11:52 +0000)]
join: add -o 'auto' to output a constant number of fields per line
Lines with a different number of fields than the first line,
will be truncated or padded.
* src/join.c (prfields): A new function refactored from prjoin(),
to output all but the join field.
(prjoin): Don't swap line1 and line2 when line1 is blank
so that the padding is applied to the right place.
(main): Handle the -o 'auto' option.
* tests/misc/join: Add 6 new cases to test the auto format.
* NEWS: Mention the change in behavior.
Suggestion from Assaf Gordon
Jim Meyering [Fri, 28 Jan 2011 18:23:21 +0000 (19:23 +0100)]
tests: remove obsolete uses of "$$" in temporary file names
Those were useful when tests might have been run in the same
directory and in parallel. Now, each test is run in a newly-
created empty directory.
* tests/cp/backup-1: Remove obsolete uses of "$$".
* tests/cp/same-file: Likewise.
* tests/dd/misc: Likewise.
* tests/mv/part-symlink: Likewise.
* tests/mv/to-symlink: Likewise.
* tests/touch/fail-diag: Likewise.
Jim Meyering [Tue, 25 Jan 2011 13:29:07 +0000 (14:29 +0100)]
tests: don't hide all trace of the vc_exe_in_TESTS test
There was a non-negligible delay after running a single test.
Now, you'll know why when you see this test's name.
* tests/check.mk (vc_exe_in_TESTS): Don't @-hide commands.
Use $(AM_V_GEN) instead.
Andreas Schwab [Tue, 25 Jan 2011 17:29:07 +0000 (18:29 +0100)]
tests: minor correction
* tests/du/move-dir-while-traversing: Ignoring SIGTSTP is enough;
don't also attempt to ignore SIGSTOP, it cannot be handled or ignored.
Spotted by Andreas Schwab.
Jim Meyering [Tue, 25 Jan 2011 11:36:22 +0000 (12:36 +0100)]
tests: avoid FP failure due to suspension
* tests/du/move-dir-while-traversing: Prohibit suspension,
to avoid false-positive failure.
Jim Meyering [Tue, 25 Jan 2011 10:09:27 +0000 (11:09 +0100)]
tests: avoid rare FP failure in new du test
* tests/du/move-dir-while-traversing: Create an even larger tree
to avoid a false-positive failure due to du terminating before
the rename is triggered.
Jim Meyering [Mon, 24 Jan 2011 08:38:40 +0000 (09:38 +0100)]
split: avoid a new, spurious warning from gcc-4.6.0
* src/split.c (lines_rr) [IF_LINT]: Initialize files, now that
rawhide's gcc-4.6.0 would otherwise warn about use-uninitialized.
Jim Meyering [Mon, 24 Jan 2011 08:37:10 +0000 (09:37 +0100)]
tail: avoid new diagnostic when applying -f to a pipe on linux-2.3.38
* src/tail.c (fremote): Do not print a diagnostic when
fstatfs (pipe_FD, &buf) fails, as it now does on linux-2.3.38.
This avoids the spurious failure of tests/misc/tail's f-pipe-1
test, when running in input-from-pipe mode.
Jim Meyering [Fri, 21 Jan 2011 13:02:27 +0000 (14:02 +0100)]
doc: fix wording in warning about potential conflict with built-in
* doc/coreutils.texi (mayConflictWithShellBuiltIn): Fix wording.
Paul Eggert [Fri, 21 Jan 2011 18:59:32 +0000 (10:59 -0800)]
manual: document floating point better
* doc/coreutils.texi (Floating point): New section.
(od invocation, tail invocation, sort invocation, printf invocation):
(sleep invocation, seq invocation): Refer and defer to it. See
<http://lists.gnu.org/archive/html/bug-coreutils/2011-01/msg00031.html>.
Jim Meyering [Thu, 20 Jan 2011 09:50:11 +0000 (10:50 +0100)]
build: update gnulib submodule to latest
The previous gnulib submodule reference was *still* to a
non-public commit. My submodule had a stray commit, so
the reference was always to a local merge commit.
Reported by Rob Vermaas.
Jim Meyering [Thu, 20 Jan 2011 09:00:42 +0000 (10:00 +0100)]
build: update gnulib submodule to latest
The previous gnulib submodule reference was to a non-public commit.
Reported by Rob Vermaas.