Artem Bityutskiy [Mon, 26 Nov 2012 11:39:05 +0000 (13:39 +0200)]
test_api_base: re-structure the test some more
Kill the 'setUp()' and 'tearDown()' methods and open/close the temporary files
in '_do_test()' instead. This works better for me because I will call
'_do_test()' many times and it is simpler when it starts with new temporary
files every time, rather than teach it cleaning-up the old ones.
Change-Id: I135b8423d1e7d95a6c2de8d8bc0d3f8597ca5418
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 11:26:06 +0000 (13:26 +0200)]
BmapCopy: validate image size
Add some validation of image size - when we have the bmap, bmake sure that real
image size is the same as image size from the bmap.
Change-Id: I8503e0a14d7eb1df071780058b71b7ef6be225a4
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 11:01:47 +0000 (13:01 +0200)]
BmapCopy: bugfix: handle unaligned files properly
When the image size is not aligned to the block size, we made an aligned copy
anyway, which is wrong, because the copy has to have the same length as the
image. This patch fixes the issue.
Change-Id: I012496e315b47c1381715fdd1f8921d347213b29
Artem Bityutskiy [Mon, 26 Nov 2012 11:02:59 +0000 (13:02 +0200)]
TODO: add a reminder about testing compressed files.
Change-Id: I56506ee85f11c2fe03828b06f538313533971759
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 10:51:24 +0000 (12:51 +0200)]
BmapCopy: simplify the code a bit
Stop returning 'length' in '_get_data()', because it is redundant and identical
to 'end - start + 1'.
Also add another assertion to improve robustness.
Change-Id: I693d40882fd1f47709db2687446be284de35c01f
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 10:15:29 +0000 (12:15 +0200)]
test_api_base: re-structure the test a bit
I want to run the test for various different files, not just for one file.
Re-structure the test to allow this:
1. Rename the old 'test' function to '_do_test()'
2. Make the '_f_image' attribute to be an argument of '_do_test()'. So
'self._f_image' is renamed to 'f_image'.
3. Introduce a new 'test()' function which creates the file file to test
and invokes '_do_test()'. Later this function will run '_do_test()' for
more than one file.
Change-Id: I252f396e45a780b1e20b024e5053d04c3ab11b0e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 10:13:36 +0000 (12:13 +0200)]
test_helpers: generate file of correct length
The 'create_random_sparse_file()' alwasy generates files of the length aligned
to block size, even when the requested size is unaligned. Fix this by adding a
truncation at the end.
Change-Id: Id3b7f8f0e6c1d9f11aa68d06d4883b8227991e50
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 08:48:13 +0000 (10:48 +0200)]
test_api_base: saner temprorary file creation
Instead of using the low-level mkstemp - use the NamedTemporaryFile function
instead. It removes the temporary files on close.
Change-Id: I8b8b0ee9db36d626d4d7d0a00c79adf2af186ddc
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 26 Nov 2012 07:52:57 +0000 (09:52 +0200)]
tests: create a separate module for helper functions
I am going to add a test for the Fiemap module, and it will need the
'create_random_sparse_file()' function from the base API test. Therefor, move
this function to a separate 'test_helpers.py' module which will be shared
across tests.
Additionally, remove the holes verification from 'create_random_sparse_file()'
- the test which I will add soon verifies holes separately.
Also change the egging to exclude the tests.
Change-Id: I2c81251c5e1884a4f644e89b417c8a18050aeada
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Sat, 24 Nov 2012 16:29:14 +0000 (18:29 +0200)]
Fiemap: introduce a separate helper for invoking FIEMAP
Introduce a '_invoke_fiemap' helper function which hides all the cruft related
to the FIEMAP ioctl invocation.
Change-Id: I8a45591f03f1d77ac24b661af96cd4cbe583216f
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Sat, 24 Nov 2012 15:27:25 +0000 (17:27 +0200)]
Fiemap: introduce 'buf_size' parameter for __init__
We are going to use full power of FIEMAP. It requires an array where it will
place the block map on output. Generally, big arrays are good because fiemap
will be called less times, which is good for performance.
Make the default to be a 1MiB buffer, and allocate it in the constructur
for further use.
Change-Id: I4e271794bf747cc7d611d86a6a76afd554c8e09f
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Sat, 24 Nov 2012 15:38:16 +0000 (17:38 +0200)]
Fiemap: more input arguments validation
Validate the input argument of 'block_is_mapped()'.
Change-Id: I31ec0e4b5ff6250ed80cf4b2ea1a095ecf517927
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Sat, 24 Nov 2012 15:34:18 +0000 (17:34 +0200)]
Fiemap: validate the input arguments
Validate the input arguments for 'get_mapped_ranges()' and
'get_unmapped_ranges()' - they should be positive and should not go beyond file
size.
Change-Id: Ib992e03c6d651bd4e00025bc1a75b69a90d95a31
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Sat, 24 Nov 2012 15:17:10 +0000 (17:17 +0200)]
Fiemap: fix error message
Fix a typo in one of the error messages: s/FIBMAP/FIEMAP/
Change-Id: Iec26e20aa014eb1b28c710698dd4a69e2133649d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Sat, 24 Nov 2012 15:15:19 +0000 (17:15 +0200)]
Fiemap: introduce constants
Introduce constants for various FIEMAP-related stuff like format strings for C
structures, their sizes, and the ioctl number. First of all, this is just
nicer. But more importantly, we'll soon introduce another function which will
use these constants as well.
Change-Id: Iac00bc93d8a590dc242b8ce7b20f2fe3d98a186d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 12:29:53 +0000 (14:29 +0200)]
BmapCopy.py: remove few useless initializations
We initialize these variables right after setting them to None, which is
probably an overkill, remove the None assignment.
Change-Id: Ibf419b2d138ab025e0aa60f1eb3ac6a477a92734
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 12:27:12 +0000 (14:27 +0200)]
BmapCopy: disable a couple of pylint recommendations
Change-Id: Ibae22f9aad139904f21fa0f63c5bdde78ba05a38
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 12:26:12 +0000 (14:26 +0200)]
BmapCopy: use "_" prefix for private variables
Change-Id: I91f3875aa2f12ae485f6b757ffe5f721aa3dc87d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 12:06:50 +0000 (14:06 +0200)]
test_api_base: improve the test
Not only compare that the destination file has the same contens as the source
file, but also make sure they have identical holes. This improves the test.
Change-Id: Idb53ed7c74eda2cd72608a845687c509c5adaa7e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 11:56:16 +0000 (13:56 +0200)]
Fiemap: improve the API
Improve the API and allow to specify the area of the file to generate ranges
for, so that it would be possible to generate ranges only for part of the file.
Change-Id: I1e8455be739d9835bc3f750709fbe7d8e9374ee7
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 11:26:51 +0000 (13:26 +0200)]
test_api_base: fix the test
The test apparently was half-broken because:
1. It did not check results of 'filecmp'.
2. It did not seek the bmap file descriptor so the bmap comparison did not
work.
Change-Id: I7d6e65acf7ee3c2524064a71faccdb6554f4ffe7
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 11:18:38 +0000 (13:18 +0200)]
BmapCopy: bugfix: make destination file to be of the same size
When we copy to a regular file using bmap, and the image has holes at the end,
the resulting copy does not contain these holes and its size is shorter.
Fix this bug by truncating the destination file to the image size at the end.
Change-Id: Ie9586a1972e8b1027f76e1973fe6e10fe939f035
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 10:24:55 +0000 (12:24 +0200)]
test_api_base: add a Fiemap module check
This patch improves tesst coverage by checking the sparse file. Namely,
when we create the random sparse file, we remember where the holes are. Then
we make sure that Fiemap reports absolutely the same holes.
Change-Id: I2cf97bcf3de456b09b4138cec9be3255c88ee22e
Artem Bityutskiy [Fri, 23 Nov 2012 08:32:27 +0000 (10:32 +0200)]
BmapCreate: separate out the FIEMAP functionality
Create a separate class for the FIEMAP ioctl API. I do this because
I am going to use full power of FIEMAP and the code will become a lot
more complex, so it is nicer to have it separate.
Besides, we need a stand-alone FIEMAP API for testing.
Change-Id: Ibc4f18f4f143e6bf878730a9777e2ef8dd3cdede
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 07:57:02 +0000 (09:57 +0200)]
Bmap API: remove useless 'bmap' prefix
The 'BmapCopy' and 'BmapCreate' modules provide a set of useful attributes like
block size, blocks count, mapped blocks count, etc. All these attributes start
with a 'bmap_' prefix, which is useless and makes the code less readable.
Remove the 'bmap_' prefix.
Change-Id: I9ec859c1bc26286c7328e0a7cf26c91c47fd43d9
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 07:25:17 +0000 (09:25 +0200)]
BmapCreate: handle flush() exception
We handle file.flush() exceptions for the image, but not for bmap. This is
inconsistent. Let's handle them for both.
Change-Id: Ie4e9e28b865d2dcb9da278d577443f3665a4a907
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 07:19:27 +0000 (09:19 +0200)]
BmapCreate: simplify the exception class
Once we removed the FIBMAP support, we do not need the 'errno' attribute in the
exception class - no one uses it. Let's kill it to have the code simpler and
more clear, as well as less error-prone.
Change-Id: I97d20cf1b6550359133fff5c81136b72bc77ec2f
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 23 Nov 2012 07:15:14 +0000 (09:15 +0200)]
BmapCreate.py: simplify the module by killing FIBMAP support
We do not need to support FIBMAP because FIEMAP is a lot better and does not
require root privileges. FIEMAP is supported since 2008's kernel version
2.6.28.
Thus, kill the FIBMAP ioctl support and remove a lot of cruft.
Change-Id: Iac0c8653291c937cb4ed3e20eef88dc3fc4647d6
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 14 Nov 2012 10:38:08 +0000 (12:38 +0200)]
Pre-release version 0.6
Also change the date in the debian changelog for the 0.5 release - I forgot
to update it.
Change-Id: Ie69aba677b2812b7405327c32142583cad610856
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 21 Nov 2012 14:38:21 +0000 (16:38 +0200)]
BmapCopy.py: fix a bug when copying without bmap
Fix a regression introduced in version 0.5: when copying without bmap the
script fails with the following error:
AttributeError: BmapCopy instance has no attribute '_f_bmap'
This is because we never define this attribute when there is no bmap, but still
use it. The fix is:
1. Define all attributes in the __init__ constructor.
2. Always check if _f_bmap is not None before using it.
While on it, do a minor space clean-up.
Change-Id: I3f729588d33be8e2a447fca817b994500840503a
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 21 Nov 2012 14:36:32 +0000 (16:36 +0200)]
tests: test_api_base: test copying without bmap as well
Improve the test a little by:
1. Calling the 'copy' function with the first 'sync' argument being 'True' as
well
2. Adding another pass which tests file copying without bmap.
This discovered a bug, actually, which will be fixed in the next commit.
Change-Id: I34441b1151588c31bb202d2cdfdeb1e874f4e0bb
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 14 Nov 2012 10:38:08 +0000 (12:38 +0200)]
Pre-release version 0.5
Change-Id: I304827a5204fd3aa4c807f236ead82d1113a3e3d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 07:36:57 +0000 (09:36 +0200)]
tests: add the base functionality test
Change-Id: I035f71eadda2d2da83a1c8776f035cc1d5ccde12
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 13:54:15 +0000 (15:54 +0200)]
BmapCreate: fix and silence pylint warnings
Fix several pylint warnings and silence a couple.
Change-Id: If714828815ac72c026e05e022fb03f6ffed2fa6c
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 13:46:08 +0000 (15:46 +0200)]
BmapCopy.py: fix and silence pylint warnings
Fix a couple of pylint warnings and silence one.
Change-Id: I392e95b6bf0c3805e3c07cb8d80e0d0aef1b7311
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 13:07:44 +0000 (15:07 +0200)]
BmapCopy.py: flush the destination file on exit
Always flush the destination file on exit to make sure that if the user
opens the same file by path - he/she sees all the data.
Change-Id: I493d0c92d4818e289216925e00111128cba88db6
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 13:06:21 +0000 (15:06 +0200)]
BmapCopy.py: open the destination file for writing
Open the destination file in write-only mode, because we only write to
this file, and never read from.
Change-Id: I5bdef84ecdb7faf38190a00bb34fb1b183fb9430
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 13:04:53 +0000 (15:04 +0200)]
bmaptool: open the destination file in binary mode
This is just a small correction - the destination file is a binary file,
so open it in binary mode.
Change-Id: I38ca25dabc38dabf65a18b6f50d8de066c8f18b1
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 11:52:08 +0000 (13:52 +0200)]
BmapCreate.py: support file-like objects
It is very useful give BmapCreate file-like objects instead of file paths, and
this is exactly what this patch implements.
Change-Id: I6d342ea64584b8b203f62a0df33e2a6df5eb9da2
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 11:19:33 +0000 (13:19 +0200)]
BmapCreate.py: stop using logger for the bmap output
It is an overkill to use a logger object to output the bmap, and it is also
rather difficult for the users. Instead, make 'BmapCreate' accept a file-like
object for the output, not a logger object.
Change-Id: I7cf83dc8e0c43f762449e7fcf200b1f8b66359c5
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 10:42:01 +0000 (12:42 +0200)]
BmapCopy.py: support file-like objects
It is very useful give BmapCopy file-like objects instead of file paths, and
this is exactly what this patch implements.
Change-Id: I41b40e8369743bf53e018f503b65719a1c823464
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 09:28:43 +0000 (11:28 +0200)]
TODO: add an entry about interruptions
Change-Id: Icd9f43eb8b9b9c48624cfaf4553b9fa2acf5f509
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 09:37:08 +0000 (11:37 +0200)]
BmapCreate: remove useless seek
We do not need to seek the image to the beginning because we do not care
about the file position - we do not read of write the file.
Change-Id: Ib5e9abd8560e77b1a095d1882594de15e97be4e7
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 20 Nov 2012 09:11:24 +0000 (11:11 +0200)]
BmapCopy.py: fix incorrect assignment
This patch fixes the following bug:
UnboundLocalError: local variable 'last' referenced before assignment
which happens when the range consists only of one block.
Change-Id: I4d91d2abdd45408b6d2a6131e3ddd7fbbe3492f9
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 19 Nov 2012 15:02:40 +0000 (17:02 +0200)]
BmapHelpers.py: introduce a 'get_block_size()' function
Move the piece of code which finds out block size from 'BmapCreate.py'
to 'BmapHelpers.py' - we need this code in tests as well.
Change-Id: I1682a0cffc0348132409bdf085c9fb4fce5b4473
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 16 Nov 2012 11:31:44 +0000 (13:31 +0200)]
BmapCopy.py: save and restore block device settings
We change block device settings to improve I/O speed (e.g., switch to the
'noop' I/O scheduler). This patch also teaches the BmapCopy module to restore
the settings when the copying is done.
Change-Id: I31caddf5d86510452a7b58be3268ed973d057354
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 16 Nov 2012 10:31:56 +0000 (12:31 +0200)]
BmapCopy.py: make blkdev optimization work for partitions as wall
If we are writing to something like /dev/sdc1 instead of /dev/sdc, we should
go one level up in the sysfs hierarchy to access the block device configuration
files like 'scheduler'.
Change-Id: I845a145197aa15513a288214f0b58c7a1da3b8cf
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 14 Nov 2012 10:38:08 +0000 (12:38 +0200)]
Pre-release version 0.4
Change-Id: I50a735e3dedcdfd3035d58adb7edcc355994689a
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 15 Nov 2012 09:28:10 +0000 (11:28 +0200)]
BmapCopy.py: write without bmap the same way as with bmap
Unify the code paths for the situations when we have bmap and when we do not
have it.
Change-Id: I7a8a47cf0a475077953590ea9ebbd4b905acb4ed
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 15 Nov 2012 09:00:34 +0000 (11:00 +0200)]
BmapCreate: generate more readable bmap
When the blocks rage consists of a single block (say 18282), write it
in form of <>18282</> instead of less readable <>18282-18282</> form.
'bmaptool copy' supports both.
Change-Id: I96cd97b1913227de878004fc29eb65b76f11058e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 15 Nov 2012 08:51:54 +0000 (10:51 +0200)]
BmapCopy.py: fix another pylint complaint
Change-Id: I526140d32b362cb7669f1ce10dbae793b8937229
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 15 Nov 2012 08:48:56 +0000 (10:48 +0200)]
BmapCreate.py: remove too long lines
Pylint complains a lot about long lines, which is fixed by this patch.
Change-Id: Ib615a44d7529ce35717cd00ac2714b6e36301e68
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 14 Nov 2012 09:59:38 +0000 (11:59 +0200)]
BmapCopy.py: handle exceptions from the reader thread
Use the batch queue to pass exceptions from the reader thread to the main
thread in case of errors, and raise the exceptions from the reader thread.
Change-Id: I02801278b76bb1b390b6e36372ee0d5b487e79c2
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 13 Nov 2012 13:43:50 +0000 (15:43 +0200)]
BmapCopy: implement threaded reader
Read the data from a separate thread. This does not change the performance
measurably in case of uncompressed images, but does improve writing speed in
case of compressed images - Tizen.bz2 image flashing drops from 2m13s to
1m43s.
Change-Id: I3300a569042d16357fcc0920c3f03f3fff08d29d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 13 Nov 2012 12:52:26 +0000 (14:52 +0200)]
BmapCopy.py: implement an iterator for reading the image
Implement a '_get_batches()' iterator which reads batches of data from
the image file.
Change-Id: If914efc60ea6a259f8430ded8c2d4143b07adc3c
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 13 Nov 2012 11:57:56 +0000 (13:57 +0200)]
BmapCopy.py: introduce a helper iterator
The '_copy_data()' function is a bit ugly and large. Simplify it by introducing
an iterator which splits the entire blocks range on smaller batch ranges, and
we do actual I/O in these small batches.
Change-Id: I6c6e2cb23672a6abbdb679776c82046b5706b5c8
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 13 Nov 2012 11:37:03 +0000 (13:37 +0200)]
BmapCopy.py: move chunk_size to the class level
We use 'chunk_size' variable in several places and hard-code it to various
values. Clean this up by introducing class-level variables for size of the
I/O operations we do. Call them 'batch_blocks' and 'batch_bytes' for the
I/O batch size in blocks and bytes.
Change-Id: I0ec6058a20e89d685e917c13268c2f70f385c053
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 13 Nov 2012 10:18:12 +0000 (12:18 +0200)]
BmapCopy.py: introduce a iterator for blocks ranges
Add a 'get_block_ranges()' iterator function which parses the bmap
file and returns block ranges. This makes the code a bit nicer and
modular.
Change-Id: I09270f4bc3f74b676fcf3b3f6e87c57e222c709a
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 14 Nov 2012 13:25:05 +0000 (15:25 +0200)]
bmaptool: fix a print
We write to regular files too, not only to block devices, fix the
message correspondingly.
Change-Id: I169fc51bfd2833a01749fd30d936da4e6bd8b98e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 14 Nov 2012 13:23:43 +0000 (15:23 +0200)]
Fix a number of pylint warnings
Change-Id: I18007ca6c50e3a733d5ee9e944636bfdaab97f45
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Tue, 13 Nov 2012 09:00:05 +0000 (11:00 +0200)]
bmaptool: pre-release version 0.3
Change-Id: I1dacb92a73445600497a608917367ba4a2aff295
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 13:39:48 +0000 (15:39 +0200)]
BmapCopy.py: synchronize periodically
Do not let the kernel cache too much data and run fsync() periodically. This
should impreve the Ctrl-C handling and terminate the program faster. So this
is mostly for the sake of user-friendliness, although it hurts write speed
a little bit.
Based on the idea of Patrick Ohly <patrick.ohly@intel.com>.
Change-Id: I0fa8358455623b0fc709b2a845b4753a1e25a9be
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 12:32:33 +0000 (14:32 +0200)]
BmapCopy: move a local variable to the class level
Move the 'blocks_written' local variable from the 'copy()' method to the
class lever. We'll need it in one of the next patches in other methods to find
out how much blocks has already been written.
Change-Id: I28220b85af44a39bd3643fb6d62a14804f56f9d3
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 13:41:49 +0000 (15:41 +0200)]
bmaptool: remove stale attribute reference
We do not have the 'writer.target_is_block_device' attribute anymore, remove
it.
Change-Id: Id055402c5195dc10f9ced04e276610a7008fea6e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 12:19:24 +0000 (14:19 +0200)]
Add the TODO file to store the TODO list
Change-Id: I33e82a3882a1fc382f93e161348ab908f9ff0565
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 12:12:46 +0000 (14:12 +0200)]
bmaptool: fix flashing speed calculation
We write 'mapped_size' amount of data, not the entire image, so use the correct
variable when calculating the flashing speed.
Change-Id: I68a7eb597c3e29e4c4795ca907d13cdb466e5363
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 10:29:43 +0000 (12:29 +0200)]
Debian packaging: add python 2.7 dependency
Change-Id: I3a56337c0b9006e9a86a8118b62bd96c9a664e92
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Mon, 12 Nov 2012 07:25:50 +0000 (09:25 +0200)]
BmapCopy.py: fix exeptions handling in bdev optimization
The 'open()' function throws IOError exceptions, not OSError. Fix our
exception handling.
Change-Id: I13761072efa62cb96fbf01164691e114f68074de
Reported-by: Patrick Ohly <patrick.ohly@intel.com>
Reported-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 9 Nov 2012 16:36:37 +0000 (18:36 +0200)]
BmapCreate.py: remove junk white-space
We do not need a white-space after 'sha1' attribute.
Change-Id: I4a66026d4e77309bfe2fdd092b06afa752a89b1e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 9 Nov 2012 16:35:33 +0000 (18:35 +0200)]
BmapCreate.py: fix a brown-paperbag bug
We did not seek the image file, so BmapCreate calculated SHA1 incorrectly.
Fix this.
Change-Id: Ie99cf26ac3af4a319832dc79bc82ba89b7b31280
Reported-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Fri, 9 Nov 2012 10:22:31 +0000 (12:22 +0200)]
Pre-release version 0.2
Change-Id: Iff5a35b4a0bd9781ae160249a669b0cd7d32b53e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 16:47:36 +0000 (18:47 +0200)]
BmapCopy: renames to "copy"
Rename the write() method to copy() because it matches the class name and the
idea. Similarly, rename several variables and change 'write' to 'copy' in few
comments.
Change-Id: I441eb3cdc9fc94210ed0d9e91d3fa1689edcc0ef
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 16:43:03 +0000 (18:43 +0200)]
BmapCopy: amend comments
After all the rework - go through all the commentaries and docstrings and amend
them to match the current state of art.
Change-Id: I3e0de168a697795c498045414d294979f35d7efa
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 16:08:43 +0000 (18:08 +0200)]
BmapCopy: move block device tuning to BmapBdevCopy
This seems to be the final piece of code which has to be moved. Now
BmapCopy is really independent of the destination file type.
Change-Id: I2a74ccea7db5984ee3940bd95a1ebfd21b6bdedb
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 15:59:41 +0000 (17:59 +0200)]
BmapCopy: move block device capacity check to BmapBdevCopy
Change-Id: Ia09788918ce5ad7a2fd27d8cd4d0c2965d4821c4
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 15:44:29 +0000 (17:44 +0200)]
BmapBdevCopy: move block device opening
Start moving the block device - specific stuff to the BmapBdevCopy class.
Move the open funcion first.
Change-Id: I06afd6554ec51b11f9c148a3da76dab5a9fd80f0
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 15:37:40 +0000 (17:37 +0200)]
BmapCopy: rename 'bdev_path' to 'dest_path'
Similarly to the previous commits - rename the path attribute to 'dest_path',
because the destination does not have to be a block device.
Change-Id: I769abc160f52574aa466a71965be401d91ed243e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 15:33:39 +0000 (17:33 +0200)]
BmapCopy: rename 'f_bdev' to 'f_dest'
Just like we did in bmaptools - our destination file may be anything,
not only block device, so use 'f_dest' for the name.
Change-Id: Ia2a863238ceefbff483e025940e931b62e06e3a8
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 15:26:04 +0000 (17:26 +0200)]
bmaptool: distinguish between block devices and regular file
Teach bmaptool to distinguish between block devices and regular files and
use the specialized version of 'BmapCopy' in case of block devices. The
specialized version will be implemented in the 'BmapBdevCopy' class, which
is just a copy of the base class so far.
Change-Id: I444f121893dfd5d637a08300191c7a5dbc082d4e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 15:13:55 +0000 (17:13 +0200)]
bmaptool: rename the bdev argument to dest
Sinc the 'bmaptool copy' command is not only for block device, naming
the destination file 'bdev' is a bad idea. Let's name it 'dest' instead,
which stands for 'destination' and has the same length as 'bdev', so that
renaming is simple.
Change-Id: I7c0382446dccd569aa26a69c0e06663cd139e3ae
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 14:52:07 +0000 (16:52 +0200)]
Rename BmapFlash to BmapCopy
Rename the module file name and the class name. There is a lot more to rename
and this is just the first step.
Change-Id: Ibee4082c783c5417401d10a41ef7787607108283
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 14:46:25 +0000 (16:46 +0200)]
BmapCreate: amend comments
Just refresh the comments and change the identation style in docstrigs to
something I like more nowadays.
Change-Id: I430a4f8eb71c6b739b5714d84e19d723412e761e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 14:06:15 +0000 (16:06 +0200)]
Rename bmap to bmaptool
The word 'bmap' is already reserved for the bmap file, it is bad idea to use
it for the tool. Let's call the tool 'bmaptool' instead.
Change-Id: I0c6fb70bdf3bbdcba11985c0b646f9e5a538c872
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 14:00:28 +0000 (16:00 +0200)]
BmapFlash: fix a bunch of pylint complaints
Change-Id: Ica510861824f16af29382adda47a8a2ba5675058
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 13:56:45 +0000 (15:56 +0200)]
BmapCreate: fix function name
Function name was mis-spelled, fix it.
Change-Id: Ib3c58367a4fd757fbee4753925b69d9dc6c38a5e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 13:52:18 +0000 (15:52 +0200)]
bmap: remove useless warning
No need to print a warning when copying to a regualr file, it makes little
sense and useless.
Change-Id: I77ccfe2b0a20e2ad90f97a19fdde1d8a06bd8026
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 13:49:02 +0000 (15:49 +0200)]
bmap: rename flash to copy
Rename the 'flash' command to 'copy' command. Indeed, we just copy data from
the image to a destination file, which may be a block device or something else.
So let's use better naming.
Change-Id: I135668f3aa8573b9c3ce7dd571751936d47a6552
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 13:39:30 +0000 (15:39 +0200)]
bmap: amend comments
This patch changes commentaries and few messages we print.
I have a lot of changes planned, and the biggest is that I want to rename
the 'flash' command to 'copy' command. This commit is a preparation for
this change.
Change-Id: Ifc3d602d953592fba6a2892629516a08b067ec9f
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 13:41:20 +0000 (15:41 +0200)]
bmap: correct Ctrl-C message
The long synching we warn the user about happens only for block devices
and does not happen for regualr files. So warn only if we are writing to
a block device. Also, correct the warning spelling a bit.
Change-Id: I0b1fc3362da371b150b2ea09a9b07fe83e8b149e
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 11:47:55 +0000 (13:47 +0200)]
BmapFlash: spelling fixes
Change-Id: I95c3097563677348e3149b61aceebef58cd8f7cc
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 11:45:23 +0000 (13:45 +0200)]
BmapCreate: spelling fixes
Change-Id: Iad4f07f7d15bcebf7620d8fa2380bec03f911025
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 11:42:34 +0000 (13:42 +0200)]
bmap: a couple of spelling fixes
Change-Id: I57ccc904fdfdebc1342fc5a401ddb73895cbfd4d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 11:33:41 +0000 (13:33 +0200)]
Fix pylint warning
Too long line...
Change-Id: I6c0d0b2aee94e7996db5dcac6a5513da7f47e067
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 11:33:41 +0000 (13:33 +0200)]
Implement sub-commands support
Remove separate 'bmap-creator' and 'bmap-flasher' tools and instead, implement
a single 'bmap' tool with 'create' and 'flash' subcommands.
Change-Id: I6c0d0b2aee94e7996db5dcac6a5513da7f47e067
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 10:16:02 +0000 (12:16 +0200)]
Rename API classes
Change API class names from BmapCreator/BmapFlasher to BmapCreate/BmapFlash.
I plan to get rid of 'bmap-creator' and 'bmap-flasher' and instead, have
a single 'bmap' tool with sub-commands: bmap create and bmap flash.
Change-Id: I5b6fbb0b75954fc27ce39ad5e4317f4957461595
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Thu, 8 Nov 2012 10:13:23 +0000 (12:13 +0200)]
Rename API modules
I am planning to switch to 'bmap [flash|create]' sub-commands, and this
patch is a preparation which renames API modules to match the new scheme.
Change-Id: I3ae8d726e91d8605a01d044f86a0d89b5458346d
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 7 Nov 2012 09:39:53 +0000 (11:39 +0200)]
Release version 0.1.1
Change-Id: I86c186f8a8ee99cde051fd626c9e8854ae90a9cf
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 7 Nov 2012 09:26:49 +0000 (11:26 +0200)]
packaging: correct dependency
Although xml.etree is part of python, Fedora and OpenSuse provide this
module in separate packages. Add the corresponding packaging requirements.
Change-Id: I705fcb091eb7f977f2d564fe0d86ede1bad295f3
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 7 Nov 2012 09:09:17 +0000 (11:09 +0200)]
bmap-flasher: act on KeyboardInterrupt
When one pressess Ctrl-C, the program hands for several minutes and does
not exit. The reason is that the kernel synchronized the block device on
the last close.
Let's at least print a message to the user and tell that there is no need
to worry.
Change-Id: I61368dddf697161c2a47012c5f9e2c96b74ef273
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>
Artem Bityutskiy [Wed, 7 Nov 2012 08:52:34 +0000 (10:52 +0200)]
bmap-flasher: kill the --no-sync option
The kernel sychronizes block devices on close automatically (in case of
the last reference becomes 0), so --no-sync does not work anyway.
Change-Id: Ib4783b85363a74b8d92a09c9a1a7636999569f4c
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@intel.com>