Bug-fix release 2.4 ~~~~~~~~~~~~~~~~~~~ 1. Add SSH URLs support. These URLs start with "ssh://" and have the following format: ssh://user:password@host:path, where * user - user name (optional) * password - the password (optional) * host - hostname * path - path to the image file on the remote host If the password was given in the URL, bmaptool will use password-based SSH authentication, otherwise key-based SSH authentication will be used. Bug-fix release 2.3 ~~~~~~~~~~~~~~~~~~~ 1. Add bmap file SHA1 checksum into the bmap file itself in order to improve robustness of bmaptool. Now we verify bmap file integrity before using it, and if it is corrupted or incomplete, we should be able to detect this. The reason for this change was a bug report from a user who somehow ended up with a corrupted bmap file and experienced weird issues. This also means that manual changes the bmap file will end up with a SHA1 mismatch failure. In order to prevent the failure, one has to update the bmap file's SHA1 by putting all ASCII "0" symbols (should be 40 zeroes) to the "BmapFileSHA1" tag, then generating SHA1 of the resulting file, and then put the calculated real SHA1 back to the "BmapFileSHA1" tag. In the future, if needed, we can create a "bmaptool checksum" command which could update SHA1s in the bmap file. 2. Re-structure the bmap file layout and put information about mapped blocks count at the beginning of the bmap XML file, not after the block map table. This will make it possible to optimize bmap file parsing in the future. This also makes the bmap file a little bit more human-readable. 2. Make the test-suite work on btrfs. Bug-fix release 2.2 ~~~~~~~~~~~~~~~~~~~ 1. Made bmaptool understand URLs which include user name and password (the format is: https://user:password@server.com) Bug-fix release 2.1 ~~~~~~~~~~~~~~~~~~~ 1. Fixed the out of memory problems when copying .bz2 files. 2. Added CentOS 6 support in packaging. Release 2.0 ~~~~~~~~~~~ There are several user-visible changes in 'bmaptool copy': 1. In order to copy an image without bmap, the user now has to explicitly specify the "--nobmap" option. In v1.0 this was not necessary. The reason for this change is that users forget to use --bmap and do not realize that they are copying entire the image. IOW, this is a usability improvement. 2. The bmap file auto-discovery feature has been added. Now when the user does not specify the bmap file using the --bmap option, 'bmaptool copy' will try to find it at the same place where the image resides. It will look for files with a similar base name and ".bmap" extension. This should make it easier to use bmaptool. 3. 'bmaptool copy' now can open remote files, so it is not necessary to download the images anymore, and you can specify the URL to bmaptool. For example: bmaptool copy download.tizen.org/snapshots/ivi/.../ivi-2.0.raw.bz2 The tool will automatically discover the bmap file, read from the image from the 'download.tizen.org' server, decompress it on-the-fly, and copy to the target file/device. The proxy is supported via the standard environment variables like 'http_proxy', 'https_proxy', 'no_proxy', etc. 4. Now 'bmaptool' prints the progress while copying. This improves usability as well: copying may take minutes, and it is nice to let the user know how much has already been copied. 5. Warnings and errors are high-lighted using yellow and red labels now. 6. Added bmaptool man page. 'bmaptool create' has no changes comparing to release v1.0. Release 1.0 ~~~~~~~~~~~ The first bmap-tools release. All the planned features are implemented, automated tests are implemented. We provide nice API modules for bmap creation ('BmapCreate.py') and copying with bmap ('BmapCopy.py'). The 'Fiemap.py' API module provides python API to the FIEMAP Linux ioctl. The 'bmaptool' command-line tool is a basically a small wrapper over the API modules. It implements the 'create' and 'copy' sub-commands, which allow creating bmap for a given file and copying a file to another file or to a block device using bmap. The 'bmaptools copy' command (and thus, 'BmapCopy.py' module) support accept compressed files and transparently de-compress them. The following compression types are supported: .bz2, .gz, .tar.bz2, .tar.gz. The original user of this project is Tizen IVI where the OS images are sparse 2.6GiB files which are distributed as .bz2 file. Since the images are only 40% full, the .bz2 file weights about 300MiB. Tizen IVI uses the 'BmapCreate.py' API module to generate the bmap file for the 2.6GiB images (before the image was compressed, because once it is compressed with bzip2, the information about holes gets lost). Then the bmap file is distributed together with the .bz2 image. And Tizen IVI users are able to flash the images to USB stick using the following command: $ bmaptool copy --bmap image.bmap image.bz2 /dev/usb_stick This command decompresses the image (image.bz2) on-the-fly, and copies all the mapped blocks (listed in 'image.bmap') to the USB stick (the '/dev/usb_stick' block device). This is a lot faster than the old method: $ bzcat image.bz2 | dd of=/dev/usb_stick Additionally, 'bmaptool copy' verifies the image - the bmap stores SHA1 checksums for all mapped regions. However, bmap-tools may be useful for other projects as well - it is generic and just implements the idea of fast block-based flashing (as opposed to file-based flashing). Block-based flashing has a lot of benefits. The 'BmapCopy.py' module implements a couple of important optimization when copying to block device: 1. Switch the block device I/O scheduler to 'Noop', which is a lot faster than 'CFQ' for sequential writes. 2. Limits the amount of memory which the kernel uses for buffering, in order to have less impact on the overall system performance. 3. Reads in a separate thread, which is a lot faster when copying compressed images, because we read/uncompress/verify SHA1 in parallel to writing to a potentially slow block device. We support bmap format versioning. The current format is 1.2. The minor version number must not break backward compatibility, while the major numbers indicates some incompatibility.