README revision 263286
11590Srgrimes 21590SrgrimesXZ Utils 31590Srgrimes======== 41590Srgrimes 51590Srgrimes 0. Overview 61590Srgrimes 1. Documentation 71590Srgrimes 1.1. Overall documentation 81590Srgrimes 1.2. Documentation for command-line tools 91590Srgrimes 1.3. Documentation for liblzma 101590Srgrimes 2. Version numbering 111590Srgrimes 3. Reporting bugs 121590Srgrimes 4. Translating the xz tool 131590Srgrimes 5. Other implementations of the .xz format 141590Srgrimes 6. Contact information 151590Srgrimes 161590Srgrimes 171590Srgrimes0. Overview 181590Srgrimes----------- 191590Srgrimes 201590Srgrimes XZ Utils provide a general-purpose data-compression library plus 211590Srgrimes command-line tools. The native file format is the .xz format, but 221590Srgrimes also the legacy .lzma format is supported. The .xz format supports 231590Srgrimes multiple compression algorithms, which are called "filters" in the 241590Srgrimes context of XZ Utils. The primary filter is currently LZMA2. With 251590Srgrimes typical files, XZ Utils create about 30 % smaller files than gzip. 261590Srgrimes 271590Srgrimes To ease adapting support for the .xz format into existing applications 281590Srgrimes and scripts, the API of liblzma is somewhat similar to the API of the 291590Srgrimes popular zlib library. For the same reason, the command-line tool xz 301590Srgrimes has a command-line syntax similar to that of gzip. 311590Srgrimes 321590Srgrimes When aiming for the highest compression ratio, the LZMA2 encoder uses 331590Srgrimes a lot of CPU time and may use, depending on the settings, even 341590Srgrimes hundreds of megabytes of RAM. However, in fast modes, the LZMA2 encoder 3527919Scharnier competes with bzip2 in compression speed, RAM usage, and compression 361590Srgrimes ratio. 371590Srgrimes 381590Srgrimes LZMA2 is reasonably fast to decompress. It is a little slower than 391590Srgrimes gzip, but a lot faster than bzip2. Being fast to decompress means 401590Srgrimes that the .xz format is especially nice when the same file will be 4129922Smarkm decompressed very many times (usually on different computers), which 4227919Scharnier is the case e.g. when distributing software packages. In such 4350477Speter situations, it's not too bad if the compression takes some time, 441590Srgrimes since that needs to be done only once to benefit many people. 451590Srgrimes 461590Srgrimes With some file types, combining (or "chaining") LZMA2 with an 471590Srgrimes additional filter can improve the compression ratio. A filter chain may 481590Srgrimes contain up to four filters, although usually only one or two are used. 491590Srgrimes For example, putting a BCJ (Branch/Call/Jump) filter before LZMA2 501590Srgrimes in the filter chain can improve compression ratio of executable files. 511590Srgrimes 521590Srgrimes Since the .xz format allows adding new filter IDs, it is possible that 531590Srgrimes some day there will be a filter that is, for example, much faster to 541590Srgrimes compress than LZMA2 (but probably with worse compression ratio). 551590Srgrimes Similarly, it is possible that some day there is a filter that will 561590Srgrimes compress better than LZMA2. 571590Srgrimes 588232Sdg XZ Utils doesn't support multithreaded compression or decompression 591590Srgrimes yet. It has been planned though and taken into account when designing 6027919Scharnier the .xz file format. 611590Srgrimes 621590Srgrimes 6340103Smarkm1. Documentation 641590Srgrimes---------------- 651590Srgrimes 661590Srgrimes1.1. Overall documentation 671590Srgrimes 681590Srgrimes README This file 691590Srgrimes 701590Srgrimes INSTALL.generic Generic install instructions for those not familiar 711590Srgrimes with packages using GNU Autotools 721590Srgrimes INSTALL Installation instructions specific to XZ Utils 7329922Smarkm PACKAGERS Information to packagers of XZ Utils 741590Srgrimes 751590Srgrimes COPYING XZ Utils copyright and license information 7614024Smarkm COPYING.GPLv2 GNU General Public License version 2 7729922Smarkm COPYING.GPLv3 GNU General Public License version 3 781590Srgrimes COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1 7934897Smarkm 801590Srgrimes AUTHORS The main authors of XZ Utils 811590Srgrimes THANKS Incomplete list of people who have helped making 821590Srgrimes this software 831590Srgrimes NEWS User-visible changes between XZ Utils releases 841590Srgrimes ChangeLog Detailed list of changes (commit log) 851590Srgrimes TODO Known bugs and some sort of to-do list 861590Srgrimes 871590Srgrimes Note that only some of the above files are included in binary 881590Srgrimes packages. 891590Srgrimes 901590Srgrimes 911590Srgrimes1.2. Documentation for command-line tools 921590Srgrimes 931590Srgrimes The command-line tools are documented as man pages. In source code 941590Srgrimes releases (and possibly also in some binary packages), the man pages 951590Srgrimes are also provided in plain text (ASCII only) and PDF formats in the 961590Srgrimes directory "doc/man" to make the man pages more accessible to those 971590Srgrimes whose operating system doesn't provide an easy way to view man pages. 9857232Sshin 991590Srgrimes 1001590Srgrimes1.3. Documentation for liblzma 1011590Srgrimes 1021590Srgrimes The liblzma API headers include short docs about each function 1031590Srgrimes and data type as Doxygen tags. These docs should be quite OK as 1041590Srgrimes a quick reference. 1059881Sache 10617284Spst I have planned to write a bunch of very well documented example 1071590Srgrimes programs, which (due to comments) should work as a tutorial to 1081590Srgrimes various features of liblzma. No such example programs have been 1091590Srgrimes written yet. 1101590Srgrimes 1111590Srgrimes For now, if you have never used liblzma, libbzip2, or zlib, I 1121590Srgrimes recommend learning the *basics* of the zlib API. Once you know that, 1131590Srgrimes it should be easier to learn liblzma. 1141590Srgrimes 1151590Srgrimes http://zlib.net/manual.html 1161590Srgrimes http://zlib.net/zlib_how.html 1171590Srgrimes 1181590Srgrimes 1191590Srgrimes2. Version numbering 1201590Srgrimes-------------------- 12118286Sbde 12218286Sbde The version number format of XZ Utils is X.Y.ZS: 1231590Srgrimes 1241590Srgrimes - X is the major version. When this is incremented, the library 1251590Srgrimes API and ABI break. 1261590Srgrimes 1271590Srgrimes - Y is the minor version. It is incremented when new features 1281590Srgrimes are added without breaking the existing API or ABI. An even Y 1291590Srgrimes indicates a stable release and an odd Y indicates unstable 1301590Srgrimes (alpha or beta version). 1311590Srgrimes 1321590Srgrimes - Z is the revision. This has a different meaning for stable and 1331590Srgrimes unstable releases: 13418286Sbde 1351590Srgrimes * Stable: Z is incremented when bugs get fixed without adding 1361590Srgrimes any new features. This is intended to be convenient for 1371590Srgrimes downstream distributors that want bug fixes but don't want 1381590Srgrimes any new features to minimize the risk of introducing new bugs. 1391590Srgrimes 1401590Srgrimes * Unstable: Z is just a counter. API or ABI of features added 1411590Srgrimes in earlier unstable releases having the same X.Y may break. 1421590Srgrimes 1431590Srgrimes - S indicates stability of the release. It is missing from the 1441590Srgrimes stable releases, where Y is an even number. When Y is odd, S 1451590Srgrimes is either "alpha" or "beta" to make it very clear that such 1461590Srgrimes versions are not stable releases. The same X.Y.Z combination is 1471590Srgrimes not used for more than one stability level, i.e. after X.Y.Zalpha, 1481590Srgrimes the next version can be X.Y.(Z+1)beta but not X.Y.Zbeta. 1491590Srgrimes 1501590Srgrimes 1511590Srgrimes3. Reporting bugs 1521590Srgrimes----------------- 1538232Sdg 15447549Sbde Naturally it is easiest for me if you already know what causes the 15540103Smarkm unexpected behavior. Even better if you have a patch to propose. 15640103Smarkm However, quite often the reason for unexpected behavior is unknown, 15740103Smarkm so here are a few things to do before sending a bug report: 15856590Sshin 15956590Sshin 1. Try to create a small example how to reproduce the issue. 1601590Srgrimes 1618232Sdg 2. Compile XZ Utils with debugging code using configure switches 1621590Srgrimes --enable-debug and, if possible, --disable-shared. If you are 16347549Sbde using GCC, use CFLAGS='-O0 -ggdb3'. Don't strip the resulting 1641590Srgrimes binaries. 16529922Smarkm 1661590Srgrimes 3. Turn on core dumps. The exact command depends on your shell; 1671590Srgrimes for example in GNU bash it is done with "ulimit -c unlimited", 1681590Srgrimes and in tcsh with "limit coredumpsize unlimited". 1691590Srgrimes 1701590Srgrimes 4. Try to reproduce the suspected bug. If you get "assertion failed" 1711590Srgrimes message, be sure to include the complete message in your bug 1721590Srgrimes report. If the application leaves a coredump, get a backtrace 1731590Srgrimes using gdb: 1741590Srgrimes $ gdb /path/to/app-binary # Load the app to the debugger. 1751590Srgrimes (gdb) core core # Open the coredump. 1761590Srgrimes (gdb) bt # Print the backtrace. Copy & paste to bug report. 1771590Srgrimes (gdb) quit # Quit gdb. 1781590Srgrimes 1791590Srgrimes Report your bug via email or IRC (see Contact information below). 18057232Sshin Don't send core dump files or any executables. If you have a small 1811590Srgrimes example file(s) (total size less than 256 KiB), please include 18257232Sshin it/them as an attachment. If you have bigger test files, put them 1831590Srgrimes online somewhere and include a URL to the file(s) in the bug report. 18424360Simp 1851590Srgrimes Always include the exact version number of XZ Utils in the bug report. 18657232Sshin If you are using a snapshot from the git repository, use "git describe" 18757232Sshin to get the exact snapshot version. If you are using XZ Utils shipped 18857232Sshin in an operating system distribution, mention the distribution name, 18957232Sshin distribution version, and exact xz package version; if you cannot 19057232Sshin repeat the bug with the code compiled from unpatched source code, 19157232Sshin you probably need to report a bug to your distribution's bug tracking 19257232Sshin system. 19357232Sshin 1941590Srgrimes 1951590Srgrimes4. Translating the xz tool 1961590Srgrimes-------------------------- 1978232Sdg 1988232Sdg The messages from the xz tool have been translated into a few 1998232Sdg languages. Before starting to translate into a new language, ask 2001590Srgrimes the author whether someone else hasn't already started working on it. 2011590Srgrimes 2021590Srgrimes Test your translation. Testing includes comparing the translated 2031590Srgrimes output to the original English version by running the same commands 2041590Srgrimes in both your target locale and with LC_ALL=C. Ask someone to 2051590Srgrimes proof-read and test the translation. 2061590Srgrimes 2071590Srgrimes Testing can be done e.g. by installing xz into a temporary directory: 2081590Srgrimes 2091590Srgrimes ./configure --disable-shared --prefix=/tmp/xz-test 2101590Srgrimes # <Edit the .po file in the po directory.> 2111590Srgrimes make -C po update-po 2121590Srgrimes make install 2131590Srgrimes bash debug/translation.bash | less 2141590Srgrimes bash debug/translation.bash | less -S # For --list outputs 2151590Srgrimes 2161590Srgrimes Repeat the above as needed (no need to re-run configure though). 2171590Srgrimes 21847488Speter Note especially the following: 21947549Sbde 22047549Sbde - The output of --help and --long-help must look nice on 22147488Speter an 80-column terminal. It's OK to add extra lines if needed. 22247488Speter 2231590Srgrimes - In contrast, don't add extra lines to error messages and such. 2241590Srgrimes They are often preceded with e.g. a filename on the same line, 2251590Srgrimes so you have no way to predict where to put a \n. Let the terminal 2261590Srgrimes do the wrapping even if it looks ugly. Adding new lines will be 2271590Srgrimes even uglier in the generic case even if it looks nice in a few 2281590Srgrimes limited examples. 2291590Srgrimes 2301590Srgrimes - Be careful with column alignment in tables and table-like output 2311590Srgrimes (--list, --list --verbose --verbose, --info-memory, --help, and 2321590Srgrimes --long-help): 2331590Srgrimes 2341590Srgrimes * All descriptions of options in --help should start in the 2351590Srgrimes same column (but it doesn't need to be the same column as 2361590Srgrimes in the English messages; just be consistent if you change it). 2371590Srgrimes Check that both --help and --long-help look OK, since they 2381590Srgrimes share several strings. 2391590Srgrimes 2401590Srgrimes * --list --verbose and --info-memory print lines that have 2411590Srgrimes the format "Description: %s". If you need a longer 2421590Srgrimes description, you can put extra space between the colon 2431590Srgrimes and %s. Then you may need to add extra space to other 2441590Srgrimes strings too so that the result as a whole looks good (all 2451590Srgrimes values start at the same column). 24634897Smarkm 2471590Srgrimes * The columns of the actual tables in --list --verbose --verbose 2481590Srgrimes should be aligned properly. Abbreviate if necessary. It might 24934897Smarkm be good to keep at least 2 or 3 spaces between column headings 2501590Srgrimes and avoid spaces in the headings so that the columns stand out 2511590Srgrimes better, but this is a matter of opinion. Do what you think 25227919Scharnier looks best. 25327919Scharnier 2541590Srgrimes - Be careful to put a period at the end of a sentence when the 2551590Srgrimes original version has it, and don't put it when the original 25647488Speter doesn't have it. Similarly, be careful with \n characters 25747488Speter at the beginning and end of the strings. 2581590Srgrimes 2591590Srgrimes - Read the TRANSLATORS comments that have been extracted from the 2601590Srgrimes source code and included in xz.pot. If they suggest testing the 26140103Smarkm translation with some type of command, do it. If testing needs 26240103Smarkm input files, use e.g. tests/files/good-*.xz. 26340103Smarkm 2641590Srgrimes - When updating the translation, read the fuzzy (modified) strings 2651590Srgrimes carefully, and don't mark them as updated before you actually 2661590Srgrimes have updated them. Reading through the unchanged messages can be 2671590Srgrimes good too; sometimes you may find a better wording for them. 26829922Smarkm 2691590Srgrimes - If you find language problems in the original English strings, 2701590Srgrimes feel free to suggest improvements. Ask if something is unclear. 2711590Srgrimes 2721590Srgrimes - The translated messages should be understandable (sometimes this 2731590Srgrimes may be a problem with the original English messages too). Don't 2741590Srgrimes make a direct word-by-word translation from English especially if 27527919Scharnier the result doesn't sound good in your language. 27627919Scharnier 2771590Srgrimes In short, take your time and pay attention to the details. Making 27817284Spst a good translation is not a quick and trivial thing to do. The 27917284Spst translated xz should look as polished as the English version. 28017284Spst 28117284Spst 28217284Spst5. Other implementations of the .xz format 2831590Srgrimes------------------------------------------ 2841590Srgrimes 2851590Srgrimes 7-Zip and the p7zip port of 7-Zip support the .xz format starting 2861590Srgrimes from the version 9.00alpha. 2871590Srgrimes 2881590Srgrimes http://7-zip.org/ 2891590Srgrimes http://p7zip.sourceforge.net/ 2901590Srgrimes 2911590Srgrimes XZ Embedded is a limited implementation written for use in the Linux 2921590Srgrimes kernel, but it is also suitable for other embedded use. 2931590Srgrimes 2941590Srgrimes http://tukaani.org/xz/embedded.html 2951590Srgrimes 2961590Srgrimes 2971590Srgrimes6. Contact information 2981590Srgrimes---------------------- 2991590Srgrimes 3001590Srgrimes If you have questions, bug reports, patches etc. related to XZ Utils, 3011590Srgrimes contact Lasse Collin <lasse.collin@tukaani.org> (in Finnish or English). 3021590Srgrimes I'm sometimes slow at replying. If you haven't got a reply within two 3031590Srgrimes weeks, assume that your email has got lost and resend it or use IRC. 30434897Smarkm 3051590Srgrimes You can find me also from #tukaani on Freenode; my nick is Larhzu. 3061590Srgrimes The channel tends to be pretty quiet, so just ask your question and 3071590Srgrimes someone may wake up. 3081590Srgrimes 3091590Srgrimes