#
0ecd7516 |
|
28-Feb-2023 |
Augustin Cavalier <waddlesplash@gmail.com> |
Package Kit: Use an object_cache in kernel mode for decompression buffers. The kernel heap only uses object caches for objects up to size 8192. Larger allocations have to go through the raw allocator. That can get pretty expensive. Adding instrumentation around the malloc/free calls in this function showed that on my machine, some 596ms during boot were spent on *malloc/free alone*, all else aside. After this change, we are at around 110ms, or a >5x improvement. Running an fgrep -R on /system/ after boot increased the cumulative time in memory functions to over 5 seconds, while after this change it is "only" 1170ms. Honestly, it seems like the object depots should be able to be faster than that, even if this function is called thousands of times. But that is a problem for a different investigation. It would be even faster for every consumer of this data in packagefs just allocated one set of buffers up front, or at least for a single "read session", but plumbing that all the way through the myriad abstractions of the Package Kit will not be easy, and is left for another time, as well.
|
#
8f5130ed |
|
12-Jul-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
package kit: Actually add support for B_HPKG_COMPRESSION_NONE Until now we always declared in the HPKG header that the package file is zlib compressed. For uncompressed files we would just store all individual chunks uncompressed. Now we handle completely uncompressed files slightly differently: We don't write the redundant chunk size table anymore. The size savings are minor, but it makes the uncompressed format read-streamable which may be handy.
|
#
e527b796 |
|
12-Jul-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Switch package file accessor classes to use BPositionIO * PackageFileHeap{Reader,Writer} as well as Package{Reader,Writer} and their implementation and super classes do now internally use a BPositionIO instead of a FD to access the package file. This provides more flexibility needed for features to come. * BPackageReader has already grown a new Init() version with a BPositionIO* parameter.
|
#
cdfeba5a |
|
08-Jul-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
BPackageWriter: Add Recompress() Allows rewriting an existing package file with a different compression.
|
#
b3263ad3 |
|
30-Jun-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Switch package kit to BZlibCompressionAlgorithm ... and remove the Zlib{Compressor,Decompressor} API.
|
#
32832cbe |
|
23-Jun-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Remove BPackageKit::BDataOutput Use BDataIO instead.
|
#
d2d1af83 |
|
18-Jun-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Revert "Move ZlibDecompressor to libshared" This reverts commit 9af2105d36856be60528c361edf14a263f242338. Conflicts: src/kits/package/Jamfile
|
#
9af2105d |
|
10-Feb-2014 |
Adrien Destugues <pulkomandy@pulkomandy.tk> |
Move ZlibDecompressor to libshared * This will be used to implement compressed http streams * Remove the custom BDataOutput class, and use BDataIO instead, for easier integration with existing code.
|
#
520a7a76 |
|
18-May-2013 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
PackageFileHeapAccessorBase: small refactoring * Pull method DecompressChunkData() out of ReadAndDecompressChunkData() for potential reuse. * Also fix/improve some error output strings.
|
#
1f633814 |
|
17-May-2013 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
hpkg format: compress the whole heap Instead of handling compression for individual file/attribute data we do now compress the whole heap where they are stored. This significantly improves compression ratios. We still divide the uncompressed data into 64 KiB chunks and use a chunk offset array for the compressed chunks to allow for quick random access without too much overhead. The tradeoff is a limited possible compression ratio -- i.e. we won't be as good as tar.gz (though surprisingly with my test archives we did better than zip). The other package file sections (package attributes and TOC) are no longer compressed individually. Their uncompressed data are simply pushed onto the heap where the usual compression strategy applies. To simplify things the repository format has been changed in the same manner although it doesn't otherwise use the heap, since it only stores meta data. Due to the data compression having been exposed in public and private API, this change touches a lot of package kit using code, including packagefs and the boot loader packagefs support. The latter two haven't been tested yet. Moreover packagefs needs a new kind of cache so we avoid re-reading the same heap chunk for two different data items it contains.
|
#
8f5130edfa454cc7ee21e4c3aef2a151f7c3c567 |
|
12-Jul-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
package kit: Actually add support for B_HPKG_COMPRESSION_NONE Until now we always declared in the HPKG header that the package file is zlib compressed. For uncompressed files we would just store all individual chunks uncompressed. Now we handle completely uncompressed files slightly differently: We don't write the redundant chunk size table anymore. The size savings are minor, but it makes the uncompressed format read-streamable which may be handy.
|
#
e527b796319f21ca025f68e1964df140daa6de35 |
|
12-Jul-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Switch package file accessor classes to use BPositionIO * PackageFileHeap{Reader,Writer} as well as Package{Reader,Writer} and their implementation and super classes do now internally use a BPositionIO instead of a FD to access the package file. This provides more flexibility needed for features to come. * BPackageReader has already grown a new Init() version with a BPositionIO* parameter.
|
#
cdfeba5a1e49d0993e7ab6853c77392bbc10d3c4 |
|
08-Jul-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
BPackageWriter: Add Recompress() Allows rewriting an existing package file with a different compression.
|
#
b3263ad3e1662cc419849a2d2cf539f9636613dc |
|
30-Jun-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Switch package kit to BZlibCompressionAlgorithm ... and remove the Zlib{Compressor,Decompressor} API.
|
#
32832cbe47f991cc6d2b29824903181d8baaaa63 |
|
23-Jun-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Remove BPackageKit::BDataOutput Use BDataIO instead.
|
#
d2d1af830b3f144937ea5e4c3f0e5b74c4c3d737 |
|
18-Jun-2014 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
Revert "Move ZlibDecompressor to libshared" This reverts commit 9af2105d36856be60528c361edf14a263f242338. Conflicts: src/kits/package/Jamfile
|
#
9af2105d36856be60528c361edf14a263f242338 |
|
10-Feb-2014 |
Adrien Destugues <pulkomandy@pulkomandy.tk> |
Move ZlibDecompressor to libshared * This will be used to implement compressed http streams * Remove the custom BDataOutput class, and use BDataIO instead, for easier integration with existing code.
|
#
520a7a76c7c6528f654a4e9a6d0d291df63e7532 |
|
18-May-2013 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
PackageFileHeapAccessorBase: small refactoring * Pull method DecompressChunkData() out of ReadAndDecompressChunkData() for potential reuse. * Also fix/improve some error output strings.
|
#
1f633814fa347b0bd822f683af731615df600347 |
|
17-May-2013 |
Ingo Weinhold <ingo_weinhold@gmx.de> |
hpkg format: compress the whole heap Instead of handling compression for individual file/attribute data we do now compress the whole heap where they are stored. This significantly improves compression ratios. We still divide the uncompressed data into 64 KiB chunks and use a chunk offset array for the compressed chunks to allow for quick random access without too much overhead. The tradeoff is a limited possible compression ratio -- i.e. we won't be as good as tar.gz (though surprisingly with my test archives we did better than zip). The other package file sections (package attributes and TOC) are no longer compressed individually. Their uncompressed data are simply pushed onto the heap where the usual compression strategy applies. To simplify things the repository format has been changed in the same manner although it doesn't otherwise use the heap, since it only stores meta data. Due to the data compression having been exposed in public and private API, this change touches a lot of package kit using code, including packagefs and the boot loader packagefs support. The latter two haven't been tested yet. Moreover packagefs needs a new kind of cache so we avoid re-reading the same heap chunk for two different data items it contains.
|