History log of /haiku/src/kits/package/hpkg/RepositoryWriterImpl.cpp
Revision Date Author Comments
# b08627f3 04-May-2018 Murai Takashi <tmurai01@gmail.com>

Fix catching polymorphic type by value

Replace catching polymorphic type std::bad_alloc 'by value'
with 'by reference'.
Pointed by gcc8


# 44c47711 12-Jul-2014 Ingo Weinhold <ingo_weinhold@gmx.de>

BPackageWriter: Add BPositionIO support


# 8f5130ed 12-Jul-2014 Ingo Weinhold <ingo_weinhold@gmx.de>

package kit: Actually add support for B_HPKG_COMPRESSION_NONE

Until now we always declared in the HPKG header that the package file is
zlib compressed. For uncompressed files we would just store all
individual chunks uncompressed. Now we handle completely uncompressed
files slightly differently: We don't write the redundant chunk size
table anymore. The size savings are minor, but it makes the uncompressed
format read-streamable which may be handy.


# 50ae264d 17-Sep-2013 Oliver Tappe <zooey@hirschkaefer.de>

Add support for adding a package-info to BRepositoryWriter.


# e8533402 04-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Repository file format: group attributes of a package

* Introduce new package attribute B_HPKG_ATTRIBUTE_ID_PACKAGE (valid
only in a repository file) to group the attributes belonging to a
package.
* BRepositoryContentHandler:
- No longer derive from BPackageContentHandler.
- Add hooks HandlePackage() and HandlePackageDone() that bracket the
attributes for a package. This is more explicit and robust than
handlers having to guess when one package ended and the next began.
* BRepositoryCache: Make use of BPackageInfoContentHandler. No need to
duplicate the code for reading a package info from package info
attributes.


# 695a1b24 04-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

repository writer: fix use of incorrect header size

The header size of the package file header was used, which resulted in
broken repository files.


# 3c3dd841 03-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

RepositoryWriterImpl::_AddPackage(): more info in error messages


# fd28164d 03-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Also allow source packages in a repository for any architecture


# bac0bb2e 21-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

RepositoryWriterImpl: Write the heap fields in the header


# 47039b85 21-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Package/repository file format: Add a minor version header field

* Add minor_version to hpkg_header and hpkg_repo_header and make
heap_compression uint16.
* If the minor version of a package/repository file is greater than the
current one unknown attributes are ignored without error. This allows
introducing new harmless attributes without making the resulting files
unreadable for older package kit versions.


# 796343ed 18-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Allow specifying the compression level for package creation

* Introduce BPackageWriterParameters which comprises all parameters
for package creation, currently flags and compression level. Such an
object can be passed to BPackageWriter::Init() and is passed on to
PackageWriterImpl and WriterImplBase.
* PackageFileHeapWriter: Add compressionLevel property and pass the
value on to ZlibCompressor.
* package add/create: Add options -0 ... -9 to specify the compression
level to be used.


# 1f633814 17-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

hpkg format: compress the whole heap

Instead of handling compression for individual file/attribute data we
do now compress the whole heap where they are stored. This
significantly improves compression ratios. We still divide the
uncompressed data into 64 KiB chunks and use a chunk offset array for
the compressed chunks to allow for quick random access without too much
overhead. The tradeoff is a limited possible compression ratio -- i.e.
we won't be as good as tar.gz (though surprisingly with my test
archives we did better than zip).

The other package file sections (package attributes and TOC) are no
longer compressed individually. Their uncompressed data are simply
pushed onto the heap where the usual compression strategy applies. To
simplify things the repository format has been changed in the same
manner although it doesn't otherwise use the heap, since it only stores
meta data.

Due to the data compression having been exposed in public and private
API, this change touches a lot of package kit using code, including
packagefs and the boot loader packagefs support. The latter two haven't
been tested yet. Moreover packagefs needs a new kind of cache so we
avoid re-reading the same heap chunk for two different data items it
contains.


# 0ee16518 14-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Rename BBlockBufferCache and friends to *Pool*

Also move BBlockBufferPoolNoLock into BHPKG namespace with the other
classes. Not sure why it wasn't there before.


# 2c32402d 12-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

package kit: internalize BPackageDataReader

It is no longer public (or even private) API. BPackageDataReaderFactory
returns a BAbstractBufferedDataReader instead. The advantage is that
the latter doesn't have hpkg format specific dependencies.


# 26265b7b 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add class BPackageInfoContentHandler

A BPackageContentHandler subclass that initializes a BPackageInfo from
the read package attributes. Pulled out of RepositoryWriterImpl's
PackageContentHandler.


# 9968845d 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Use BStringList in package kit

Replace all instances of BObjectList<BString> by BStringList.


# 56fed01d 12-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Handle the newer package attributes


# 00bc8e9c 02-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add support for adding/updating package entries

Add flags parameter to BPackageWriter::Init() (and the private
implementation classes) to indicate that an existing package file shall
be updated instead of created. Currently that always happens in-place.


# 10b8f702 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add class BPackageInfoContentHandler

A BPackageContentHandler subclass that initializes a BPackageInfo from
the read package attributes. Pulled out of RepositoryWriterImpl's
PackageContentHandler.


# 2021c984 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Use BStringList in package kit

Replace all instances of BObjectList<BString> by BStringList.


# 30ab1339 12-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Handle the newer package attributes


# 327b38d6 02-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add support for adding/updating package entries

Add flags parameter to BPackageWriter::Init() (and the private
implementation classes) to indicate that an existing package file shall
be updated instead of created. Currently that always happens in-place.


# f5644b8d 15-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* prefer HashSet and HashMap to respective std-classes where possible

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40518 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 34d56c1b 14-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* implemented repository reader and fixed some bugs in writer that
have been exposed during testing of reader

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40494 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c2388992 14-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* added embedding of license texts that need approval into repository

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40491 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 33bc4425 12-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Largish adjustments to PackagerReaderImpl and package attribute
handling:
* package attributes are now compatible with the low level attribute
handling of other HPKG attributes (such that 'package dump' now shows
package attributes, too)
* dropped type names from hpkg format, the attributes were identified
by IDs already and this simplifies the code considerably. Type names
are now handled in BLowLevelPackageHandler only.
* instead of rolling their own mechanism, high-level package attributes
handling is now implemented via a corresonding set of
AttributeHandler-subclasses
* adjusted package writer to only write package attributes that are
needed (empty ones are left out)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40466 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4dcc5232 10-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Fix problems in repository writer
* actually write the checksum
* compress the repository info archive, too (handle it as separate
section)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40426 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 91ebce66 10-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Some cleanup, no functional change
* typedef base classes as 'inherited' and use that instead of the
explicit base class name
* remove superfluous scopes in case blocks


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40420 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0d68f6af 09-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Implemented repository writing:
* fleshed out RepositoryWriterImpl
* renamed BRepositoryHeader to BRepositoryInfo (in accordance with
BPackageInfo)
* adjusted BRepositoryInfo to be able to parse itself from a
driver_settings file
* added package_repo binary (only 'create' works as of yet)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40405 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8b94d2ff 08-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* add stubbed RepositoryWriter to build

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40394 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4ee7d007 08-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Cleanup:
* PackageWriterImpl no longer accesses members of WriterImplBase
directly, but uses getters/setters instead
* package attribute registration has been moved from PackageWriterImpl
to WriterImplBase, as it will be used in the same manner by the
RepositoryWriter


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40388 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 44c4771163b9166dc042f69cb5373c9de9e37006 12-Jul-2014 Ingo Weinhold <ingo_weinhold@gmx.de>

BPackageWriter: Add BPositionIO support


# 8f5130edfa454cc7ee21e4c3aef2a151f7c3c567 12-Jul-2014 Ingo Weinhold <ingo_weinhold@gmx.de>

package kit: Actually add support for B_HPKG_COMPRESSION_NONE

Until now we always declared in the HPKG header that the package file is
zlib compressed. For uncompressed files we would just store all
individual chunks uncompressed. Now we handle completely uncompressed
files slightly differently: We don't write the redundant chunk size
table anymore. The size savings are minor, but it makes the uncompressed
format read-streamable which may be handy.


# 50ae264dad49d8dbb806be0c4b5a4838fa742bbf 17-Sep-2013 Oliver Tappe <zooey@hirschkaefer.de>

Add support for adding a package-info to BRepositoryWriter.


# e85334022eac93e41fc48b61784336b25ea2bbc3 04-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Repository file format: group attributes of a package

* Introduce new package attribute B_HPKG_ATTRIBUTE_ID_PACKAGE (valid
only in a repository file) to group the attributes belonging to a
package.
* BRepositoryContentHandler:
- No longer derive from BPackageContentHandler.
- Add hooks HandlePackage() and HandlePackageDone() that bracket the
attributes for a package. This is more explicit and robust than
handlers having to guess when one package ended and the next began.
* BRepositoryCache: Make use of BPackageInfoContentHandler. No need to
duplicate the code for reading a package info from package info
attributes.


# 695a1b248a193f9a403d76d3b32666240ee76fcc 04-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

repository writer: fix use of incorrect header size

The header size of the package file header was used, which resulted in
broken repository files.


# 3c3dd841a7d5686c53414606648eb0197a74f3cd 03-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

RepositoryWriterImpl::_AddPackage(): more info in error messages


# fd28164da0fc4bb89fc22731e0c63957d4aafe93 03-Jul-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Also allow source packages in a repository for any architecture


# bac0bb2e8f4278b3d2b706e983bd8817aea09cc0 21-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

RepositoryWriterImpl: Write the heap fields in the header


# 47039b852eeebb91396d19e0f9313cf23bf535ee 21-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Package/repository file format: Add a minor version header field

* Add minor_version to hpkg_header and hpkg_repo_header and make
heap_compression uint16.
* If the minor version of a package/repository file is greater than the
current one unknown attributes are ignored without error. This allows
introducing new harmless attributes without making the resulting files
unreadable for older package kit versions.


# 796343ed73b1707ed97e91e22b72b8ca81bcdbad 18-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Allow specifying the compression level for package creation

* Introduce BPackageWriterParameters which comprises all parameters
for package creation, currently flags and compression level. Such an
object can be passed to BPackageWriter::Init() and is passed on to
PackageWriterImpl and WriterImplBase.
* PackageFileHeapWriter: Add compressionLevel property and pass the
value on to ZlibCompressor.
* package add/create: Add options -0 ... -9 to specify the compression
level to be used.


# 1f633814fa347b0bd822f683af731615df600347 17-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

hpkg format: compress the whole heap

Instead of handling compression for individual file/attribute data we
do now compress the whole heap where they are stored. This
significantly improves compression ratios. We still divide the
uncompressed data into 64 KiB chunks and use a chunk offset array for
the compressed chunks to allow for quick random access without too much
overhead. The tradeoff is a limited possible compression ratio -- i.e.
we won't be as good as tar.gz (though surprisingly with my test
archives we did better than zip).

The other package file sections (package attributes and TOC) are no
longer compressed individually. Their uncompressed data are simply
pushed onto the heap where the usual compression strategy applies. To
simplify things the repository format has been changed in the same
manner although it doesn't otherwise use the heap, since it only stores
meta data.

Due to the data compression having been exposed in public and private
API, this change touches a lot of package kit using code, including
packagefs and the boot loader packagefs support. The latter two haven't
been tested yet. Moreover packagefs needs a new kind of cache so we
avoid re-reading the same heap chunk for two different data items it
contains.


# 0ee1651856a4364bfa7ffd10f796baf35fb524d3 14-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

Rename BBlockBufferCache and friends to *Pool*

Also move BBlockBufferPoolNoLock into BHPKG namespace with the other
classes. Not sure why it wasn't there before.


# 2c32402da583cbd0a091a08308237592679c05c4 12-May-2013 Ingo Weinhold <ingo_weinhold@gmx.de>

package kit: internalize BPackageDataReader

It is no longer public (or even private) API. BPackageDataReaderFactory
returns a BAbstractBufferedDataReader instead. The advantage is that
the latter doesn't have hpkg format specific dependencies.


# 26265b7b4de39aee69a0d000c688bfa975cd9b5d 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add class BPackageInfoContentHandler

A BPackageContentHandler subclass that initializes a BPackageInfo from
the read package attributes. Pulled out of RepositoryWriterImpl's
PackageContentHandler.


# 9968845d6988000a9235d7eb09031e80d997f31d 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Use BStringList in package kit

Replace all instances of BObjectList<BString> by BStringList.


# 56fed01db3f999d9e85635c9ccacf92497360975 12-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Handle the newer package attributes


# 00bc8e9cbd92f6a812d75ad75ca91c34beb67e72 02-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add support for adding/updating package entries

Add flags parameter to BPackageWriter::Init() (and the private
implementation classes) to indicate that an existing package file shall
be updated instead of created. Currently that always happens in-place.


# 10b8f702768b083af8107f4418fdc910151d6d09 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add class BPackageInfoContentHandler

A BPackageContentHandler subclass that initializes a BPackageInfo from
the read package attributes. Pulled out of RepositoryWriterImpl's
PackageContentHandler.


# 2021c9842e1e61a3bf0f6550725ec3baf53d44b1 16-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Use BStringList in package kit

Replace all instances of BObjectList<BString> by BStringList.


# 30ab13397ba65e2454ee4bb971565fe8457a34e5 12-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Handle the newer package attributes


# 327b38d69c8d9a128c64d0535f15b0274b51c51b 02-Jul-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

Add support for adding/updating package entries

Add flags parameter to BPackageWriter::Init() (and the private
implementation classes) to indicate that an existing package file shall
be updated instead of created. Currently that always happens in-place.


# f5644b8d3b228dc83dec139c43c0f6e55a3d984f 15-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* prefer HashSet and HashMap to respective std-classes where possible

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40518 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 34d56c1bf5e8990731ac21d629807f4b8a83f921 14-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* implemented repository reader and fixed some bugs in writer that
have been exposed during testing of reader

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40494 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c2388992ec05d5252a3c8a71a6420effce548952 14-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* added embedding of license texts that need approval into repository

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40491 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 33bc4425be0b545ed3d0b7e7a9a873d31e5a0330 12-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Largish adjustments to PackagerReaderImpl and package attribute
handling:
* package attributes are now compatible with the low level attribute
handling of other HPKG attributes (such that 'package dump' now shows
package attributes, too)
* dropped type names from hpkg format, the attributes were identified
by IDs already and this simplifies the code considerably. Type names
are now handled in BLowLevelPackageHandler only.
* instead of rolling their own mechanism, high-level package attributes
handling is now implemented via a corresonding set of
AttributeHandler-subclasses
* adjusted package writer to only write package attributes that are
needed (empty ones are left out)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40466 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4dcc52327b52d043011c084e4ae8894b1f2d1481 10-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Fix problems in repository writer
* actually write the checksum
* compress the repository info archive, too (handle it as separate
section)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40426 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 91ebce663d5f11e5763851ad131e55d4c020421b 10-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Some cleanup, no functional change
* typedef base classes as 'inherited' and use that instead of the
explicit base class name
* remove superfluous scopes in case blocks


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40420 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0d68f6afb34d8e01421123b216e34771d1c7fbfe 09-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Implemented repository writing:
* fleshed out RepositoryWriterImpl
* renamed BRepositoryHeader to BRepositoryInfo (in accordance with
BPackageInfo)
* adjusted BRepositoryInfo to be able to parse itself from a
driver_settings file
* added package_repo binary (only 'create' works as of yet)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40405 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8b94d2ff6a9d32f99c56f0146765fe2e90d6b33c 08-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

* add stubbed RepositoryWriter to build

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40394 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4ee7d007d6aa14cd7d9286e52dbb57580fd8f8f7 08-Feb-2011 Oliver Tappe <zooey@hirschkaefer.de>

Cleanup:
* PackageWriterImpl no longer accesses members of WriterImplBase
directly, but uses getters/setters instead
* package attribute registration has been moved from PackageWriterImpl
to WriterImplBase, as it will be used in the same manner by the
RepositoryWriter


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@40388 a95241bf-73f2-0310-859d-f6bbb57e9c96