You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Alex Blewitt <al...@gmail.com> on 2006/06/21 00:59:54 UTC
Pack200 -- can now obtain files stored in archive
I've jumped ahead a bit on the pack200 archive, and written the
unpacking for file data stored in a pack200 archive. The current
implementation will barf if the file size is > 2Gb, because I'm
pulling all the data into a byte array at present (it's pretty memory
hungry anyway).
I've not got any output mechanisms coded yet, but performing a
Segment.parse(in) on a pack200 will return you a Segment, and from
there you can get the file_bits to reconstitute the data files. Should
be relatively easy to export those into an appropriate output format
at a later stage.
If there's any classes in the pack file at the moment, it will fall
over -- but that's because the format is organised as [constant pool,
class/bytecode stuff, file data]. So if there'sno class/bytecode
stuff, it just falls through to the file data afterwards. Obviously
this isn't a particularly likely scenario (unless source files are
stored in a Zip and then packed?) but at least it shows it's on the
right track.
The prior caveats still apply; it only works for un-Gzipped pack files
(i.e. those created with --no-gzip) and there's no reconstitution of
the goods at the other end. I'll probably spend some time on decoding
the bytecode in the near future, but that will take some time.
In other news, there was interest in getting the pack200 stuff under a
dual licence so that it could be included in the Gnu Classpath libs
too ... I'm open to that idea, as long as forking doesn't become an
issue. I've also put some notes together in OPML as to the ordering of
bands in a pack200 file in a separate harmony bug; not sure if it will
make it into the SVN tree in a suitable location.
I'd also like to suggest that we try to move the pack200 code out of
the archive module into its own dedicated archive-pack200 module. If
this code is to be reused in other environments (whether part of a
classlib/J2SE implementation, or as a library/driver for other VMs)
then it would be a good idea to separate out the implementation from
the Java interfaces. After all, the standard Sun VM allows you to
switch to a different pack200 provider using the
java.util.jar.Pack200.Unpacker system property. It would probably not
make sense for a provider to also have the remainder of the
java.util.jar classes in there.
Alex.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: Pack200 -- can now obtain files stored in archive
Posted by Tim Ellison <t....@gmail.com>.
Alex,
Don't worry too much about the code being in the archive module, I agree
that it makes sense to keep the Pack200 code separate so that it can be
reused in Eclipse/Classpath/whatever and putting it into archive does
not preclude that.
The harmony archive component keeps it separated for access control
reasons (based on people's prior access as declared in the contributor
questionnaire) and for harmony corresponds to a packaging/modularity
story (the code will go into archive.jar).
As you know, you have a separate package space in archive
(o.a.h.a.i.pack200) and there is no reason why that package (or any
others that you need with that prefix) cannot be extracted, built and
packaged separately for sharing. Obviously you have to continue to
ensure that you implement with Java APIs that will be found elsewhere,
and harmony will ensure that there is loose coupling into the pack200
implementation.
So keep going! If you would prefer to see a separate target in the
archive build.xml that produces just a pack200.jar then I'm sure that
could be arranged too.
Thanks for the continued contribution.
Regards,
Tim
Alex Blewitt wrote:
> On 21/06/06, Stepan Mishura <st...@gmail.com> wrote:
>> On 6/21/06, Alex Blewitt wrote:
>>
>> We agreed to separate providers implementation [1]. But I'm not sure the
>> same should be done for a concrete class implementation. In you case it
>> is implementation of java.util.jar.Pack200 but there are lots of other
>> cases
>> ,for example, java.security.Policy. So we'll got a lot of modules with
>> only
>> one class inside.
>
> It's not just a concrete class, though. The API basically boils down
> to two methods:
>
> unpack()
> pack()
>
> The thing is, that an implementation of this JSR isn't just a simple
> class that provides a couple of methods. There's a lot of detail in
> the coding, decoding, and subsequent parsing that goes on. There's
> also a few other utilities that would be useful for performing
> compression on data streams if people would be interested in using
> them.
>
> So don't confuse a simple API with a simple solution :-) The
> providers/ would be exactly the right place for this kind of code to
> live (at least, for the implementation). There are stub classes that
> are needed in the archive/ module, such as java.util.jar.Packer along
> with the factory code to instantiate the provider; but the factory
> instantiating the provider is actually specified as part of the
> Packer's static access methods.
>
> Alex.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: Pack200 -- can now obtain files stored in archive
Posted by Alex Blewitt <al...@gmail.com>.
On 21/06/06, Stepan Mishura <st...@gmail.com> wrote:
> On 6/21/06, Alex Blewitt wrote:
>
> We agreed to separate providers implementation [1]. But I'm not sure the
> same should be done for a concrete class implementation. In you case it
> is implementation of java.util.jar.Pack200 but there are lots of other cases
> ,for example, java.security.Policy. So we'll got a lot of modules with only
> one class inside.
It's not just a concrete class, though. The API basically boils down
to two methods:
unpack()
pack()
The thing is, that an implementation of this JSR isn't just a simple
class that provides a couple of methods. There's a lot of detail in
the coding, decoding, and subsequent parsing that goes on. There's
also a few other utilities that would be useful for performing
compression on data streams if people would be interested in using
them.
So don't confuse a simple API with a simple solution :-) The
providers/ would be exactly the right place for this kind of code to
live (at least, for the implementation). There are stub classes that
are needed in the archive/ module, such as java.util.jar.Packer along
with the factory code to instantiate the provider; but the factory
instantiating the provider is actually specified as part of the
Packer's static access methods.
Alex.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: Pack200 -- can now obtain files stored in archive
Posted by Stepan Mishura <st...@gmail.com>.
On 6/21/06, Alex Blewitt wrote:
>
> <SNIP>
> I'd also like to suggest that we try to move the pack200 code out of
> the archive module into its own dedicated archive-pack200 module. If
> this code is to be reused in other environments (whether part of a
> classlib/J2SE implementation, or as a library/driver for other VMs)
> then it would be a good idea to separate out the implementation from
> the Java interfaces. After all, the standard Sun VM allows you to
> switch to a different pack200 provider using the
> java.util.jar.Pack200.Unpacker system property. It would probably not
> make sense for a provider to also have the remainder of the
> java.util.jar classes in there.
We agreed to separate providers implementation [1]. But I'm not sure the
same should be done for a concrete class implementation. In you case it
is implementation of java.util.jar.Pack200 but there are lots of other cases
,for example, java.security.Policy. So we'll got a lot of modules with only
one class inside.
Thanks,
Stepan
[1]
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200601.mbox/%3c6e47b64f0601170332k3d418fabwd25a264c5e0f1532@mail.gmail.com%3e
------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: Pack200 -- can now obtain files stored in archive
Posted by Tim Ellison <t....@gmail.com>.
Alex Blewitt wrote:
> By the way, one of the tests has a Pack200 archive with Java classes
> in it, and one of the tests attempts to unpack that archive. With the
> recent change in 415891, it's possible that this test will fail now.
Looks ok on the latest test results (for r415945).
> But then, I'm not sure whether the tests are being run ?
Yes, the tests are being run. I see the following test counts:
org.apache.harmony.archive.tests.internal.pack200
CodecTest = 6
SegmentOptionstest = 1
Segmenttest = 1
Regards,
Tim
> If not, it's
> relatively easy to replace the HelloWorld.pack with a packed Jar file
> that just contains resources (e.g. GIFs, text files) to resolve the
> problem.
>
> Alex.
>
> On 20/06/06, Alex Blewitt <al...@gmail.com> wrote:
>> I've jumped ahead a bit on the pack200 archive, and written the
>> unpacking for file data stored in a pack200 archive. The current
>> implementation will barf if the file size is > 2Gb, because I'm
>> pulling all the data into a byte array at present (it's pretty memory
>> hungry anyway).
>>
>> I've not got any output mechanisms coded yet, but performing a
>> Segment.parse(in) on a pack200 will return you a Segment, and from
>> there you can get the file_bits to reconstitute the data files. Should
>> be relatively easy to export those into an appropriate output format
>> at a later stage.
>>
>> If there's any classes in the pack file at the moment, it will fall
>> over -- but that's because the format is organised as [constant pool,
>> class/bytecode stuff, file data]. So if there'sno class/bytecode
>> stuff, it just falls through to the file data afterwards. Obviously
>> this isn't a particularly likely scenario (unless source files are
>> stored in a Zip and then packed?) but at least it shows it's on the
>> right track.
>>
>> The prior caveats still apply; it only works for un-Gzipped pack files
>> (i.e. those created with --no-gzip) and there's no reconstitution of
>> the goods at the other end. I'll probably spend some time on decoding
>> the bytecode in the near future, but that will take some time.
>>
>> In other news, there was interest in getting the pack200 stuff under a
>> dual licence so that it could be included in the Gnu Classpath libs
>> too ... I'm open to that idea, as long as forking doesn't become an
>> issue. I've also put some notes together in OPML as to the ordering of
>> bands in a pack200 file in a separate harmony bug; not sure if it will
>> make it into the SVN tree in a suitable location.
>>
>> I'd also like to suggest that we try to move the pack200 code out of
>> the archive module into its own dedicated archive-pack200 module. If
>> this code is to be reused in other environments (whether part of a
>> classlib/J2SE implementation, or as a library/driver for other VMs)
>> then it would be a good idea to separate out the implementation from
>> the Java interfaces. After all, the standard Sun VM allows you to
>> switch to a different pack200 provider using the
>> java.util.jar.Pack200.Unpacker system property. It would probably not
>> make sense for a provider to also have the remainder of the
>> java.util.jar classes in there.
>>
>> Alex.
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: Pack200 -- can now obtain files stored in archive
Posted by Alex Blewitt <al...@gmail.com>.
By the way, one of the tests has a Pack200 archive with Java classes
in it, and one of the tests attempts to unpack that archive. With the
recent change in 415891, it's possible that this test will fail now.
But then, I'm not sure whether the tests are being run ? If not, it's
relatively easy to replace the HelloWorld.pack with a packed Jar file
that just contains resources (e.g. GIFs, text files) to resolve the
problem.
Alex.
On 20/06/06, Alex Blewitt <al...@gmail.com> wrote:
> I've jumped ahead a bit on the pack200 archive, and written the
> unpacking for file data stored in a pack200 archive. The current
> implementation will barf if the file size is > 2Gb, because I'm
> pulling all the data into a byte array at present (it's pretty memory
> hungry anyway).
>
> I've not got any output mechanisms coded yet, but performing a
> Segment.parse(in) on a pack200 will return you a Segment, and from
> there you can get the file_bits to reconstitute the data files. Should
> be relatively easy to export those into an appropriate output format
> at a later stage.
>
> If there's any classes in the pack file at the moment, it will fall
> over -- but that's because the format is organised as [constant pool,
> class/bytecode stuff, file data]. So if there'sno class/bytecode
> stuff, it just falls through to the file data afterwards. Obviously
> this isn't a particularly likely scenario (unless source files are
> stored in a Zip and then packed?) but at least it shows it's on the
> right track.
>
> The prior caveats still apply; it only works for un-Gzipped pack files
> (i.e. those created with --no-gzip) and there's no reconstitution of
> the goods at the other end. I'll probably spend some time on decoding
> the bytecode in the near future, but that will take some time.
>
> In other news, there was interest in getting the pack200 stuff under a
> dual licence so that it could be included in the Gnu Classpath libs
> too ... I'm open to that idea, as long as forking doesn't become an
> issue. I've also put some notes together in OPML as to the ordering of
> bands in a pack200 file in a separate harmony bug; not sure if it will
> make it into the SVN tree in a suitable location.
>
> I'd also like to suggest that we try to move the pack200 code out of
> the archive module into its own dedicated archive-pack200 module. If
> this code is to be reused in other environments (whether part of a
> classlib/J2SE implementation, or as a library/driver for other VMs)
> then it would be a good idea to separate out the implementation from
> the Java interfaces. After all, the standard Sun VM allows you to
> switch to a different pack200 provider using the
> java.util.jar.Pack200.Unpacker system property. It would probably not
> make sense for a provider to also have the remainder of the
> java.util.jar classes in there.
>
> Alex.
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org