You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Daniel Gredler <dg...@dhlglobalmail.com> on 2013/02/11 16:34:51 UTC
Support for zip (de)compression
Hi guys,
I've submitted a (backwards-compatible) patch adding support for the zip
(de)compression format [1]. The current ZipDataFormat actually
implements "deflate" (de)compression. Please take a look when you get a
chance.
Thanks!
Daniel
[1] https://issues.apache.org/jira/browse/CAMEL-6061
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> I'll handle this.
Release notes have been updated as well.
--
Henryk Konsek
http://henryk-konsek.blogspot.com
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> We should though add a note to this new data format at the 2.11
> release notes page.
I'll handle this.
I also applied some minor fixes to the wiki page of the component.
--
Henryk Konsek
http://henryk-konsek.blogspot.com
Re: Support for zip (de)compression
Posted by Claus Ibsen <cl...@gmail.com>.
On Sat, Feb 16, 2013 at 9:02 PM, Henryk Konsek <he...@gmail.com> wrote:
>> Version 4 uploaded and ready for review!
>
> Component is in the trunk now. Documentation is available as well.
>
> Thank you for the contribution Daniel. :)
>
Great and thanks for contribution. Don't let this stop you and just
dig in and help with other stuff that interrest you.
We should though add a note to this new data format at the 2.11
release notes page.
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> Version 4 uploaded and ready for review!
Component is in the trunk now. Documentation is available as well.
Thank you for the contribution Daniel. :)
--
Henryk Konsek
http://henryk-konsek.blogspot.com
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
Version 4 uploaded and ready for review!
-----Original Message-----
From: Henryk Konsek [mailto:hekonsek@gmail.com]
Sent: Friday, February 15, 2013 5:51 AM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
> A common module sounds nice. After all they are small libraries. Less
> overhead on wiki etc.
Actually I would prefer to stick to our original strategy i.e.
creating camel-zipfile component. After we release this data format,
we could start another discussion regarding the way to organize
compression-related data formats. I think we shouldn't mix adding new
component and reorganizing the existing ones.
Thank you for your input and contribution. I'm looking forward to
final version of the patch :) .
--
Henryk Konsek
http://henryk-konsek.blogspot.com
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> A common module sounds nice. After all they are small libraries. Less
> overhead on wiki etc.
Actually I would prefer to stick to our original strategy i.e.
creating camel-zipfile component. After we release this data format,
we could start another discussion regarding the way to organize
compression-related data formats. I think we shouldn't mix adding new
component and reorganizing the existing ones.
Thank you for your input and contribution. I'm looking forward to
final version of the patch :) .
--
Henryk Konsek
http://henryk-konsek.blogspot.com
RE: Support for zip (de)compression
Posted by David Karlsen <da...@gmail.com>.
A common module sounds nice. After all they are small libraries. Less
overhead on wiki etc.
Den 14. feb. 2013 18:08 skrev "Daniel Gredler" <dg...@dhlglobalmail.com>
følgende:
> Sounds good. The only thing I wanted to check first is if maybe we
> wanted to create a camel-compression component (containing the existing
> GzipDataFormat and ZipDataFormat, plus the new ZipFileDataFormat) or if
> the long-term goal is to have three different components (camel-zip,
> camel-zipfile, camel-gzip) -- in which case just creating the
> camel-zipfile component would be the best option.
>
>
> -----Original Message-----
> From: Henryk Konsek [mailto:hekonsek@gmail.com]
> Sent: Thursday, February 14, 2013 11:44 AM
> To: dev@camel.apache.org
> Subject: Re: Support for zip (de)compression
>
> > I have no idea what an enhancement to allow it to handle
> > multi-file Zips would look like :-)
>
> In the future we could support passing collection (maps in particular)
> of objects as a message to support nested directory hierarchies. But
> for the first version of this data format single-file-zip is fine, as
> this is pretty common use case for IT systems to generate files like
> this.
>
> We will create separated Maven module for your data format to keep
> camel-core as small as possible. Let's give it some nice artifactId
> like camel-zipfile. Please move the data format itself there and keep
> DSL in camel-core. Would you be so kind and apply appropriate changes
> to the patch?
>
> Thank you in advance.
>
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com
>
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
Sounds good. The only thing I wanted to check first is if maybe we
wanted to create a camel-compression component (containing the existing
GzipDataFormat and ZipDataFormat, plus the new ZipFileDataFormat) or if
the long-term goal is to have three different components (camel-zip,
camel-zipfile, camel-gzip) -- in which case just creating the
camel-zipfile component would be the best option.
-----Original Message-----
From: Henryk Konsek [mailto:hekonsek@gmail.com]
Sent: Thursday, February 14, 2013 11:44 AM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
> I have no idea what an enhancement to allow it to handle
> multi-file Zips would look like :-)
In the future we could support passing collection (maps in particular)
of objects as a message to support nested directory hierarchies. But
for the first version of this data format single-file-zip is fine, as
this is pretty common use case for IT systems to generate files like
this.
We will create separated Maven module for your data format to keep
camel-core as small as possible. Let's give it some nice artifactId
like camel-zipfile. Please move the data format itself there and keep
DSL in camel-core. Would you be so kind and apply appropriate changes
to the patch?
Thank you in advance.
--
Henryk Konsek
http://henryk-konsek.blogspot.com
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> I have no idea what an enhancement to allow it to handle
> multi-file Zips would look like :-)
In the future we could support passing collection (maps in particular)
of objects as a message to support nested directory hierarchies. But
for the first version of this data format single-file-zip is fine, as
this is pretty common use case for IT systems to generate files like
this.
We will create separated Maven module for your data format to keep
camel-core as small as possible. Let's give it some nice artifactId
like camel-zipfile. Please move the data format itself there and keep
DSL in camel-core. Would you be so kind and apply appropriate changes
to the patch?
Thank you in advance.
--
Henryk Konsek
http://henryk-konsek.blogspot.com
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
Hi James,
It uses Java's built-in Zip API, which AFAIK supports ZIP64 if you're on
Java 7 or later [1]. This ZipFileDataFormat is also for single-file
archives. I have no idea what an enhancement to allow it to handle
multi-file Zips would look like :-)
Take care,
Daniel
[1]
https://blogs.oracle.com/xuemingshen/entry/zip64_support_for_4g_zipfile
-----Original Message-----
From: jhart [mailto:james.hart@nokia.com]
Sent: Wednesday, February 13, 2013 1:55 PM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
Does this support multi file zip64 files? I hacked a zip64 handler that
only
handles expansion of single file zip64 files, but we really need to be
able
to handle multifile zip64 files.
--
View this message in context:
http://camel.465427.n5.nabble.com/Support-for-zip-de-compression-tp57273
57p5727547.html
Sent from the Camel Development mailing list archive at Nabble.com.
Re: Support for zip (de)compression
Posted by jhart <ja...@nokia.com>.
Does this support multi file zip64 files? I hacked a zip64 handler that only
handles expansion of single file zip64 files, but we really need to be able
to handle multifile zip64 files.
--
View this message in context: http://camel.465427.n5.nabble.com/Support-for-zip-de-compression-tp5727357p5727547.html
Sent from the Camel Development mailing list archive at Nabble.com.
RE: Support for zip (de)compression
Posted by Christian Müller <ch...@gmail.com>.
+1 for a new component.
In Camel 3.0 we try to stream line camel-core as much as possible.
Sent from a mobile device
Am 13.02.2013 19:54 schrieb "Daniel Gredler" <dg...@dhlglobalmail.com>:
> Ah, OK. I've uploaded a third patch that adds the DSL elements but keeps
> everything in camel-core. I'll wait for the outcome of the "camel-core
> vs new component" discussion before making any other changes.
>
> Take care,
>
> Daniel
>
>
> -----Original Message-----
> From: Henryk Konsek [mailto:hekonsek@gmail.com]
> Sent: Wednesday, February 13, 2013 1:44 PM
> To: dev@camel.apache.org
> Subject: Re: Support for zip (de)compression
>
> > Yep, I can probably do that today or tomorrow.
>
> Great to hear that.
>
> > Wouldn't it need to be part of camel-core in order to have DSL
> elements?
>
> The component code can be in separated module, while DSL definition
> only will stay in camel-core (org.apache.camel.model.dataformat
> package). DSL builders can create DataFormat instance without explicit
> reference to the concrete implementation.
>
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com
>
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
Ah, OK. I've uploaded a third patch that adds the DSL elements but keeps
everything in camel-core. I'll wait for the outcome of the "camel-core
vs new component" discussion before making any other changes.
Take care,
Daniel
-----Original Message-----
From: Henryk Konsek [mailto:hekonsek@gmail.com]
Sent: Wednesday, February 13, 2013 1:44 PM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
> Yep, I can probably do that today or tomorrow.
Great to hear that.
> Wouldn't it need to be part of camel-core in order to have DSL
elements?
The component code can be in separated module, while DSL definition
only will stay in camel-core (org.apache.camel.model.dataformat
package). DSL builders can create DataFormat instance without explicit
reference to the concrete implementation.
--
Henryk Konsek
http://henryk-konsek.blogspot.com
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> Yep, I can probably do that today or tomorrow.
Great to hear that.
> Wouldn't it need to be part of camel-core in order to have DSL elements?
The component code can be in separated module, while DSL definition
only will stay in camel-core (org.apache.camel.model.dataformat
package). DSL builders can create DataFormat instance without explicit
reference to the concrete implementation.
--
Henryk Konsek
http://henryk-konsek.blogspot.com
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
> Could you create Java and XML DSLs for the data format as well?
Yep, I can probably do that today or tomorrow.
> do we prefer to have this data format in camel-core ... or as a
separate module?
Wouldn't it need to be part of camel-core in order to have DSL elements?
Sorry if it's a dumb question, but I haven't contributed any DSL-related
code yet.
Take care,
Daniel
-----Original Message-----
From: Henryk Konsek [mailto:hekonsek@gmail.com]
Sent: Wednesday, February 13, 2013 11:48 AM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
> I've uploaded an updated patch. Let me know if there's anything else
> that needs changing.
Thank you for the fast update Daniel.
Could you create Java and XML DSLs for the data format as well? Feel
free to take existing zip [1] component's code as an example.
@Claus, do we prefer to have this data format in camel-core (as
existing zip data format) or as a separated module (like Stream [2]
component)? I personally prefer to keep it out of the core (just like
Stream component) in order to keep it out of the core as focused and
clean as possible.
[1] http://camel.apache.org/zip-dataformat.html
[2] http://camel.apache.org/stream.html
--
Henryk Konsek
http://henryk-konsek.blogspot.com
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
> I've uploaded an updated patch. Let me know if there's anything else
> that needs changing.
Thank you for the fast update Daniel.
Could you create Java and XML DSLs for the data format as well? Feel
free to take existing zip [1] component's code as an example.
@Claus, do we prefer to have this data format in camel-core (as
existing zip data format) or as a separated module (like Stream [2]
component)? I personally prefer to keep it out of the core (just like
Stream component) in order to keep it out of the core as focused and
clean as possible.
[1] http://camel.apache.org/zip-dataformat.html
[2] http://camel.apache.org/stream.html
--
Henryk Konsek
http://henryk-konsek.blogspot.com
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
Hi guys,
I've uploaded an updated patch. Let me know if there's anything else
that needs changing.
Take care,
Daniel
-----Original Message-----
From: Daniel Gredler [mailto:dgredler@dhlglobalmail.com]
Sent: Tuesday, February 12, 2013 12:13 PM
To: dev@camel.apache.org
Subject: RE: Support for zip (de)compression
Hi Henryk, Claus,
Cool, thanks for the feedback! I'll update the patch and upload it.
Take care,
Daniel
-----Original Message-----
From: Claus Ibsen [mailto:claus.ibsen@gmail.com]
Sent: Tuesday, February 12, 2013 9:09 AM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
On Tue, Feb 12, 2013 at 11:34 AM, Henryk Konsek <he...@gmail.com>
wrote:
> Hi Daniel,
>
>> The current ZipDataFormat actually
>> implements "deflate" (de)compression.
>
> I like this idea, as I needed such component myself.
>
> I suggest however to rename it to camel-zipfile data format, because
> a) zip2 should be reserved for better version of existing zip
> component b) 'zipfile' is more descriptive in the terms of what this
> data format actually does.
>
> I suggest also to use the exchange id as the default entry name while
> marshaling (instead of "file").
>
> Updated version of the patch will be highly appreciated :) .
>
Yeah well spotted Henryk. I like your suggestions.
> Best regards.
>
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
RE: Support for zip (de)compression
Posted by Daniel Gredler <dg...@dhlglobalmail.com>.
Hi Henryk, Claus,
Cool, thanks for the feedback! I'll update the patch and upload it.
Take care,
Daniel
-----Original Message-----
From: Claus Ibsen [mailto:claus.ibsen@gmail.com]
Sent: Tuesday, February 12, 2013 9:09 AM
To: dev@camel.apache.org
Subject: Re: Support for zip (de)compression
On Tue, Feb 12, 2013 at 11:34 AM, Henryk Konsek <he...@gmail.com>
wrote:
> Hi Daniel,
>
>> The current ZipDataFormat actually
>> implements "deflate" (de)compression.
>
> I like this idea, as I needed such component myself.
>
> I suggest however to rename it to camel-zipfile data format, because
> a) zip2 should be reserved for better version of existing zip
> component b) 'zipfile' is more descriptive in the terms of what this
> data format actually does.
>
> I suggest also to use the exchange id as the default entry name while
> marshaling (instead of "file").
>
> Updated version of the patch will be highly appreciated :) .
>
Yeah well spotted Henryk. I like your suggestions.
> Best regards.
>
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Support for zip (de)compression
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Feb 12, 2013 at 11:34 AM, Henryk Konsek <he...@gmail.com> wrote:
> Hi Daniel,
>
>> The current ZipDataFormat actually
>> implements "deflate" (de)compression.
>
> I like this idea, as I needed such component myself.
>
> I suggest however to rename it to camel-zipfile data format, because
> a) zip2 should be reserved for better version of existing zip
> component b) 'zipfile' is more descriptive in the terms of what this
> data format actually does.
>
> I suggest also to use the exchange id as the default entry name while
> marshaling (instead of "file").
>
> Updated version of the patch will be highly appreciated :) .
>
Yeah well spotted Henryk. I like your suggestions.
> Best regards.
>
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Support for zip (de)compression
Posted by Henryk Konsek <he...@gmail.com>.
Hi Daniel,
> The current ZipDataFormat actually
> implements "deflate" (de)compression.
I like this idea, as I needed such component myself.
I suggest however to rename it to camel-zipfile data format, because
a) zip2 should be reserved for better version of existing zip
component b) 'zipfile' is more descriptive in the terms of what this
data format actually does.
I suggest also to use the exchange id as the default entry name while
marshaling (instead of "file").
Updated version of the patch will be highly appreciated :) .
Best regards.
--
Henryk Konsek
http://henryk-konsek.blogspot.com