You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicemix.apache.org by "James Bradt (JIRA)" <ji...@apache.org> on 2007/03/09 04:29:21 UTC
[jira] Created: (SM-875) TrueZIP binding component
TrueZIP binding component
-------------------------
Key: SM-875
URL: https://issues.apache.org/activemq/browse/SM-875
Project: ServiceMix
Issue Type: New Feature
Affects Versions: 3.1
Reporter: James Bradt
suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SM-875) TrueZIP binding component
Posted by bradtwurst <na...@xoxy.net>.
>
> protected String dynamicURI =
> "truezip://target/dynamicEndpoint.zip?file.tempFilePrefix=dynamicEp-";
>
> Note the "truezip:" protocol used.
> You should be able to debug the component's getResolvedEPR method ...
>
> Also, if you could change the package of the test classes to match the one
> of the sources
> (org.apache.servicemix.truezip instead of org.apache.servicemix.file)
> that would be cool.
>
> On the class loader potential problem, we need to dig that a bit more. I
> guess the problem
> happen when several copies of the libraries try to access the same files.
> This should not be
> the case, as the library will be embedded inside the component
> classloader, and unless you
> install several different copies of this component, it should be fine.
>
>
Thanks for the information about the dynamic endpoint - I will investigate.
Also, I'll correct the test class package name - that's what I get for doing
this on the quick :)
I will also continue to monitor the potential class loader problem and see
if anything occurs.
James
--
View this message in context: http://www.nabble.com/Created%3A-%28SM-875%29-TrueZIP-binding-component-tf3373581s12049.html#a9401053
Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "James Lorenzen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38770 ]
James Lorenzen commented on SM-875:
-----------------------------------
Guillaume,
To Compress/Decompress for any component you might look at our Encoding SE available here: http://www.glassfishwiki.org/jbiwiki/Wiki.jsp?page=EncodingSE
It provides the ability to compress (encode) and decompress (decode) the content of a NormalizedMessage. Currently it supports GZIP and Fast InfoSet. For low bandwidth environments its been proven to drastically improve performance. Its been used in conjunction with the ServiceMix HTTP BC, ServiceMix EIP SE, and the ServiceMix JSR181 component.
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Juergen Mayrbaeurl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38739 ]
Juergen Mayrbaeurl commented on SM-875:
---------------------------------------
Just looked at the TrueZIP website:
Under 'Current Limitations / Caveats' on the home page there's the following note:
"TrueZIP may seem to behave erratically if it is loaded and used by multiple class loaders. For each archive file which has been presented to TrueZIP (via the de.schlichtherle.io.File* classes), TrueZIP associates some internal state with it. This data is mostly held in some static maps. Either make sure that multiple class loader instances never use the same archive file or prevent the loading of multiple instances by assigning a shared parent class loader which loads all TrueZIP classes first"
Since ServiceMix is heavily using multiple classloaders I'm not sure how to make sure that the above noted conditions can be achieved. Maybe Guillaume can comment on this!
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38740 ]
Guillaume Nodet commented on SM-875:
------------------------------------
Is the purpose of this component to behave exactly as the servicemix-file, but compressing / decompressing data at the same time ? I mean will it poll for compressed files in a standard folder, or will it poll for files in a compressed archive ?
In the first case, I agree it would be better to use a derived Marshaler for that. One of the reason is that it would allow other
BC to leverage it too, like servicemix-ftp. In the second one, I wonder if using commons-vfs would be a better bet ?
I had some concerns with commons-vfs, but it seems the cache layer that caused some troubles in the past can now be disabled, so it might be a good idea to take a look..
James, can you comment ?
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "James Bradt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38735 ]
James Bradt commented on SM-875:
--------------------------------
I'm attaching a zip file with a binding component that appears to work.
The only issue that I encountered is that I could not get dynamic endpoint resolution to work.
Hopefully, someone with more experience in that area can resolve the matter quickly.
I've included a readme.txt to explain things as much as I can.
TrueZIP is not of my making. I am just a user of it. It does have an ASL2.0 license
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Juergen Mayrbaeurl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38738 ]
Juergen Mayrbaeurl commented on SM-875:
---------------------------------------
Maybe you should consider generating a FileMarshaler implementation as a subclass of DefaultFileMarshaler.
Regarding your problems with the dynamic endpoint resolution: Did you look at the sources of the servicemix-file binding component?
Why did you choose to use TrueZIP? I don't know TrueZIP. In the past we've used the Plexus archiver component (used by Maven, too) to do this kind of things. Can you change your binding component to use different implementations, so the user can configure if he wants to use TrueZIP or Plexus?
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "James Bradt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38747 ]
James Bradt commented on SM-875:
--------------------------------
Also, I took a quick look at the plexus archiver. I could not find a lot of information about it. Does anyone have a link for clean doc's for it?
My concern with using a marshaler for compression is that my need is to not just compress a message, it is to create a zip file and be able to add additional messages(files) to it in the future. This involves dealing with the current zip file, adding the data to it correctly, and then updating the zip header information as needed.
It appears to me that this get beyonds what the marshaller should be doing. I could be wrong, maybe the plexus archiver can do this. I just can't find the doc's for it.
If I was looking at just compressing a message, wouldn't the standard gzip input / output streams be applicable. Maybe that is a different jira to create this type of marshaler.
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "James Bradt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38866 ]
James Bradt commented on SM-875:
--------------------------------
I will get to creating the wiki entry in the next few days.
I'm short of free cycles right at the moment, but I'll do it as soon as I can.
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Components: servicemix-truezip
> Affects Versions: 3.1
> Reporter: James Bradt
> Assigned To: Guillaume Nodet
> Fix For: 3.2
>
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38746 ]
Guillaume Nodet commented on SM-875:
------------------------------------
Hi James,
I only had a very brief look yesterday, so your explanations are welcome. I do understand much better
how it works now. I don't think there is a way to use a marshaler for such a case (it would be interesting when
polling compressed files containing a single file entry).
It's a pitty that commons-vfs does not handle writing in zips ... but now I understand why you have chosen this
small library.
On the endpoint resolution side, this is not much documented yet.
But the whole point is that the protocol URI is used to automatically find if the component supports it.
So you can try changing the uri in the test to something like:
protected String dynamicURI = "truezip://target/dynamicEndpoint.zip?file.tempFilePrefix=dynamicEp-";
Note the "truezip:" protocol used.
You should be able to debug the component's getResolvedEPR method ...
Also, if you could change the package of the test classes to match the one of the sources
(org.apache.servicemix.truezip instead of org.apache.servicemix.file)
that would be cool.
On the class loader potential problem, we need to dig that a bit more. I guess the problem
happen when several copies of the libraries try to access the same files. This should not be
the case, as the library will be embedded inside the component classloader, and unless you
install several different copies of this component, it should be fine.
Sounds like a good patch, thanks !
Btw, if you envision to become a committer,
you should try to sign and fax a CLA to the ASF.
See http://incubator.apache.org/servicemix/becoming-a-committer.html
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SM-875) TrueZIP binding component
Posted by "James Bradt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Bradt updated SM-875:
---------------------------
Attachment: trueZip-smx.zip
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38760 ]
Guillaume Nodet commented on SM-875:
------------------------------------
I have uploaded truezip in servicemix private repo:
http://servicemix.org/m2-repo/de/schlichtherle/io/truezip/6.5.4/truezip-6.5.4.jar
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Juergen Mayrbaeurl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38778 ]
Juergen Mayrbaeurl commented on SM-875:
---------------------------------------
Unfortunately there's no documentation for the Plexus archiver. Just the sources and some JavaDoc.
I agree with Guillaume that a FileMarshaller extension should only add compress/decompress capabilities, but no archive manipulation. So let's create another JIRA for the FileMarshaller.
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guillaume Nodet updated SM-875:
-------------------------------
Component/s: servicemix-truezip
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Components: servicemix-truezip
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guillaume Nodet resolved SM-875.
--------------------------------
Resolution: Fixed
Fix Version/s: 3.2
Assignee: Guillaume Nodet
URL: http://svn.apache.org/viewvc?view=rev&rev=519081
Btw, I have fixed the dynamic endpoint problem.
Would you mind creating a wiki page to describe this new component ?
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Components: servicemix-truezip
> Affects Versions: 3.1
> Reporter: James Bradt
> Assigned To: Guillaume Nodet
> Fix For: 3.2
>
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38748 ]
Guillaume Nodet commented on SM-875:
------------------------------------
Hopefully Juergen will comment, but I think we were thinking the same, that
the component was just to compresse / decompress a stream.
So I agree with you (and I think we may need another JIRA for handling compressed data
in a FileMarshaler).
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SM-875) TrueZIP binding component
Posted by "Guillaume Nodet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guillaume Nodet updated SM-875:
-------------------------------
Patch Info: [Patch Available]
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SM-875) TrueZIP binding component
Posted by "James Bradt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/SM-875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_38745 ]
James Bradt commented on SM-875:
--------------------------------
Guillaume and Juergen,
It will poll for files within a compressed archive. It will also write (add) new files to a compress archive. Although I havent tried it out, according to the docs, it also supports the normal various compression algo's that are available today.
For polling, if I set the directory to target/myArchive.zip and that zip file contains, for example, 4 xml files within that zip, then it will create a separate message for each of the 4 xml files. If I set my directory to target/myArchive.zip/someSubArchiveDir, it will look for files only in that subdirectory of the archive.
I came across truezip last night as I was looking for functionality to write messages to zip files for archival purposes. It appears that VFS doesnt support writing of zip files. Also, there were some concerns, as was pointed about above, about the cache layer.
I decided to use truezip toolset as it extends the normal java.io.File. It was very easy to implement. Basically, all I needed to do was copy the normal file binder and substitute the File class with TrueZip's File class.
I did see the note about limitations. In my testing so far, I have not encounter any issues. But I am also not an expert in class loaders and will gladly welcome any comments on how we can address this caveat.
I don't know how well I could refactor this into a marshaler. I thought that marshelers are more for modifying the message contents and preparing the message for binding. Right now, the TrueZIP component, as I am using it, works more at the file reading/writing level. I understand your concerns, just don't have the expertise as of yet to know if I can take truezip's compression engine and use it stand alone. I will need to investigate.
As for the dynamic endpoint resolution, the code that I used was a direct copy of the File component so I was expecting to have that logic there already. Can someone, maybe in a dev-list thread or wiki page, explain how dynamic endpoint resolution is process and how to code for it?
Again, I am not an expert in all things java, just a developer that needs to get things done. I appreciate the comments and I am willing to modify the submission. I am very impressed with ServiceMix's capabilities and I want to give back to the community. Also, if there is enough concern that this doesn't fit in well with SM's capabilities, I don't have any problem with throwing it into the bit bucket.
James
> TrueZIP binding component
> -------------------------
>
> Key: SM-875
> URL: https://issues.apache.org/activemq/browse/SM-875
> Project: ServiceMix
> Issue Type: New Feature
> Affects Versions: 3.1
> Reporter: James Bradt
> Attachments: trueZip-smx.zip
>
>
> suggest new binding component using TrueZIP toolset - allows files to be read/written to compressed archives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.