You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by "Fraser, Dougal" <Do...@aucklandcouncil.govt.nz> on 2011/03/16 22:52:46 UTC
Large Messages and WS-RM
CXF 2.3.3
I am using WS-RM to send large messages with attachments. The attachments are currently only up to 15 MB, but they could grow to 400 MB or higher.
If I understand the code correctly, both client and sender convert the complete message (including the attachment) to a byte array for processing. This works fine for the smaller messages, but as the messages grow the memory burden of this will become a significant issue.
Is my understanding correct?
Is there any way to stream the data to avoid the memory overhead?
I look forward to your feedback,
Dougal Fraser | Websphere Developer
Integration Stream
Auckland Council
Visit our website: www.aucklandcouncil.govt.nz<http://www.aucklandcouncil.govt.nz>
[Auckland Council]
CAUTION: This email message and any attachments contain information that may be confidential and may be LEGALLY PRIVILEGED. If you are not the intended recipient, any use, disclosure or copying of this message or attachments is strictly prohibited. If you have received this email message in error please notify us immediately and erase all copies of the message and attachments. We do not accept responsibility for any viruses or similar carried with our email, or any effects our email may have on the recipient computer system or network. Any views expressed in this email may be those of the individual sender and may not necessarily reflect the views of Council.
Re: Large Messages and WS-RM
Posted by Dennis Sosnoski <dm...@sosnoski.com>.
On 03/17/2011 10:46 PM, Edumudi Viswanath wrote:
> ...
>
> 1. Reliable messaging assures that messages sent from RM source(client) will reach to RM Destination(server) and vice versa. Right??
Yes to the first part, maybe to the second. The client has the option of
allowing the server to offer a return sequence, used for response
messages sent from the server to the client. Normally the server should
offer the return sequence if the client allows it, but I suppose there
are situations where you wouldn't want to.
> 2. Reliable messaging mechanism comes into picture only when RM-Source(client) sending messages to RM-Destination(server) for processing& while sending response back to client(RM Destination) from server(RM Source) and It has no role, if in case db is down due to some reason or processing of request can't be done due to some issue. Is it right??
The RM Destination is not supposed to acknowledge a message until it has
been delivered to the application code at that end. What happens after
that is up to the application. If the service provider is unable to
function, I'd expect it to return a Fault.
>
> 3. We can test an application which implements ws-rm even from a client which doesn't implement ws-rm (for example: soapUI tool). Right?
As long as the server policy makes WS-RM optional I'd think this should
work. If WS-RM is required, the server should reject any attempts to
communicate without using it.
- Dennis
RE: Large Messages and WS-RM
Posted by Edumudi Viswanath <ed...@oracle.com>.
Hi
Currently I am working on cxf2.3.3 webservice reliable messaging sample.
Plzz find my questions below.
1. Reliable messaging assures that messages sent from RM source(client) will reach to RM Destination(server) and vice versa. Right??
2. Reliable messaging mechanism comes into picture only when RM-Source(client) sending messages to RM-Destination(server) for processing & while sending response back to client(RM Destination) from server(RM Source) and It has no role, if in case db is down due to some reason or processing of request can't be done due to some issue. Is it right??
3. We can test an application which implements ws-rm even from a client which doesn't implement ws-rm (for example: soapUI tool). Right?
Thanks & Regards,
vishy
-----Original Message-----
From: Dennis Sosnoski [mailto:dms@sosnoski.com]
Sent: Thursday, March 17, 2011 4:51 AM
To: users@cxf.apache.org
Cc: Aki Yoshida
Subject: Re: Large Messages and WS-RM
Hi Aki,
There are some deeper issues here around the way messages are persisted.
Right now the message is just captured as sent on the wire. This works
fine for plain messages, but once you add in WS-Security the handling
needs to change so that the security information (timestamps, in
particular) can be updated for each retransmission. That's going to
require intercepting the message at a different point in the processing,
and persisting in a different form.
So by all means go ahead with the fix in CXF-1100, but there are going
to be more changes needed a little down the line.
- Dennis
Dennis M. Sosnoski
Java SOA and Web Services Consulting <http://www.sosnoski.com/consult.html>
Axis2/CXF/Metro SOA and Web Services Training
<http://www.sosnoski.com/training.html>
Web Services Jump-Start <http://www.sosnoski.com/jumpstart.html>
On 03/17/2011 11:56 AM, Aki Yoshida wrote:
> Hi,
> Yes. That is correct. And there is already a JIRA ticket CXF=1100 for
> this limitaiton. I was planning to work on this along the line
> suggested in this ticket. If no one is working on it, I would like to
> look into it next week and work on it.
>
> Regards, aki
>
> 2011/3/16 Fraser, Dougal<Do...@aucklandcouncil.govt.nz>:
>> CXF 2.3.3
>> I am using WS-RM to send large messages with attachments. The attachments are currently only up to 15 MB, but they could grow to 400 MB or higher.
>>
>> If I understand the code correctly, both client and sender convert the complete message (including the attachment) to a byte array for processing. This works fine for the smaller messages, but as the messages grow the memory burden of this will become a significant issue.
>>
>> Is my understanding correct?
>> Is there any way to stream the data to avoid the memory overhead?
>>
>> I look forward to your feedback,
>>
>> Dougal Fraser | Websphere Developer
>> Integration Stream
>> Auckland Council
>> Visit our website: www.aucklandcouncil.govt.nz<http://www.aucklandcouncil.govt.nz>
>>
>> [Auckland Council]
>>
>> CAUTION: This email message and any attachments contain information that may be confidential and may be LEGALLY PRIVILEGED. If you are not the intended recipient, any use, disclosure or copying of this message or attachments is strictly prohibited. If you have received this email message in error please notify us immediately and erase all copies of the message and attachments. We do not accept responsibility for any viruses or similar carried with our email, or any effects our email may have on the recipient computer system or network. Any views expressed in this email may be those of the individual sender and may not necessarily reflect the views of Council.
>>
Re: Large Messages and WS-RM
Posted by Dennis Sosnoski <dm...@sosnoski.com>.
Hi Aki,
There are some deeper issues here around the way messages are persisted.
Right now the message is just captured as sent on the wire. This works
fine for plain messages, but once you add in WS-Security the handling
needs to change so that the security information (timestamps, in
particular) can be updated for each retransmission. That's going to
require intercepting the message at a different point in the processing,
and persisting in a different form.
So by all means go ahead with the fix in CXF-1100, but there are going
to be more changes needed a little down the line.
- Dennis
Dennis M. Sosnoski
Java SOA and Web Services Consulting <http://www.sosnoski.com/consult.html>
Axis2/CXF/Metro SOA and Web Services Training
<http://www.sosnoski.com/training.html>
Web Services Jump-Start <http://www.sosnoski.com/jumpstart.html>
On 03/17/2011 11:56 AM, Aki Yoshida wrote:
> Hi,
> Yes. That is correct. And there is already a JIRA ticket CXF=1100 for
> this limitaiton. I was planning to work on this along the line
> suggested in this ticket. If no one is working on it, I would like to
> look into it next week and work on it.
>
> Regards, aki
>
> 2011/3/16 Fraser, Dougal<Do...@aucklandcouncil.govt.nz>:
>> CXF 2.3.3
>> I am using WS-RM to send large messages with attachments. The attachments are currently only up to 15 MB, but they could grow to 400 MB or higher.
>>
>> If I understand the code correctly, both client and sender convert the complete message (including the attachment) to a byte array for processing. This works fine for the smaller messages, but as the messages grow the memory burden of this will become a significant issue.
>>
>> Is my understanding correct?
>> Is there any way to stream the data to avoid the memory overhead?
>>
>> I look forward to your feedback,
>>
>> Dougal Fraser | Websphere Developer
>> Integration Stream
>> Auckland Council
>> Visit our website: www.aucklandcouncil.govt.nz<http://www.aucklandcouncil.govt.nz>
>>
>> [Auckland Council]
>>
>> CAUTION: This email message and any attachments contain information that may be confidential and may be LEGALLY PRIVILEGED. If you are not the intended recipient, any use, disclosure or copying of this message or attachments is strictly prohibited. If you have received this email message in error please notify us immediately and erase all copies of the message and attachments. We do not accept responsibility for any viruses or similar carried with our email, or any effects our email may have on the recipient computer system or network. Any views expressed in this email may be those of the individual sender and may not necessarily reflect the views of Council.
>>
Re: Large Messages and WS-RM
Posted by Aki Yoshida <el...@googlemail.com>.
Hi,
Yes. That is correct. And there is already a JIRA ticket CXF=1100 for
this limitaiton. I was planning to work on this along the line
suggested in this ticket. If no one is working on it, I would like to
look into it next week and work on it.
Regards, aki
2011/3/16 Fraser, Dougal <Do...@aucklandcouncil.govt.nz>:
> CXF 2.3.3
> I am using WS-RM to send large messages with attachments. The attachments are currently only up to 15 MB, but they could grow to 400 MB or higher.
>
> If I understand the code correctly, both client and sender convert the complete message (including the attachment) to a byte array for processing. This works fine for the smaller messages, but as the messages grow the memory burden of this will become a significant issue.
>
> Is my understanding correct?
> Is there any way to stream the data to avoid the memory overhead?
>
> I look forward to your feedback,
>
> Dougal Fraser | Websphere Developer
> Integration Stream
> Auckland Council
> Visit our website: www.aucklandcouncil.govt.nz<http://www.aucklandcouncil.govt.nz>
>
> [Auckland Council]
>
> CAUTION: This email message and any attachments contain information that may be confidential and may be LEGALLY PRIVILEGED. If you are not the intended recipient, any use, disclosure or copying of this message or attachments is strictly prohibited. If you have received this email message in error please notify us immediately and erase all copies of the message and attachments. We do not accept responsibility for any viruses or similar carried with our email, or any effects our email may have on the recipient computer system or network. Any views expressed in this email may be those of the individual sender and may not necessarily reflect the views of Council.
>