You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-user@axis.apache.org by Manjula Peiris <ma...@wso2.com> on 2008/05/22 11:14:24 UTC
Re: Memory is very high while sending MTOM attachment and also
transfer is slow
Hi Navin,
We are in the process of implementing a caching mechanism for large
attachments [1]. But it is not complete.
[1]https://svn.apache.org/repos/asf/webservices/axis2/branches/c/post_1_4_mtom/c
Thanks,
-Manjula.
On Thu, 2008-05-22 at 10:57 +0530, Navin Bhat wrote:
> Hi Manjula,
>
> Sorry I dint provide complete info in my previous mail.
>
> I am running client on host machine (Windows xp) and the server on
> VMWare (CentOS 4.4).I am using axis2/java client and have set a huge
> timeout. I am able to send and receive 145MB of data with this setting
> as u suggested. But the thing is, still I see that in idle mode server
> consumes around 6MB memory and after sending request it keeps
> increasing. I observed (using system monitor tool) as the control
> entered the business logic of the service memory usage was around 300MB
> and once service started sending back the same data to client memory
> usage went up to 610MB. As expected with such high memory usage my
> computer slowed down. And it took more than 3min to transfer this data
> which is very high as against other framework (gSOAP, Glassfish,
> axis2/java).Is there any alternative approach to transfer large data
> faster with lesser memory usage in this framework.
>
>
> -----Original Message-----
> From: Manjula Peiris [mailto:manjula@wso2.com]
> Sent: Wednesday, May 21, 2008 11:09 PM
> To: Apache AXIS C User List
> Subject: RE: Need information on MTOMChunkBufferSize
> andMTOMMaxChunkBuffers parameters
>
> Hi Navin,
>
> If you are running both client and server on the same machine and both
> send and receive the same 145.1MB attachment then your system memory
> usage can be grown up to at least 145.1 * 4 = 580.1 MB of memory at
> least. So your apache https server memory usage will require 145.1 * 2 =
> 290.2 MB of memory at least for processing the attachment. So an
> apr_pool which is much greater than 290.1MB should be allocated for
> processing this request. Remember Apache frees the memory after the
> request. So it starts swapping and that makes your system to become
> slow. The client fails because it timeouts. If you increase the time out
> using the following function in your client you will definitely see
> better results.
>
> axis2_options_set_timeout_in_milli_seconds(
> axis2_options_t * options,
> const axutil_env_t * env,
> const long timeout_in_milli_seconds);
>
> Actually I have tested a 160MB attachment where both server and client
> sends and receives the same by increasing the time out and it succeed. I
> also tested the case where client only sends the 160MB attachment and
> server does not send any attachment and it succeed with out increasing
> the default time out.
>
> Actually we are in the process of developing caching large attachment
> model for MTOM.
>
> Thanks,
> Manjula.
>
>
>
> On Wed, 2008-05-21 at 15:21 +0530, Navin Bhat wrote:
> > Thank you for the information regarding these parameters.
> >
> > I am seeing a problem with the service I have written where in I
> receive
> > an MTOM attachment from client and the service sends back the same
> > attachment.
> >
> > Problem description:
> > As per my observation apache_http_server uses around 5.5MB of
> > memory in idle state(When it is not serving any request).When I invoke
> > the service to send an attachment of size around 145MB. The memory
> usage
> > of apache_http_server went up and as I observed it went up to 601.1MB
> > and my computer slowed down significantly.I am unable to figure this
> > out.I have worked with other frameworks like
> > axis2/java,gSOAP,Glassfish.I never came across such problem even when
> I
> > transferred 1GB files across.I have specified my computer
> configuration
> > below.I have attached axis2.xml file used while running server. I am
> > using default values MTOMChunkBufferSize is 1 and MTOMMaxChunkBuffers
> is
> > 1000 please refer attached axis2.xml.Kindly let me know as to where I
> > have gone wrong.
> >
> > Computer Specs:
> > CentOS 4.4 on VMWare(host Windows xp)
> > 1GB RAM
> >
> >
> > -----Original Message-----
> > From: Kaushalye Kapuruge [mailto:kaushalye@wso2.com]
> > Sent: Wednesday, May 21, 2008 2:38 PM
> > To: Apache AXIS C User List
> > Subject: Re: Need information on MTOMChunkBufferSize and
> > MTOMMaxChunkBuffers parameters
> >
> > Manjula Peiris wrote:
> > > Hi Navin,
> > >
> > > Axis2/C reads the whole attachment at once. When there is a
> > > content-length this is not a problem.
> > >
> > > But when the incoming stream is chunked we need to read till end of
> > the
> > > chunks. So,
> > >
> > > MTOMChunkBufferSize means the size (In MB) of the buffers we keep on
> > > reading.
> > > MTOMMaxChunkBuffers means the number of such buffers the system can
> > > handle.
> > >
> > > say your ,
> > > MTOMMaxChunkBuffers 1000
> > > MTOMMaxChunkBuffers 1
> > >
> > > Then you can handle Maximum size of 1000*1 MB attachements. So By
> > > changing these parameters you can change the maximum size of
> > attachments
> > > your system can handle given that you have enough memory.
> > >
> > > The default is 1*1000MB. The values in the commented line in
> axis2.xml
> > > are just an example. I will change them to default and add more
> > comments
> > > there.
> > Also I think it's better to update the Axis2/C manual. And add another
>
> > entry to Axis2/C FAQ.
> > Cheers,
> > Kau
> > > Thanks for raising this.
> > >
> > > Thanks,
> > > -Manjula.
> > >
> > >
> > >
> > >
> > >
> > > On Wed, 2008-05-21 at 11:08 +0530, Navin Bhat wrote:
> > >
> > >> Hello group,
> > >>
> > >>
> > >>
> > >> I am new to axis2/c. I am working on axis2-1.4. I have
> a
> > >> service where is need to enable MTOM to send large attachments.
> > >>
> > >> I wanted to know how I can tune the MTOMChunkBufferSize and
> > >> MTOMMaxChunkBuffers parameters to send the attachment efficiently.
> I
> > >> am not getting information about what these parameters imply.
> > >>
> > >>
> > >>
> > >> Does MTOMChunkBufferSize mean size of buffer
> > >> allocated ???
> > >>
> > >> I see in axis2.xml that by default this parameter has
> > >> value 10.
> > >>
> > >> Is it 10 bytes buffer size????
> > >>
> > >>
> > >>
> > >> Does MTOMMaxChunkBuffers mean no.of buffers????
> > >>
> > >> I see in axis2.xml that by default this parameter has
> > >> value 1000.
> > >>
> > >> Is it 1000 buffers???
> > >>
> > >>
> > >>
> > >> I am not finding information about these parameters in axis2/c
> > >> manual.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Need help....Thanks in advance..
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Regards,
> > >>
> > >> Navin Bhat
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> The information contained in this e-mail message and in any
> > >> attachments/annexure/appendices is confidential to the
> > >> recipient and may contain privileged information.
> > >> If you are not the intended recipient, please notify the
> > >> sender and delete the message along with any
> > >> attachments/annexure/appendices. You should not disclose,
> > >> copy or otherwise use the information contained in the
> > >> message or any annexure. Any views expressed in this e-mail
> > >> are those of the individual sender except where the sender
> > >> specifically states them to be the views of
> > >> Toshiba Embedded Software India Pvt. Ltd. (TESI),Bangalore.
> > >>
> > >> Although this transmission and any attachments are believed to be
> > >> free of any virus or other defect that might affect any computer
> > >> system into which it is received and opened, it is the
> responsibility
> > >> of the recipient to ensure that it is virus free and no
> > responsibility
> > >> is accepted by Toshiba Embedded Software India Pvt. Ltd, for any
> loss
> > or
> > >> damage arising in any way from its use.
> > >>
> > >>
> > >>
> > >
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-c-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-c-user-help@ws.apache.org
> > >
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-c-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-c-user-help@ws.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-c-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-c-user-help@ws.apache.org
>
> The information contained in this e-mail message and in any
> attachments/annexure/appendices is confidential to the
> recipient and may contain privileged information.
> If you are not the intended recipient, please notify the
> sender and delete the message along with any
> attachments/annexure/appendices. You should not disclose,
> copy or otherwise use the information contained in the
> message or any annexure. Any views expressed in this e-mail
> are those of the individual sender except where the sender
> specifically states them to be the views of
> Toshiba Embedded Software India Pvt. Ltd. (TESI),Bangalore.
>
> Although this transmission and any attachments are believed to be
> free of any virus or other defect that might affect any computer
> system into which it is received and opened, it is the responsibility
> of the recipient to ensure that it is virus free and no responsibility
> is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or
> damage arising in any way from its use.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-c-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-c-user-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-c-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-c-user-help@ws.apache.org