Re: JAX-RS Attachments size limit and temp folder

[Sergey Beryozkin <sb...@gmail.com>]

Hi,

>
> http://download.oracle.com/javase/6/docs/api/javax/xml/bind/util/JAXBSource.html
>
> However I'm not sure if returning a JAXBSource will make writing large
> pieces of data more efficient.
>
>
>
>> Anyway, let me know as soon as you optimize the SourceProvider :)
>
> I will :-)

Not sure if it will help in your case, but I 'borrowed' few lines of
code from the CXF SourceDataBinding for the SourceProvider be able to
read more efficiently. But now even JAXBElementProvider converts an
InputStream to XMLStreamReader... So if you are using CXF on the
client side then may be it will help...

cheers, Sergey

>
> Cheers, Sergey
>>
>> Ivan
>>
>> -----Mensaje original-----
>> De: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Enviado el: jueves, 20 de enero de 2011 12:24
>> Para: users@cxf.apache.org
>> Asunto: Re: JAX-RS Attachments size limit and temp folder
>>
>> Hi Ivan
>>
>> On Thu, Jan 20, 2011 at 10:24 AM, Ivan Vitoria Sanchez <
>> ivitoria@grupoica.com> wrote:
>>
>> > Hi Segey,
>> >
>> > Please see my comments below
>> >
>> > (I really appreciate your effort)
>> >
>> > Ivan
>> >
>>
>>
>> >
>> > > Does it mean that i can not use these jax-rs properties to get a
>> > generated
>> > > JAXB XML with a Base64-encoded file via GET request?
>> >
>> >
>> > You may need to experiment with different options here.
>> >
>> > 1. If you return a JAXB bean with the DataHandler then it looks like it
>> can
>> > definitely be XOP-packaged.
>> > Just set an "mtom-enabled" JAX-RS property to "true" and have
>> > @Produces("multipart/related").
>> > Or is the consumer not capable of reading multiparts, is it a browser ?
>> >
>> > - Well, the client, which i'm developing too, is a Windows Mobile
>> > application, so I think XOP isn't supported out-of-the-box by Compact
>> > Framework 3.5 and I should develop a custom parser...
>> >
>> >
>> I see...
>>
>>
>> > 2. Return the big XML wrapped in the JAXP Source - it probably won't
>> > make
>> a
>> > difference right now; Dan explained to me that the JAX-WS
>> > Provider<Source>
>> > is optimized on the write side, so I will look into it and enhance the
>> > JAX-RS SourceProvider accordingly.
>> >
>> > - isn't this like my scene? (do you remember the CXF-3022 bug?). what
>> > kind
>> > of improvement would it be if you enhance the JAX-RS SourceProvider?
>> >
>> > In order to have the SourceProvider involved the method will need to
>> > have
>> a
>> Source return type.
>> I do not know right now. As far as I recall that if JAXP Source is
>> returned
>> (as opposed to say StreamSource or DOMSource) then the JAX-WS runtime can
>> opt for using a CXF StaxSource which makes it easier to stream, so the
>> SourceProvider can do something similar. You may just want to experiment
>> and
>> explicitly return the CXF StaxSource,
>>
>> new StaxSource(StaxUtils.createXMLStreamReader(xmlInputStream));
>>
>>
>> > 3. Return a link to the XML file on the disk and delegate to the
>> underlying
>> > container to serve the file to the consumer, you can configure the
>> > CXFServlet to delegate to the default servlet for doing it...
>> >
>> > - I'm not sure if this is a good solution for me
>> >
>> > 4. Return several links to individual XML sections (kept in memory or on
>> > the
>> > disk) - this could be one of the most efficient options for writing the
>> > large attachments
>> >
>> > - Are you talking about splitting up a Base64 Encoded XML attachment?
>> >
>> >
>> 3 and 4 are attempts to come with some alternative options. It all depends
>> on what sort of experience is provided to the users. I'm not sure right
>> now
>> what I'd do if I were to return a 50MB file. But I'd try to avoid writing
>> it
>> directly to the output stream representing the response to the current
>> invocation. What those XML data represent, the list of records ? For
>> example, I'd consider returning a fist section of the xml with the
>> attribute
>> containing a link to the next one, so that you can display the current
>> records and offer the option to browse to the next section. If this xml
>> blob
>> is not splittable then may be I'd consider using a
>> RequestDispatcherProvider
>> to redirect to the container which can be more optimized to serve large
>> files.
>>
>> But I'll definitely look into optimizing the SourceProvider...
>>
>> cheers, Sergey
>>
>> > I'd really like to
>> > > customize both input and output folders/sizes of attachments extracted
>> > > from/attached to XML messages.
>> > >
>> > >
>> > My understanding that at the moment it's only possible to do it on the
>> > input
>> > (assuming it is XOP in your case or 'plain' attachments).
>> >
>> > Let me know how it goes please
>> > Cheers, Sergey
>> >
>> >
>> > I've added the system properties and I've checked from code. However,
>> > the
>> > > temporal folder is still the same
>> > >
>> > >
>> >
>> >
>>
>> (C:\Users\ICA\.netbeans\6.9\apache-tomcat-6.0.26_base\temp\cxf-tmp-418517\co
>> > > s970506463177488784tmp) and the size of the XML message is 15 MB!
>> > >
>> > > Thanks!
>> > >
>> > >
>> > > -----Mensaje original-----
>> > > De: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> > > Enviado el: miércoles, 19 de enero de 2011 13:09
>> > > Para: users@cxf.apache.org
>> > > Asunto: Re: JAX-RS Attachments size limit and temp folder
>> > >
>> > > Hi Ivan
>> > >
>> > > Please see comments below:
>> > >
>> > > On Wed, Jan 19, 2011 at 11:34 AM, Ivan Vitoria Sanchez <
>> > > ivitoria@grupoica.com> wrote:
>> > >
>> > > > Hi Sergey,
>> > > >
>> > > > I'm not using XOP and yes, the properties 'attachment-directory' and
>> > > > 'attachment-memory-threshold' don't seem to have any effect. I've
>> > > > also
>> > > > tried
>> > > > adding the VM options but the result is the same.
>> > > >
>> > > > Following you can find the Spring configuration and log output:
>> > > >
>> > > > <jaxrs:server id="genericServer"
>> > > > modelRef="classpath:/WEB-INF/model/GenericModel.xml"
>> > > > abstract="true">
>> > > >        <jaxrs:features>
>> > > >            <cxf:logging/>
>> > > >        </jaxrs:features>
>> > > >        <jaxrs:inInterceptors>
>> > > >            <ref bean="gzipInInterceptor" />
>> > > >        </jaxrs:inInterceptors>
>> > > >        <jaxrs:outInterceptors>
>> > > >            <ref bean="gzipOutInterceptor" />
>> > > >        </jaxrs:outInterceptors>
>> > > >        <jaxrs:properties>
>> > > >            <entry key="attachment-directory"
>> > > > value="/temp/mobility"/>
>> > > >            <!-- 400K-->
>> > > >            <entry key="attachment-memory-threshold" value="409600"/>
>> > > >        </jaxrs:properties>
>> > > >    </jaxrs:server>
>> > > >
>> > > >
>> > > >    <bean id="attachmentServer" parent="genericServer">
>> > > >        <property name="modelRef"
>> > > > value="classpath:/WEB-INF/model/AttachmentModel.xml" />
>> > > >        <property name="address" value="/attachments" />
>> > > >        <property name="serviceBeans">
>> > > >            <!-- Service Bean omitted -->
>> > > >        </property>
>> > > >    </bean>
>> > > >
>> > > > ---------------
>> > > > Response-Code: 200
>> > > > Content-Type: application/octet-stream
>> > > > Headers: {Date=[Wed, 19 Jan 2011 10:20:55 GMT],
>> > Content-Encoding=[gzip],
>> > > > Vary=[Accept-Encoding]}
>> > > > Messages: Outbound Message (saved to tmp file):
>> > > > Filename:
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>> C:\Users\ICA\.netbeans\6.9\apache-tomcat-6.0.26_base\temp\cxf-tmp-872111\cos
>> > > > 4854851171655688610tmp
>> > > > (message truncated to 102400 bytes)
>> > > >
>> > > > Payload: omitted
>> > > >
>> > > >
>> > > >
>> > > AFAIK, These properties, "attachment-directory" and
>> > > "attachment-memory-threshold", are affecting reading the attachments
>> > only.
>> > > But it looks like you'd like them affecting writing the attachments ?
>> > >
>> > > Try setting the following system properties :
>> > >
>> > > -Dorg.apache.cxf.io.CachedOutputStream.Threshold=someValue
>> > > and
>> > > -Dorg.apache.cxf.io.CachedOutputStream.OutputDirectory=/temp/mobility
>> > >
>> > > There must be the way to set these properties as jaxrs:properties too,
>> > just
>> > > do not have an access to the code right now
>> > >
>> > > Let me know please if it helps
>> > > Cheers, Sergey
>> > >
>> > >
>> > > > Ivan
>> > > >
>> > > >
>> > > >
>> > > > -----Mensaje original-----
>> > > > De: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> > > > Enviado el: miércoles, 19 de enero de 2011 12:06
>> > > > Para: users@cxf.apache.org
>> > > > Asunto: Re: JAX-RS Attachments size limit and temp folder
>> > > >
>> > > > Hi
>> > > >
>> > > > On Wed, Jan 19, 2011 at 10:56 AM, Ivan Vitoria Sanchez <
>> > > > ivitoria@grupoica.com> wrote:
>> > > >
>> > > > > Hi everyone,
>> > > > >
>> > > > >
>> > > > >
>> > > > > I'm developing a JAX-RS server (CXF 2.3.1) which performs both GET
>> > and
>> > > > POST
>> > > > > operations with DataHandler attachments within a bean, and it
>> > > > > works
>> > > fine
>> > > > > (I'm not sending Multiparts). However, I could not set neither the
>> > temp
>> > > > > folder used as buffer nor the max size of the attachments as
>> > explained
>> > > in
>> > > > > http://cxf.apache.org/docs/jax-rs-multiparts.html ("Reading Large
>> > > > > Attachments" section).
>> > > > >
>> > > > >
>> > > > Are you using XOP ? Also, when you say you can not set, do you mean
>> the
>> > > > properties you set have no effect ?
>> > > >
>> > > > thanks, Sergey
>> > > >
>> > > >
>> > > > >
>> > > > >
>> > > > > Does it not work because it only applies to Multipart messages? If
>> > so,
>> > > > how
>> > > > > can i do it?
>> > > > >
>> > > > >
>> > > > >
>> > > > > Thanks in advance!
>> > > > >
>> > > > >
>> > > > >
>> > > > > Ivan
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>
>