You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chemistry.apache.org by Uwe Geisert <ug...@opentext.com> on 2016/01/04 16:14:53 UTC
RE: Streaming capability with WS binding
Hi,
Happy New Year everybody!
Quite a while ago we found that in OpenCMIS 0.10.0 the parseEagerly=true in StreamingAttachment caused content to be buffered in tomcat's temp directory (or in memory). See below.
This was fixed in OpenCMIS 0.11.0.
There is a client-side counterpart in chemistry-opencmis-client-bindings/src/main/java/org/apache/chemistry/opencmis/client/bindings/spi/webservices/SunRIPortProvider.java, line 65: a StreamingAttachmentFeature is created with parseEagerly=true. This causes MIME*.tmp files to be created in %TEMP% for every sufficiently big document that is downloaded.
Would that be something that could be changed or made customizable in a future OpenCMIS version?
Thanks
Uwe
-----Original Message-----
From: Florian Müller [mailto:fmui@apache.org]
Sent: Dienstag, 19. November 2013 12:22
To: Uwe Geisert
Cc: dev@chemistry.apache.org
Subject: RE: Streaming capability with WS binding
Hi Uwe,
The current code base (in trunk) does not do eager parsing.
- Florian
> Hi Florian,
>
> I investigated the streaming issue a bit and found that
> "parseEagerly=true" in the JAX-WS annotation
>
> @StreamingAttachment(parseEagerly = true, memoryThreshold = 4 * 1024
> * 1204)
>
> which is applied to OpenCMIS' ObjectService and VersioningService
> breaks streaming.
> Setting it to false has the following effect:
> - no MIME*.temp files in Tomcat's temp directory
> - the content stream's read() is called only a few times before the
> service is invoked (whereas with parseEagerly=true the complete
> content was read before the service's invokation and before all
> handler invokations)
> - in a small dev environment no negative performance impact was
> observed
>
> Would you please explain what was the intention to user eager parsing?
> Would it make sense to make this a configurable parameter as the
> memoryThreshold already is?
>
> Thanks
> Uwe
>
>
>
> -----Original Message-----
> From: Florian Müller [mailto:fmui@apache.org]
> Sent: Mittwoch, 23. Oktober 2013 19:36
> To: dev@chemistry.apache.org
> Cc: Uwe Geisert
> Subject: Re: Streaming capability with WS binding
>
> Hi Uwe,
>
> The abilities to stream, buffer in memory, buffer on disk, encrypt,
> and dealing with attachments in general are implementation details of
> the JAX-WS framework and not covered by the JAX-WS specification.
>
> OpenCMIS supports streaming if the Sun JAX-WS RI is used. The
> WSConverter checks for each stream if "read once" is available and use
> it, if it is available.
> But it is really easy to break. For example, putting a SOAP Handler
> in front of the service can force the JAX-WS implementation to read
> the whole content into memory.
>
> OpenCMIS does not use mimepull directly. It's a dependency of the Sun
> JAX-WS RI. We can't influence how the Sun JAX-WS RI uses it and when
> it uses the feature you have mentioned.
>
> The only way to get full control is to develop our own Web Services
> stack and I doubt that we will do that.
>
>
> - Florian
>
>
>> Hi,
>>
>> With Webservice-binding uploaded content gets loaded completely on
>> the server (either in memory or on disk, depending on the size, as
>> specified in the StreamingAttachment's memoryThreshold).
>> This issue has been discussed before in regards to encryption and
>> configurability of the threshold.
>> But I wonder why content must be buffered at all.
>>
>> According to https://mimepull.java.net, streaming should be possible
>> if
>> - "The parts are accessed in the same order as they appear in the
>> stream"
>> - "The parts are accessed only once."
>> I believe the read-once condition is complied with by OpenCMIS
>> (org.apache.chemistry.opencmis.commons.impl.WSConverter).
>>
>> So why is streaming not possible with OpenCMIS?
>>
>> Thanks for a short explanation (or even a solution ;-) Uwe
Re: Streaming capability with WS binding
Posted by Florian Müller <fm...@apache.org>.
Hi Uwe,
Please open an improvment issue for this.
OpenCMIS 0.14.0 switches the Web Service framework from the JAX-WS RI to
Apache CXF. It should be possible to add this to the new implementation.
- Florian
On 04.01.2016 16:14, Uwe Geisert wrote:
> Hi,
>
> Happy New Year everybody!
>
> Quite a while ago we found that in OpenCMIS 0.10.0 the parseEagerly=true in StreamingAttachment caused content to be buffered in tomcat's temp directory (or in memory). See below.
>
> This was fixed in OpenCMIS 0.11.0.
>
> There is a client-side counterpart in chemistry-opencmis-client-bindings/src/main/java/org/apache/chemistry/opencmis/client/bindings/spi/webservices/SunRIPortProvider.java, line 65: a StreamingAttachmentFeature is created with parseEagerly=true. This causes MIME*.tmp files to be created in %TEMP% for every sufficiently big document that is downloaded.
>
> Would that be something that could be changed or made customizable in a future OpenCMIS version?
>
> Thanks
> Uwe
>
> -----Original Message-----
> From: Florian Müller [mailto:fmui@apache.org]
> Sent: Dienstag, 19. November 2013 12:22
> To: Uwe Geisert
> Cc: dev@chemistry.apache.org
> Subject: RE: Streaming capability with WS binding
>
> Hi Uwe,
>
> The current code base (in trunk) does not do eager parsing.
>
>
> - Florian
>
>
>> Hi Florian,
>>
>> I investigated the streaming issue a bit and found that
>> "parseEagerly=true" in the JAX-WS annotation
>>
>> @StreamingAttachment(parseEagerly = true, memoryThreshold = 4 * 1024
>> * 1204)
>>
>> which is applied to OpenCMIS' ObjectService and VersioningService
>> breaks streaming.
>> Setting it to false has the following effect:
>> - no MIME*.temp files in Tomcat's temp directory
>> - the content stream's read() is called only a few times before the
>> service is invoked (whereas with parseEagerly=true the complete
>> content was read before the service's invokation and before all
>> handler invokations)
>> - in a small dev environment no negative performance impact was
>> observed
>>
>> Would you please explain what was the intention to user eager parsing?
>> Would it make sense to make this a configurable parameter as the
>> memoryThreshold already is?
>>
>> Thanks
>> Uwe
>>
>>
>>
>> -----Original Message-----
>> From: Florian Müller [mailto:fmui@apache.org]
>> Sent: Mittwoch, 23. Oktober 2013 19:36
>> To: dev@chemistry.apache.org
>> Cc: Uwe Geisert
>> Subject: Re: Streaming capability with WS binding
>>
>> Hi Uwe,
>>
>> The abilities to stream, buffer in memory, buffer on disk, encrypt,
>> and dealing with attachments in general are implementation details of
>> the JAX-WS framework and not covered by the JAX-WS specification.
>>
>> OpenCMIS supports streaming if the Sun JAX-WS RI is used. The
>> WSConverter checks for each stream if "read once" is available and use
>> it, if it is available.
>> But it is really easy to break. For example, putting a SOAP Handler
>> in front of the service can force the JAX-WS implementation to read
>> the whole content into memory.
>>
>> OpenCMIS does not use mimepull directly. It's a dependency of the Sun
>> JAX-WS RI. We can't influence how the Sun JAX-WS RI uses it and when
>> it uses the feature you have mentioned.
>>
>> The only way to get full control is to develop our own Web Services
>> stack and I doubt that we will do that.
>>
>>
>> - Florian
>>
>>
>>> Hi,
>>>
>>> With Webservice-binding uploaded content gets loaded completely on
>>> the server (either in memory or on disk, depending on the size, as
>>> specified in the StreamingAttachment's memoryThreshold).
>>> This issue has been discussed before in regards to encryption and
>>> configurability of the threshold.
>>> But I wonder why content must be buffered at all.
>>>
>>> According to https://mimepull.java.net, streaming should be possible
>>> if
>>> - "The parts are accessed in the same order as they appear in the
>>> stream"
>>> - "The parts are accessed only once."
>>> I believe the read-once condition is complied with by OpenCMIS
>>> (org.apache.chemistry.opencmis.commons.impl.WSConverter).
>>>
>>> So why is streaming not possible with OpenCMIS?
>>>
>>> Thanks for a short explanation (or even a solution ;-) Uwe