Re: Olingo2 in-stream errors

[Jonathan Bayle <jo...@gael.fr>]

Hi Christian,

I resend my reply because the formatting was completely messed-up and I 
really want your opinion on our issue.

Because we are serving big files (up to 20GB), copying the payload to an 
internal buffer would delay the start of the download, we are not using 
the ``EntityProvider.writeBinary()`` method, we create an ODataResponse 
that uses an InputStream.

We are using the CXF servlet, but the error message that is outputted 
looks like a genuine OData error message:
<?xml version='1.0' encoding='UTF-8'?><error 
xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><code/><message 
xml:lang="en"/></error>

We are using Olingo v2.0.7, we are not using the JPA nor the annotation 
processor.

This issue only occurs with MediaEntities $value.

This is reproducible, please look at / apply the attached patch that 
modifies the reference service of Olingo v2.0.7.
curl -o out 
"http://localhost:8080/olingo-odata2-ref-web/ReferenceScenarioNonJaxrs.svc/Employees('1')/\$value"
See the XML formatted error message that is appended at the end of the 
binary blob.

My questions are:

  - Is there a better solution to serve binary data without 
``EntityProvider.writeBinary()``?
  - Is there a workaround to avoid the writing of the error document 
in-stream?

Thanks,
Jonathan.


On 21/09/2016 15:10, Amend, Christian wrote:
> Hi Jonathan,
>
> this should actually never happen. Olingo writes the payload to an internal stream first to see if the payload is correct. Afterwards we copy everything from this internal stream to the servlet output stream. So if an exception happens before the internal buffer is filled correctly we throw away the internal buffer and write the exception to a new internal buffer.
> One cause of this could be an exception during the copy of the internal buffer into the servlet output stream. If an exception happens here and you are using the CXF servlet you will end up in the CXF Exception handling which might add new content to the already existing output stream.
> If you are using the CXF Servlet please use the plain Java Servlet instead and see if the issue persists. If this is not the case we have never seen this issue before and I would need you to answer some questions to get to the root cause here:
> Which Olingo version are you using?
> Are you using the JPA or Java Annotation Processor?
> Does this only happen for MediaEntity values?
> Is it reproducible?
> Can you send us a sample to reproduce this issue?
>
> Best Regards,
> Christian

Re: Olingo2 in-stream errors

[mibo <mi...@apache.org>]

Hi Jonathan,

The way how you use the stream in the read media is correct.

The appended error message is the default way Olingo (and OData v2) handles errors which occurs during request processing.
However you can adjust how Olingo handles exceptions by providing an ODataErrorCallback implementation (ODataServiceFactory.getCallback(...)).
In the reference scenario the ScenarioErrorCallback class is used for this.

As fix for your issue you can e.g. check in the error callback for your specific exception and then return an empty error response instead of the default (see changed method below).

Hope this helps you.

Best Regards, Michael


> @Override
> public ODataResponse handleError(final ODataErrorContext context) throws ODataApplicationException {
>   if (context.getHttpStatus() == HttpStatusCodes.INTERNAL_SERVER_ERROR) {
>     LOG.error("Internal Server Error", context.getException());
>   }
> 
>   if(context.getMessage().equals("Fake failure")) {
>     return ODataResponse.entity("").build();
>   }
>   return EntityProvider.writeErrorDocument(context);
> } 




Am 21. Oktober 2016 um 15:59:39, Jonathan Bayle (jonathan.bayle@gael.fr) schrieb:
> Hi Christian,
>  
> I resend my reply because the formatting was completely messed-up and I
> really want your opinion on our issue.
>  
> Because we are serving big files (up to 20GB), copying the payload to an
> internal buffer would delay the start of the download, we are not using
> the ``EntityProvider.writeBinary()`` method, we create an ODataResponse
> that uses an InputStream.
>  
> We are using the CXF servlet, but the error message that is outputted
> looks like a genuine OData error message:
> > xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">> xml:lang="en"/>
>  
> We are using Olingo v2.0.7, we are not using the JPA nor the annotation
> processor.
>  
> This issue only occurs with MediaEntities $value.
>  
> This is reproducible, please look at / apply the attached patch that
> modifies the reference service of Olingo v2.0.7.
> curl -o out
> "http://localhost:8080/olingo-odata2-ref-web/ReferenceScenarioNonJaxrs.svc/Employees('1')/\$value"  
> See the XML formatted error message that is appended at the end of the
> binary blob.
>  
> My questions are:
>  
> - Is there a better solution to serve binary data without
> ``EntityProvider.writeBinary()``?
> - Is there a workaround to avoid the writing of the error document
> in-stream?
>  
> Thanks,
> Jonathan.
>  
>  
> On 21/09/2016 15:10, Amend, Christian wrote:
> > Hi Jonathan,
> >
> > this should actually never happen. Olingo writes the payload to an internal stream  
> first to see if the payload is correct. Afterwards we copy everything from this internal  
> stream to the servlet output stream. So if an exception happens before the internal buffer  
> is filled correctly we throw away the internal buffer and write the exception to a new  
> internal buffer.
> > One cause of this could be an exception during the copy of the internal buffer into the  
> servlet output stream. If an exception happens here and you are using the CXF servlet  
> you will end up in the CXF Exception handling which might add new content to the already  
> existing output stream.
> > If you are using the CXF Servlet please use the plain Java Servlet instead and see if the  
> issue persists. If this is not the case we have never seen this issue before and I would  
> need you to answer some questions to get to the root cause here:
> > Which Olingo version are you using?
> > Are you using the JPA or Java Annotation Processor?
> > Does this only happen for MediaEntity values?
> > Is it reproducible?
> > Can you send us a sample to reproduce this issue?
> >
> > Best Regards,
> > Christian
>