Olingo Odata v4.0: stack overflow error while parsing very long URI expression

[Erwan LETESSIER <Er...@ingenico.com>]

Hi
I have encountered stack overflow error while parsing very long URI expression.
I’m working server side with version 4.0.0.
Of course the client side always calls using POST / $batch option.
Here is what the URI looks like:

-          $filter=storeId eq 123 or storeId eq 456 …. and so on …  was working fine so far, but now with 2000 storeIds  the tree the parser build become too big => stackoverflow!

Furthermore, it looks like the memory is not well freed afterwards, because on 2 consecutive calls, the latter will stackoverflow.

-          Parsed data goes into a List<Integer>  that’s used to feed an SQL query: storeId IN (?, ?, ?, …)   and even JOIN TABLE(#sqlArray) because of the 1000 values IN clause limit.

-          The workaround I’ve just implemented consists in passing single value instead (a comma separated joined list of integers) such as: $filter=storeId eq '123,456'

-          Validating whole expression against regex = "\\d+(,\\d+)*"  logically also stackoverflows.

-          Increasing the server’s memory does not seem to be the answer since the queries might also be bigger and resources are shared in production environment.

So Is there any cleaner way to achieve this?
A more relevant BinaryOperatorKind to use?
Something in MethodKind ?
Anything new in version 4.1.0 or 4.2.0 that would help?

Thanks.
Regards


Erwan LETESSIER
28/32 boulevard de Grenelle
75015 Paris – France.
erwan.letessier@ingenico.com<ma...@ingenico.com>
Tel : +33 (0)1 58 01 90 35
Fax : +33 (0)1 58 01 91 35
http://www.ingenico.com
[cid:image002.png@01CF76B6.DE24C1B0]

This email and its content belong to Ingenico Group. The enclosed information is confidential and may not be disclosed to any unauthorized person. If you have received it by mistake do not forward it and delete it from your system. Cet email et son contenu sont la propriété du Groupe Ingenico. L’information qu’il contient est confidentielle et ne peut être communiquée à des personnes non autorisées. Si vous l’avez reçu par erreur ne le transférez pas et supprimez-le.

Re: Olingo Odata v4.0: stack overflow error while parsing very long URI expression

[Ramesh Reddy <ra...@redhat.com>]

Erwan, 

- Try 4.2.0, it has newer parsing framework might work. 
- Since your client is always batch, why not split your query into multiple requests. 
- OData TC is also considering IN function in future versions of specifications, that might reduce size of the URL. 

SQL limit to 1000 IN values, do not apply to Olingo, but that is a separate concern, specific to the JDBC driver and DBMS in question. Typically DBMS have setting to increase it. 

Ramesh.. 
http://teiid.org 

----- Original Message -----

> Hi

> I have encountered stack overflow error while parsing very long URI
> expression.

> I’m working server side with version 4.0.0.

> Of course the client side always calls using POST / $batch option.

> Here is what the URI looks like:

> - $filter=storeId eq 123 or storeId eq 456 …. and so on … was working fine so
> far, but now with 2000 storeIds the tree the parser build become too big =>
> stackoverflow!

> Furthermore, it looks like the memory is not well freed afterwards, because
> on 2 consecutive calls, the latter will stackoverflow.

> - Parsed data goes into a List<Integer> that’s used to feed an SQL query:
> storeId IN (?, ?, ?, …) and even JOIN TABLE(#sqlArray) because of the 1000
> values IN clause limit.

> - The workaround I’ve just implemented consists in passing single value
> instead (a comma separated joined list of integers) such as: $filter=storeId
> eq '123,456'

> - Validating whole expression against regex = "\\d+(,\\d+)*" logically also
> stackoverflows.

> - Increasing the server’s memory does not seem to be the answer since the
> queries might also be bigger and resources are shared in production
> environment.

> So Is there any cleaner way to achieve this?

> A more relevant BinaryOperatorKind to use?

> Something in MethodKind ?

> Anything new in version 4.1.0 or 4.2.0 that would help?

> Thanks.

> Regards

> Erwan LETESSIER

> 28/32 boulevard de Grenelle

> 75015 Paris – France.

> erwan.letessier@ingenico.com

> Tel : +33 (0)1 58 01 90 35

> Fax : +33 (0)1 58 01 91 35

> http://www.ingenico.com

> This email and its content belong to Ingenico Group. The enclosed information
> is confidential and may not be disclosed to any unauthorized person. If you
> have received it by mistake do not forward it and delete it from your
> system. Cet email et son contenu sont la propriété du Groupe Ingenico.
> L’information qu’il contient est confidentielle et ne peut être communiquée
> à des personnes non autorisées. Si vous l’avez reçu par erreur ne le
> transférez pas et supprimez-le.