Null query result from oak queryimpl

[Zsolt Grigore <gr...@gmail.com>]

Dear all,

During the process of migrating our codebase to the new jackrabbit
implementation, I found an issue with a relatively complex Xpath
query. I simplified it as much as I can so It should be easy to
reproduce. Unfortunately i can't post the original query but i
replaced the custom namespaces and properties with JCR ones so the
example query looks pretty useless.

//*[(((@jcr:primaryType = 'nt:file') and (jcr:content/@jcr:createdBy =
'bar') and (jcr:content/@jcr:createdBy = 'baz')) or
(jcr:content/@jcr:uuid = '81f5-33deafef08b6'))] order by
jcr:content/@jcr:lastModified descending

In the original query we filter on two types of resources, I replaced
the primarytype with nt:file and for the other type of  resources we
filter on a custom property (replaced with jcr:uuid). We add
additional criterias to each of them but to reproduce the issue it is
enough to add some to the first one so I added two createdBy
constraint (as i found it is important two have the same key here with
an AND otherwise I were not able to repoduce the Null result).

I know that that Xpath has already deprecated but our codebase is
already in production and it contains some really complex generated
queries so before we start the refactoring we would like to know if
this issue is related to our code or is this an issue with the new
Jacrabbit implementation.

During the investigation I found that the problem comes from the SQL2
query which has been generated from XPath. In preparing the execution
of the SQL2 query the underlying logic goes into this
SelectorImpl.pushDown function to examine each and/or branches and
possibly optimize the query but unfortunately my knowledge is not
enough to determine what exactly happens at this point. From the JDocs
i see that it tries to involve some logic to determine if it is
possible to use indexes or not.
As an additional information i can say that this query runs smoothly
on the previous implementation. I tried searching around to get more
information and possibly a fix but no luck so far. What could we do to
fix this?


Caused by: java.lang.UnsupportedOperationException: null
at com.google.common.collect.UnmodifiableIterator.remove(UnmodifiableIterator.java:43)
at java.util.AbstractCollection.retainAll(AbstractCollection.java:406)
at org.apache.jackrabbit.oak.query.ast.AndImpl.getInMap(AndImpl.java:113)
at org.apache.jackrabbit.oak.query.ast.AndImpl.getInMap(AndImpl.java:101)
at org.apache.jackrabbit.oak.query.ast.OrImpl.getInMap(OrImpl.java:108)
at org.apache.jackrabbit.oak.query.ast.OrImpl.restrictPushDownInList(OrImpl.java:166)
at org.apache.jackrabbit.oak.query.ast.OrImpl.restrictPushDown(OrImpl.java:155)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.pushDown(SelectorImpl.java:263)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.prepare(SelectorImpl.java:277)
at org.apache.jackrabbit.oak.query.QueryImpl.prepare(QueryImpl.java:531)
//simple case no join
at org.apache.jackrabbit.oak.query.QueryEngineImpl.executeQuery(QueryEngineImpl.java:167)
at org.apache.jackrabbit.oak.jcr.query.QueryManagerImpl.executeQuery(QueryManagerImpl.java:132)
at org.apache.jackrabbit.oak.jcr.query.QueryImpl$2.perform(QueryImpl.java:101)
at org.apache.jackrabbit.oak.jcr.query.QueryImpl$2.perform(QueryImpl.java:98)
at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:308)
at org.apache.jackrabbit.oak.jcr.query.QueryImpl.execute(QueryImpl.java:97)

Thanks, Zsolt

Re: Null query result from oak queryimpl

[Thomas Mueller <mu...@adobe.com>]

Hi,

This is a bug, I created a new issue: OAK-1933 (with a simpler test case).

It's actually two bugs, the first is trying to remove an entries from an
unmodifiable set, the second problem is that jcr:createdBy could be a
multi-value property, such that it contains both "bar" and "baz". In that
case, the condition "jcr:content/@jcr:createdBy = 'bar') and
(jcr:content/@jcr:createdBy = 'baz'" would actually match the node.
Meaning the condition can't be ignored. With single valued-properties, a
property can't be both "bar" and "baz" at the same time, so it would never
match a node. But the query engine doesn't know whether a property can be
possibly multi-valued or not, so it has to be conservative and keep the
condition.

Regards,
Thomas



On 30/06/14 07:31, "Zsolt Grigore" <gr...@gmail.com> wrote:

>Dear all,
>
>During the process of migrating our codebase to the new jackrabbit
>implementation, I found an issue with a relatively complex Xpath
>query. I simplified it as much as I can so It should be easy to
>reproduce. Unfortunately i can't post the original query but i
>replaced the custom namespaces and properties with JCR ones so the
>example query looks pretty useless.
>
>//*[(((@jcr:primaryType = 'nt:file') and (jcr:content/@jcr:createdBy =
>'bar') and (jcr:content/@jcr:createdBy = 'baz')) or
>(jcr:content/@jcr:uuid = '81f5-33deafef08b6'))] order by
>jcr:content/@jcr:lastModified descending
>
>In the original query we filter on two types of resources, I replaced
>the primarytype with nt:file and for the other type of  resources we
>filter on a custom property (replaced with jcr:uuid). We add
>additional criterias to each of them but to reproduce the issue it is
>enough to add some to the first one so I added two createdBy
>constraint (as i found it is important two have the same key here with
>an AND otherwise I were not able to repoduce the Null result).
>
>I know that that Xpath has already deprecated but our codebase is
>already in production and it contains some really complex generated
>queries so before we start the refactoring we would like to know if
>this issue is related to our code or is this an issue with the new
>Jacrabbit implementation.
>
>During the investigation I found that the problem comes from the SQL2
>query which has been generated from XPath. In preparing the execution
>of the SQL2 query the underlying logic goes into this
>SelectorImpl.pushDown function to examine each and/or branches and
>possibly optimize the query but unfortunately my knowledge is not
>enough to determine what exactly happens at this point. From the JDocs
>i see that it tries to involve some logic to determine if it is
>possible to use indexes or not.
>As an additional information i can say that this query runs smoothly
>on the previous implementation. I tried searching around to get more
>information and possibly a fix but no luck so far. What could we do to
>fix this?
>
>
>Caused by: java.lang.UnsupportedOperationException: null
>at 
>com.google.common.collect.UnmodifiableIterator.remove(UnmodifiableIterator
>.java:43)
>at java.util.AbstractCollection.retainAll(AbstractCollection.java:406)
>at org.apache.jackrabbit.oak.query.ast.AndImpl.getInMap(AndImpl.java:113)
>at org.apache.jackrabbit.oak.query.ast.AndImpl.getInMap(AndImpl.java:101)
>at org.apache.jackrabbit.oak.query.ast.OrImpl.getInMap(OrImpl.java:108)
>at 
>org.apache.jackrabbit.oak.query.ast.OrImpl.restrictPushDownInList(OrImpl.j
>ava:166)
>at 
>org.apache.jackrabbit.oak.query.ast.OrImpl.restrictPushDown(OrImpl.java:15
>5)
>at 
>org.apache.jackrabbit.oak.query.ast.SelectorImpl.pushDown(SelectorImpl.jav
>a:263)
>at 
>org.apache.jackrabbit.oak.query.ast.SelectorImpl.prepare(SelectorImpl.java
>:277)
>at org.apache.jackrabbit.oak.query.QueryImpl.prepare(QueryImpl.java:531)
>//simple case no join
>at 
>org.apache.jackrabbit.oak.query.QueryEngineImpl.executeQuery(QueryEngineIm
>pl.java:167)
>at 
>org.apache.jackrabbit.oak.jcr.query.QueryManagerImpl.executeQuery(QueryMan
>agerImpl.java:132)
>at 
>org.apache.jackrabbit.oak.jcr.query.QueryImpl$2.perform(QueryImpl.java:101
>)
>at 
>org.apache.jackrabbit.oak.jcr.query.QueryImpl$2.perform(QueryImpl.java:98)
>at 
>org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDele
>gate.java:308)
>at 
>org.apache.jackrabbit.oak.jcr.query.QueryImpl.execute(QueryImpl.java:97)
>
>Thanks, Zsolt