ORDER BY

[Tobias Bocanegra <tr...@apache.org>]

Hi,

what would be needed to implement a subset of OAK-1263, so that it
supports OrderBy queries for well defined property types?

Thomas suggested that we mandate that all index keys are mandated,
also for the NodeStore.
WDYT?

regards, toby

Re: ORDER BY

[Tobias Bocanegra <tr...@apache.org>]

On Thu, Dec 12, 2013 at 9:41 AM, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> On Thu, Dec 12, 2013 at 12:13 PM, Tobias Bocanegra <tr...@apache.org> wrote:
>> what would be needed to implement a subset of OAK-1263, so that it
>> supports OrderBy queries for well defined property types?
>
> First we'd have to pass the ORDER BY list to the QueryIndex.query()
> method, as otherwise the index lookup can't possibly order the results
> properly. This could be done either by including that information in
> the Filter instance or (my preference) by passing the full AST instead
> of just the pre-processed Filter to the index lookup.
>
> Once that information is available to the index lookup, we'd need a
> method like Cursor.isSorted() so that the index can tell the query
> engine whether the result set is already sorted. It should still be
> possible for an index to return unsorted results that would then get
> sorted by the query engine as a post-processing step.
>
> Finally, we need an index implementation that can efficiently sort the
> result set. The Lucene index should be easy to adapt to support this.
> The property index could be extended to include optional sort order
> configuration, that it would then use to organize the index content
> structure to make efficient sorting of query results possible.
sounds like a plan :-(

>
>> Thomas suggested that we mandate that all index keys are mandated,
>> also for the NodeStore.
>
> I don't understand what you're saying here: "mandated" what? :-)

This is a typo. it should read:
  Thomas suggested that we mandate that all index keys are sorted,
also for the NodeStore.

The conversation went like this:

Toby: I wonder how the index works. aren't the indexed properties
sorted anyways? or is the index a hashtable based one?
Tom: do you mean MongoDB? yes, MongoDB does this. but the NodeStore
contract doesn't; I don't know if the SegmentNS sorts. This would be
the simplest solution: to change the Node-Store contract.

but maybe we didn't talk about the same :-) I frankly don't know
enough about how the property index is implemented.
regards, toby

Re: ORDER BY

[Jukka Zitting <ju...@gmail.com>]

Hi,

On Thu, Dec 12, 2013 at 12:13 PM, Tobias Bocanegra <tr...@apache.org> wrote:
> what would be needed to implement a subset of OAK-1263, so that it
> supports OrderBy queries for well defined property types?

First we'd have to pass the ORDER BY list to the QueryIndex.query()
method, as otherwise the index lookup can't possibly order the results
properly. This could be done either by including that information in
the Filter instance or (my preference) by passing the full AST instead
of just the pre-processed Filter to the index lookup.

Once that information is available to the index lookup, we'd need a
method like Cursor.isSorted() so that the index can tell the query
engine whether the result set is already sorted. It should still be
possible for an index to return unsorted results that would then get
sorted by the query engine as a post-processing step.

Finally, we need an index implementation that can efficiently sort the
result set. The Lucene index should be easy to adapt to support this.
The property index could be extended to include optional sort order
configuration, that it would then use to organize the index content
structure to make efficient sorting of query results possible.

> Thomas suggested that we mandate that all index keys are mandated,
> also for the NodeStore.

I don't understand what you're saying here: "mandated" what? :-)

BR,

Jukka Zitting