You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by "Thomas Mueller (Created) (JIRA)" <ji...@apache.org> on 2012/03/16 16:25:39 UTC
[jira] [Created] (OAK-28) Query implementation
Query implementation
--------------------
Key: OAK-28
URL: https://issues.apache.org/jira/browse/OAK-28
Project: Jackrabbit Oak
Issue Type: New Feature
Components: core
Reporter: Thomas Mueller
Assignee: Thomas Mueller
A query engine needs to be implemented.
This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
If somebody has a better idea please tell me :-)
Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
import javax.jcr.query.qom.And;
import javax.jcr.query.qom.Constraint;
public class OakToJcrQueryTreeConverter {
public static void main(String... args) {
OakAnd oak = new OakAnd();
oak.constraint1 = new OakConstraint();
oak.constraint1.s = "x=1";
oak.constraint2 = new OakConstraint();
oak.constraint2.s = "y=1";
System.out.println("Oak constraint: " + oak);
JcrConstraint jcr = convertOakToJcr(oak);
System.out.println("JCR constraint: " + jcr);
}
static JcrConstraint convertOakToJcr(OakConstraint constraint) {
if (constraint instanceof OakAnd) {
OakAnd o = (OakAnd) constraint;
return new JcrAnd(o.constraint1, o.constraint2);
} else {
JcrConstraint c = new JcrConstraint();
c.s = constraint.s;
return c;
}
}
}
class OakConstraint {
String s;
public String toString() {
return "(OakConstraint) " + s;
}
}
class OakAnd extends OakConstraint {
public OakConstraint constraint1, constraint2;
public String toString() {
return constraint1 + " and " + constraint2;
}
}
class JcrConstraint implements Constraint {
String s;
public String toString() {
return "(JcrConstraint) " + s;
}
}
class JcrAnd extends JcrConstraint implements And {
JcrConstraint constraint1, constraint2;
JcrAnd(OakConstraint constraint1, OakConstraint constraint2) {
this.constraint1 = OakToJcrQueryTreeConverter.convertOakToJcr(constraint1);
this.constraint2 = OakToJcrQueryTreeConverter.convertOakToJcr(constraint2);
}
public JcrConstraint getConstraint1() {
return constraint1;
}
public JcrConstraint getConstraint2() {
return constraint2;
}
public String toString() {
return constraint1 + " and " + constraint2;
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234515#comment-13234515 ]
Marcel Reutegger commented on OAK-28:
-------------------------------------
> query string
see also the executeQuery method in the Jackrabbit 2.4 SPI:
http://jackrabbit.apache.org/api/2.4/org/apache/jackrabbit/spi/RepositoryService.html#executeQuery%28org.apache.jackrabbit.spi.SessionInfo,%20java.lang.String,%20java.lang.String,%20java.util.Map,%20long,%20long,%20java.util.Map%29
It also passes limit and offset because JCR 2.0 Query has setLimit() and setOffset(). Furthermore it passes a map of bound values, but I'm not sure if that's really necessary. I'd say the oak-jcr client could just as well put the values as literals into the serialized query statement.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Michael Dürig (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234526#comment-13234526 ]
Michael Dürig commented on OAK-28:
----------------------------------
Ok agreed. Why do we need to implement a query parser in the jcr bindings module then? Isn't passing the query string along with the name space mappings and the query language descriptor down to the oak-core sufficient?
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445785#comment-13445785 ]
Thomas Mueller commented on OAK-28:
-----------------------------------
Thanks Chetan! Fixed in revision 1379373.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with value bindings, limit, offset, and name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and offset
> * Implement a query builder in the jcr component which takes care of translating JQOM queries to statements in string form
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OAK-28) Query implementation
Posted by "Michael Dürig (Updated JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Dürig updated OAK-28:
-----------------------------
Component/s: jcr
Description:
A query engine needs to be implemented.
A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with name space mappings to the oak-core.
We need to:
* Define the oak-core API for handling queries. How are do we handle name space mappings, limit and size
* Implement a query builder in the jcr component which takes care of translating JQOM queries
* Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
* Implement the actual query execution engine which interprets the query AST
was:
A query engine needs to be implemented.
This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
If somebody has a better idea please tell me :-)
Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and size
> * Implement a query builder in the jcr component which takes care of translating JQOM queries
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OAK-28) Query implementation
Posted by "Thomas Mueller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated OAK-28:
------------------------------
Description:
A query engine needs to be implemented.
This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
If somebody has a better idea please tell me :-)
Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
was:
A query engine needs to be implemented.
This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
If somebody has a better idea please tell me :-)
Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
import javax.jcr.query.qom.And;
import javax.jcr.query.qom.Constraint;
public class OakToJcrQueryTreeConverter {
public static void main(String... args) {
OakAnd oak = new OakAnd();
oak.constraint1 = new OakConstraint();
oak.constraint1.s = "x=1";
oak.constraint2 = new OakConstraint();
oak.constraint2.s = "y=1";
System.out.println("Oak constraint: " + oak);
JcrConstraint jcr = convertOakToJcr(oak);
System.out.println("JCR constraint: " + jcr);
}
static JcrConstraint convertOakToJcr(OakConstraint constraint) {
if (constraint instanceof OakAnd) {
OakAnd o = (OakAnd) constraint;
return new JcrAnd(o.constraint1, o.constraint2);
} else {
JcrConstraint c = new JcrConstraint();
c.s = constraint.s;
return c;
}
}
}
class OakConstraint {
String s;
public String toString() {
return "(OakConstraint) " + s;
}
}
class OakAnd extends OakConstraint {
public OakConstraint constraint1, constraint2;
public String toString() {
return constraint1 + " and " + constraint2;
}
}
class JcrConstraint implements Constraint {
String s;
public String toString() {
return "(JcrConstraint) " + s;
}
}
class JcrAnd extends JcrConstraint implements And {
JcrConstraint constraint1, constraint2;
JcrAnd(OakConstraint constraint1, OakConstraint constraint2) {
this.constraint1 = OakToJcrQueryTreeConverter.convertOakToJcr(constraint1);
this.constraint2 = OakToJcrQueryTreeConverter.convertOakToJcr(constraint2);
}
public JcrConstraint getConstraint1() {
return constraint1;
}
public JcrConstraint getConstraint2() {
return constraint2;
}
public String toString() {
return constraint1 + " and " + constraint2;
}
}
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OAK-28) Query implementation
Posted by "Thomas Mueller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated OAK-28:
------------------------------
Description:
A query engine needs to be implemented.
A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with value bindings, limit, offset, and name space mappings to the oak-core.
We need to:
* Define the oak-core API for handling queries. How are do we handle name space mappings, limit and offset
* Implement a query builder in the jcr component which takes care of translating JQOM queries to statements in string form
* Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
* Implement the actual query execution engine which interprets the query AST
was:
A query engine needs to be implemented.
A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with name space mappings to the oak-core.
We need to:
* Define the oak-core API for handling queries. How are do we handle name space mappings, limit and size
* Implement a query builder in the jcr component which takes care of translating JQOM queries
* Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
* Implement the actual query execution engine which interprets the query AST
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with value bindings, limit, offset, and name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and offset
> * Implement a query builder in the jcr component which takes care of translating JQOM queries to statements in string form
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Marcel Reutegger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235438#comment-13235438 ]
Marcel Reutegger commented on OAK-28:
-------------------------------------
Regarding value bindings and query parsing in the jcr binding: I don't think we can avoid parsing and at the same time use literals instead of value bindings. IMO the latter requires that you parse the query in order to find out what and where the bind variable names are. Therefore I'd rather not parse the query statement in the jcr binding and pass the map of bind variables/values.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and size
> * Implement a query builder in the jcr component which takes care of translating JQOM queries
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Michael Dürig (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235485#comment-13235485 ]
Michael Dürig commented on OAK-28:
----------------------------------
One thing we did not think of yet is access control: the jcr component needs to pass information about the authenticated user down to the query API of oak-core such that the query will only return items which the user is allowed to see.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with value bindings, limit, offset, and name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and offset
> * Implement a query builder in the jcr component which takes care of translating JQOM queries to statements in string form
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Thomas Mueller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234409#comment-13234409 ]
Thomas Mueller commented on OAK-28:
-----------------------------------
> Should it take a query string of some intermediate query language?
I think it should take a query string of any of the supported languages, which are SQL-2 and XPath at the moment.
> Should it take an AST of a query?
I think oak-core should take a query string only. The JQOM implementation in oak-jcr can create a SQL-2 query string. Creating and parsing a query string is quite fast, I don't expect performance to be an issue here. (Calculating the query plan might actually take longer - if we want to speed up things we should look at caching those and possibly caching query results).
> What about an extension to JQOM as was proposed by Felix?
>From what I heard it's a bit complicated to support as the JQOM classes are not easily extensible (this is a problem of the JCR API which can't be easily solved). But I don't know the details here yet.
Extending the query language on the other hand is quite simple. As an example, the current implementation supports "explain select ..." in addition to "select ...".
> avoid code duplications
There is some duplication: the JQOM implementation roughly matches the parse tree (at the moment). I don't see how this code duplication could be avoided, unless if we want to make it less modular (use the JCR API within oak-core, or move the query implementation in oak-jcr). On the other hand, it simplifies extending the parse tree. I currently don't see it as a big problem. It was a bit of work to copy the classes, but now it's done.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Michael Dürig (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234786#comment-13234786 ]
Michael Dürig commented on OAK-28:
----------------------------------
I updated the issue description to reflect the current state of the discussion. Also I made this issue into a container issue for the query implementations and created separate issues for the individual tasks for the respective components:
* OAK-34: Define query API
* OAK-35: Implement query builder
* OAK-36: Implement a query parser
At some later point we should also create an issue for the actual query engine which evaluates the AST.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and size
> * Implement a query builder in the jcr component which takes care of translating JQOM queries
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Thomas Mueller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235455#comment-13235455 ]
Thomas Mueller commented on OAK-28:
-----------------------------------
> Why do we need to implement a query parser in the jcr bindings module then?
Actually, it seems we don't need the parser in oak-jcr. I was wrong about that. I will update the issue description.
> Therefore I'd rather not parse the query statement in the jcr binding and pass the map of bind variables/values.
I would do the same.
> setLimit() and setOffset()
As for "limit" and "offset", it's not a problem to support them within the query statement as well - supporting it would actually simplify testing, as more tests can be written in
the script style currently used in oak-core/src/test/org.apache.jackrabbit.oak.query/queryTest.txt
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and size
> * Implement a query builder in the jcr component which takes care of translating JQOM queries
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Chetan Mehrotra (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445716#comment-13445716 ]
Chetan Mehrotra commented on OAK-28:
------------------------------------
A minor fix is required in {{OsgiIndexProvider}} at Line 87
{code}
- return providers.get(0).getQueryIndexes(mk);
+ return providers.entrySet().iterator().next().getValue().getQueryIndexes(mk);
{code}
Could not find a better way to get the only value entry. One other way can be to use NavigableMap
{code}
- private final Map<ServiceReference, QueryIndexProvider> providers =
- new HashMap<ServiceReference, QueryIndexProvider>();
+ private final NavigableMap<ServiceReference, QueryIndexProvider> providers =
+ new TreeMap<ServiceReference, QueryIndexProvider>();
{code}
And then
{code}
- return providers.get(0).getQueryIndexes(mk);
+ return providers.firstEntry().getValue().getQueryIndexes(mk);
{code}
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with value bindings, limit, offset, and name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and offset
> * Implement a query builder in the jcr component which takes care of translating JQOM queries to statements in string form
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OAK-28) Query implementation
Posted by "Thomas Mueller (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated OAK-28:
------------------------------
Attachment: OakToJcrQueryTreeConverter.java
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Chetan Mehrotra (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445740#comment-13445740 ]
Chetan Mehrotra commented on OAK-28:
------------------------------------
Another change required in Indexer. The flushBuffer method is currently hiding the exception thrown
{code}
} catch (MicroKernelException e) {
if (!mk.nodeExists(indexRootNode, revision)) {
// the index node itself was removed, which is
// unexpected but possible
// this will cause all indexes to be removed, so
// it can be ignored here
+ }else{
+ throw e;
}
}
{code}
With a bit older codebase with IndexWrapper in use I saw some exceptions on restart like
{noformat}
org.apache.jackrabbit.mk.api.MicroKernelException: org.h2.jdbc.JdbcSQLException: Unique index or primary key violation: "PRIMARY_KEY_2 ON PUBLIC.REVS(ID)"; SQL statement:
insert into REVS (ID, DATA, TIME) select ?, ?, ? [23505-158]
at org.apache.jackrabbit.mk.core.MicroKernelImpl.commit(MicroKernelImpl.java:502)
at org.apache.jackrabbit.oak.plugins.index.Indexer.commitChanges(Indexer.java:344)
at org.apache.jackrabbit.oak.plugins.index.Indexer.flushBuffer(Indexer.java:412)
at org.apache.jackrabbit.oak.plugins.index.Indexer.updateEnd(Indexer.java:406)
at org.apache.jackrabbit.mk.index.IndexWrapper.commitStream(IndexWrapper.java:122)
at org.apache.jackrabbit.mk.wrapper.MicroKernelWrapperBase.commit(MicroKernelWrapperBase.java:33)
at org.apache.jackrabbit.oak.core.ContentRepositoryImpl.<init>(ContentRepositoryImpl.java:122)
{noformat}
Would try with updated codebase and open an issue if problem persists
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core, jcr
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Labels: query
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> A query parser in oak-core should be able to handle xpath, sql2 and optionally other query languages. The jcr component must generate a valid query in one of those languages from JQOM queries and pass that statement along with value bindings, limit, offset, and name space mappings to the oak-core.
> We need to:
> * Define the oak-core API for handling queries. How are do we handle name space mappings, limit and offset
> * Implement a query builder in the jcr component which takes care of translating JQOM queries to statements in string form
> * Implement a query parser in oak-core and decide on a versatile AST representation which works with all query languages and which is extensible to future query languages.
> * Implement the actual query execution engine which interprets the query AST
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Jukka Zitting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234437#comment-13234437 ]
Jukka Zitting commented on OAK-28:
----------------------------------
bq. I think it should take a query string ...
+1 together with the query language and a map of applicable namespace mappings
bq. ... it's a bit complicated to support as the JQOM classes are not easily extensible ...
Right. For example JQOM makes pretty hard assumptions about the structure of a query, making features like "EXPLAIN", "DISTINCT" or "COUNT(*)" difficult to add. Other issues are that JQOM assumes that the right-hand side of all comparisons are static values and that each function maps to a separate interface and related factory method. Basically the API is designed in a way that makes it very cumbersome to extend even for fairly minor things.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Michael Dürig (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234394#comment-13234394 ]
Michael Dürig commented on OAK-28:
----------------------------------
Before we go too much into details of the implementation, how would the query API for oak-core look like?
* Should it take a query string of some intermediate query language?
* Should it take an AST of a query? Can we leverage JQOM here? What about an extension to JQOM as was proposed by Felix? Or should we go for another AST representation?
I think we should push very hard here in order to avoid code duplications.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-28) Query implementation
Posted by "Jukka Zitting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234551#comment-13234551 ]
Jukka Zitting commented on OAK-28:
----------------------------------
bq. ... limit and offset ...
Either add those as explicit parameters or allow them to be passed as a part of the query string. I think it would be useful to in any case support the latter option, so perhaps that's all we need?
bq. ... bound values ...
Agreed that those are best serialized along with the query itself. We can add support for value bindings later on if there's a convincing use case for it.
bq. Why do we need to implement a query parser in the jcr bindings module then?
We don't.
> Query implementation
> --------------------
>
> Key: OAK-28
> URL: https://issues.apache.org/jira/browse/OAK-28
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Attachments: OakToJcrQueryTreeConverter.java
>
>
> A query engine needs to be implemented.
> This includes a query parser in oak-core (where we don't want to use the JCR API), and a query parser in oak-jcr (where the parsed query tree needs to implement the JCR API).
> To avoid writing two independent parsers, I suggest to change the parser to emit a non-JCR query tree (so the parser can be used in oak-core). There needs to be a converter from the non-JCR query tree to a JCR query tree, so the same parser can be used in oak-jcr.
> This will still require two independent query tree implementations (about 37 duplicated classes: 37 classes for oak-core and 37 classes in oak-jcr). Plus it requires a tree converter.
> If somebody has a better idea please tell me :-)
> Prototype implementation of the query tree converter. Please note the class names are only to for illustration, but I don't know yet a good naming convention. Ideas are welcome!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira