Query integration update

[Jens Hübel <jh...@opentext.com>]

Hi Chemistry,

 

in the past weeks I have done some work with query integration. Currently all is part of the InMemory server, but I will move the reusable parts to the server-support package and a documentation page to the wiki.

 

There is one problem remaining:

 

As discussed in CMIS-212 I have managed to separate the CMIS core grammar from the proposed set of extensions. This also is a nice pattern for others who want to extend the grammar. 

 

The "composite grammars" seems to be relatively new feature in ANTLR, not being 100% mature. With some trial and error I got this working, but there is one drawback. The gunit test framework is currently incompatible with composite grammars, all tests result in a ClassNotFoundException. I tracked this down and this is caused by some reflection magic in the test framework. The code that generates the method names  to call the parser is incorrect for composite grammars and hard coded. I also tried to upgrade ANTLR to 3.2 but it did not help. Another disadvantage of composite grammars is that the generated Java code is more difficult to read.

 

Now we have two ways to resolve this:

1)

Duplicated grammars with having some redundant rules and use gunit

 

2) 

I have worked around this bug and now can do the same in a junit test. The tests then don't have such a nice syntax but basically it is the same. This would let us get rid of the gunit dependency. I have adapted all available tests and a few new ones. 

Instead of:

"123.345E78" OK

" SELECT * FROM WHITE_PAPER" -> (SELECT * (FROM (TABLE WHITE_PAPER)))

 

Your will now have

    @Test

    public void testQuery () throws Exception {

        testParserOk("literal", "123.345E78");

        testParser("query", "SELECT * FROM WHITE_PAPER", "(SELECT * (FROM (TABLE WHITE_PAPER)))");

    }

 

 

I am happy with both approaches. If there are no comments I will continue with 2).2) integrates better with Java and our remaining test framework.  As the CMIS grammar is not overly complex 1  is not a big deal either. But for example the full text part (being still a todo) needs to be done twice.

 

The reusable part for query integration also needs access to the type system. Therefore  a dependency on a type manager is introduced. This is still on our TODO-list in https://issues.apache.org/jira/browse/CMIS-196. I have added in the supported manager an interface called "TypeManager" that is implemented in the inmemory server. This is nothing but a placeholder until we get a real type manager. There are only very few methods needed for query support.

 

Jens

 

 

Re: Query integration update

[Florent Guillaume <fg...@nuxeo.com>]

Hi,

This is good news Jens!
I was not aware of the composite grammar feature, but it's great if
this can be made to work cleanly.
Frankly keeping Gunit is not very important as long as we can do the
same tests. You seem to have a nice framework for testing the
lexer/parser so I think we should use it instead.

Florent

On Wed, Jul 7, 2010 at 9:33 AM, Jens Hübel <jh...@opentext.com> wrote:
> Hi Chemistry,
>
>
>
> in the past weeks I have done some work with query integration. Currently all is part of the InMemory server, but I will move the reusable parts to the server-support package and a documentation page to the wiki.
>
>
>
> There is one problem remaining:
>
>
>
> As discussed in CMIS-212 I have managed to separate the CMIS core grammar from the proposed set of extensions. This also is a nice pattern for others who want to extend the grammar.
>
>
>
> The "composite grammars" seems to be relatively new feature in ANTLR, not being 100% mature. With some trial and error I got this working, but there is one drawback. The gunit test framework is currently incompatible with composite grammars, all tests result in a ClassNotFoundException. I tracked this down and this is caused by some reflection magic in the test framework. The code that generates the method names  to call the parser is incorrect for composite grammars and hard coded. I also tried to upgrade ANTLR to 3.2 but it did not help. Another disadvantage of composite grammars is that the generated Java code is more difficult to read.
>
>
>
> Now we have two ways to resolve this:
>
> 1)
>
> Duplicated grammars with having some redundant rules and use gunit
>
>
>
> 2)
>
> I have worked around this bug and now can do the same in a junit test. The tests then don't have such a nice syntax but basically it is the same. This would let us get rid of the gunit dependency. I have adapted all available tests and a few new ones.
>
> Instead of:
>
> "123.345E78" OK
>
> " SELECT * FROM WHITE_PAPER" -> (SELECT * (FROM (TABLE WHITE_PAPER)))
>
>
>
> Your will now have
>
>    @Test
>
>    public void testQuery () throws Exception {
>
>        testParserOk("literal", "123.345E78");
>
>        testParser("query", "SELECT * FROM WHITE_PAPER", "(SELECT * (FROM (TABLE WHITE_PAPER)))");
>
>    }
>
>
>
>
>
> I am happy with both approaches. If there are no comments I will continue with 2).2) integrates better with Java and our remaining test framework.  As the CMIS grammar is not overly complex 1  is not a big deal either. But for example the full text part (being still a todo) needs to be done twice.
>
>
>
> The reusable part for query integration also needs access to the type system. Therefore  a dependency on a type manager is introduced. This is still on our TODO-list in https://issues.apache.org/jira/browse/CMIS-196. I have added in the supported manager an interface called "TypeManager" that is implemented in the inmemory server. This is nothing but a placeholder until we get a real type manager. There are only very few methods needed for query support.
>
>
>
> Jens
>
>
>
>
>
>



-- 
Florent Guillaume, Director of R&D, Nuxeo
Open Source, Java EE based, Enterprise Content Management (ECM)
http://www.nuxeo.com   http://www.nuxeo.org   +33 1 40 33 79 87