Optimizer Tests for SQLPP

[Xikui Wang <xi...@uci.edu>]

Hi Devs,

As I mentioned in the weekly meeting, I found that our OptimizerTest
actually doesn't run the SQLPP tests. Although there is a separate
directory 'queries_sqlpp' which contains all the legacy optimizer tests
translated into SQLPP, they are not picked up by the OptimizerTest, and the
new SQLPP tests are still being added to the old directory and mixed up
with AQL tests.

I tried to run those SQLPP tests. More than half of them are failed. There
are syntax error (query-issue838.sqlpp), variable name changes, join
algorithm changes (word-jaccard.sqlpp) and other changes (issue730.sqlpp).
I submitted one patch that fixed the test cases with variable name changes.
For the rests, I think we need to decide, between the two versions of the
results, which ones are the expected plans and fix the errors. There are
some obvious patterns in the plan changes, so I think we only need to fix a
few things to cover the rest 450 test cases...

Best,
Xikui

Re: Optimizer Tests for SQLPP

[Taewoo Kim <wa...@gmail.com>]

That also makes sense. Since the similarity join branch is not merged yet,
I will add them.

On Sat, Apr 14, 2018 at 16:05 Mike Carey <dt...@gmail.com> wrote:

> Question:  Aren't there similarity join tests as well, though?  (I.e.,
> is it not sufficient to let it defend itself, rather than also testing
> its component parts?)
>
>
> On 4/14/18 10:42 AM, Taewoo Kim wrote:
> > For now, please do not remove AQL optimizer test cases since AQL+
> > functionality that is used for the similarity join depends on them. I
> will
> > go through runtime test cases and optimizer test cases and remove test
> > cases that are covered by SQL++ test cases and that are not directly
> > related to AQL+ functionality.
> >
> > Best,
> > Taewoo
> >
> > On Sat, Apr 14, 2018 at 9:47 AM, Xikui Wang <xi...@uci.edu> wrote:
> >
> >> Hi Devs,
> >>
> >> As I mentioned in the weekly meeting, I found that our OptimizerTest
> >> actually doesn't run the SQLPP tests. Although there is a separate
> >> directory 'queries_sqlpp' which contains all the legacy optimizer tests
> >> translated into SQLPP, they are not picked up by the OptimizerTest, and
> the
> >> new SQLPP tests are still being added to the old directory and mixed up
> >> with AQL tests.
> >>
> >> I tried to run those SQLPP tests. More than half of them are failed.
> There
> >> are syntax error (query-issue838.sqlpp), variable name changes, join
> >> algorithm changes (word-jaccard.sqlpp) and other changes
> (issue730.sqlpp).
> >> I submitted one patch that fixed the test cases with variable name
> changes.
> >> For the rests, I think we need to decide, between the two versions of
> the
> >> results, which ones are the expected plans and fix the errors. There are
> >> some obvious patterns in the plan changes, so I think we only need to
> fix a
> >> few things to cover the rest 450 test cases...
> >>
> >> Best,
> >> Xikui
> >>
>
>

Re: Optimizer Tests for SQLPP

[Mike Carey <dt...@gmail.com>]

Question:  Aren't there similarity join tests as well, though?  (I.e., 
is it not sufficient to let it defend itself, rather than also testing 
its component parts?)


On 4/14/18 10:42 AM, Taewoo Kim wrote:
> For now, please do not remove AQL optimizer test cases since AQL+
> functionality that is used for the similarity join depends on them. I will
> go through runtime test cases and optimizer test cases and remove test
> cases that are covered by SQL++ test cases and that are not directly
> related to AQL+ functionality.
>
> Best,
> Taewoo
>
> On Sat, Apr 14, 2018 at 9:47 AM, Xikui Wang <xi...@uci.edu> wrote:
>
>> Hi Devs,
>>
>> As I mentioned in the weekly meeting, I found that our OptimizerTest
>> actually doesn't run the SQLPP tests. Although there is a separate
>> directory 'queries_sqlpp' which contains all the legacy optimizer tests
>> translated into SQLPP, they are not picked up by the OptimizerTest, and the
>> new SQLPP tests are still being added to the old directory and mixed up
>> with AQL tests.
>>
>> I tried to run those SQLPP tests. More than half of them are failed. There
>> are syntax error (query-issue838.sqlpp), variable name changes, join
>> algorithm changes (word-jaccard.sqlpp) and other changes (issue730.sqlpp).
>> I submitted one patch that fixed the test cases with variable name changes.
>> For the rests, I think we need to decide, between the two versions of the
>> results, which ones are the expected plans and fix the errors. There are
>> some obvious patterns in the plan changes, so I think we only need to fix a
>> few things to cover the rest 450 test cases...
>>
>> Best,
>> Xikui
>>

Re: Optimizer Tests for SQLPP

[Taewoo Kim <wa...@gmail.com>]

For now, please do not remove AQL optimizer test cases since AQL+
functionality that is used for the similarity join depends on them. I will
go through runtime test cases and optimizer test cases and remove test
cases that are covered by SQL++ test cases and that are not directly
related to AQL+ functionality.

Best,
Taewoo

On Sat, Apr 14, 2018 at 9:47 AM, Xikui Wang <xi...@uci.edu> wrote:

> Hi Devs,
>
> As I mentioned in the weekly meeting, I found that our OptimizerTest
> actually doesn't run the SQLPP tests. Although there is a separate
> directory 'queries_sqlpp' which contains all the legacy optimizer tests
> translated into SQLPP, they are not picked up by the OptimizerTest, and the
> new SQLPP tests are still being added to the old directory and mixed up
> with AQL tests.
>
> I tried to run those SQLPP tests. More than half of them are failed. There
> are syntax error (query-issue838.sqlpp), variable name changes, join
> algorithm changes (word-jaccard.sqlpp) and other changes (issue730.sqlpp).
> I submitted one patch that fixed the test cases with variable name changes.
> For the rests, I think we need to decide, between the two versions of the
> results, which ones are the expected plans and fix the errors. There are
> some obvious patterns in the plan changes, so I think we only need to fix a
> few things to cover the rest 450 test cases...
>
> Best,
> Xikui
>