Drill Test Framework

[rahul challapalli <ch...@gmail.com>]

Drillers,

The drill test framework [1] which some of you have been using contains a 2
level maven structure. This has always been a source of confusion for a
first time user. We would like to get rid of this and have a single pom
file to build the framework. This has a side effect of restructuring the
tests. The new repository will no longer contain a "framework" sub folder
and is more intuitive. Refer to [2] for the new structure. Let me know your
thoughts.


[1] https://github.com/mapr/drill-test-framework
[2] https://github.com/rchallapalli/drill-test-framework/tree/onelevel

- Rahul

Re: Drill Test Framework

[rahul challapalli <ch...@gmail.com>]

Jacques,

For most of us working on the test framework this is the most opportune
time to instrument such a structural change as we do not have any
uncommitted work and the impact on automation is minimal. When do you think
you can get the results from the git-lfs migration you are attempting?

I quickly put forward a branch with some of your structural suggestions (
https://github.com/rchallapalli/drill-test-framework/tree/jq) . Let me know
your thoughts.

Also what are your thoughts on moving the test framework into the drill
project itself (both framework and tests). If we wanted to treat the test
framework as a maven module within drill ( and test execution as an
optional maven target), then having the pom file in the root directory made
more sense to me.

Thanks for your other suggestions, I will add them to our task list and see
if we can prioritize them.

- Rahul

On Wed, Oct 14, 2015 at 10:43 PM, Jacques Nadeau <ja...@dremio.com> wrote:

> I'm trying to get git-lfs working so we can fix the model around change
> tracking and large files. I would propose waiting on this kind of
> structural change and we see the results of that effort.
>
> I actually would push for something slightly different. There are two main
> things: the test framework and the tests themselves. It seems like they
> should be managed in independent trees. E.g. /framework versus /tests. The
> tests suite is responsible executing the framework. Having the test data
> mixed with the test harness seems confusing.
>
> Other things worth considering.
>
>    - Better clarify/separate data preparation versus test execution steps.
>    - Remove relocation/abstraction in test files. This allows a simple sync
>    approach to matching cluster data to test data (and makes matching the
> two
>    way less confusing for devs trying to track down what the test failure
>    output means).
>    - Correct issues where scripts are used outside of the test framework
>    for Drill test generation as part of an internal test pattern.
>    - Move back to the original goal of test definition files having actual
>    test definitions (query, expected result, etc)
>    - Improve convention or configuration (there is a lot of boilerplate in
>    the test files that should be the default)
>    - Fix the odd # of queries/middle check hacks.
>
>
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Oct 14, 2015 at 12:23 PM, rahul challapalli <
> challapallirahul@gmail.com> wrote:
>
> > Drillers,
> >
> > The drill test framework [1] which some of you have been using contains
> a 2
> > level maven structure. This has always been a source of confusion for a
> > first time user. We would like to get rid of this and have a single pom
> > file to build the framework. This has a side effect of restructuring the
> > tests. The new repository will no longer contain a "framework" sub folder
> > and is more intuitive. Refer to [2] for the new structure. Let me know
> your
> > thoughts.
> >
> >
> > [1] https://github.com/mapr/drill-test-framework
> > [2] https://github.com/rchallapalli/drill-test-framework/tree/onelevel
> >
> > - Rahul
> >
>

Re: Drill Test Framework

[Jacques Nadeau <ja...@dremio.com>]

I'm trying to get git-lfs working so we can fix the model around change
tracking and large files. I would propose waiting on this kind of
structural change and we see the results of that effort.

I actually would push for something slightly different. There are two main
things: the test framework and the tests themselves. It seems like they
should be managed in independent trees. E.g. /framework versus /tests. The
tests suite is responsible executing the framework. Having the test data
mixed with the test harness seems confusing.

Other things worth considering.

   - Better clarify/separate data preparation versus test execution steps.
   - Remove relocation/abstraction in test files. This allows a simple sync
   approach to matching cluster data to test data (and makes matching the two
   way less confusing for devs trying to track down what the test failure
   output means).
   - Correct issues where scripts are used outside of the test framework
   for Drill test generation as part of an internal test pattern.
   - Move back to the original goal of test definition files having actual
   test definitions (query, expected result, etc)
   - Improve convention or configuration (there is a lot of boilerplate in
   the test files that should be the default)
   - Fix the odd # of queries/middle check hacks.




--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Wed, Oct 14, 2015 at 12:23 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:

> Drillers,
>
> The drill test framework [1] which some of you have been using contains a 2
> level maven structure. This has always been a source of confusion for a
> first time user. We would like to get rid of this and have a single pom
> file to build the framework. This has a side effect of restructuring the
> tests. The new repository will no longer contain a "framework" sub folder
> and is more intuitive. Refer to [2] for the new structure. Let me know your
> thoughts.
>
>
> [1] https://github.com/mapr/drill-test-framework
> [2] https://github.com/rchallapalli/drill-test-framework/tree/onelevel
>
> - Rahul
>