[jira] [Commented] (DRILL-4259) Add new functional tests to ensure that failures can be detected independent of the testing environment

["Victoria Markman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094182#comment-15094182 ] 

Victoria Markman commented on DRILL-4259:
-----------------------------------------

We talked about it year ago. The first step would be to enhance test framework to support something of this kind on a test suite level:
{code}
permutaitons: [
{ "planner.slice_target": 1, "planner.hash_join_enabled": true, "batch_size": 100} 
{ "planner.slice_target": 1, "planner.hash_join_enabled": false, "batch_size": 1}
]
{code}

> Add new functional tests to ensure that failures can be detected independent of the testing environment
> -------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4259
>                 URL: https://issues.apache.org/jira/browse/DRILL-4259
>             Project: Apache Drill
>          Issue Type: Test
>            Reporter: Jason Altekruse
>
> In DRILL-4243 an out of memory issue was fixed after a change to the memory allocator made memory limits more strict. While the regression tests had been run by the team at Dremio prior to merging the patch, running the tests on a cluster with more cores changed the memory limits on the queries and caused several tests to fail.
> While changes of this magnitude are not going to be common, we should have a test suite that reliably fails independent of the environment it is run (assuming that there are sufficient resources for the tests to run).
> It would be good to at least try to reproduce this failure on a few different setups (cores, nodes in cluster) by adjusting available configuration options and adding tests with those different configurations so that the tests will fail in different environments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)