You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by GitBox <gi...@apache.org> on 2020/02/05 15:02:16 UTC
[GitHub] [drill] vvysotskyi opened a new pull request #1968: DRILL-7570: Fix
unstable statistics tests
vvysotskyi opened a new pull request #1968: DRILL-7570: Fix unstable statistics tests
URL: https://github.com/apache/drill/pull/1968
# [DRILL-7570](https://issues.apache.org/jira/browse/DRILL-7570): Fix unstable statistics tests
## Description
Updated sample statistics tests
## Documentation
NA
## Testing
Checked that updated unit tests pass
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on issue #1968: DRILL-7570: Fix
unstable statistics tests
Posted by GitBox <gi...@apache.org>.
arina-ielchiieva commented on issue #1968: DRILL-7570: Fix unstable statistics tests
URL: https://github.com/apache/drill/pull/1968#issuecomment-582460192
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on a change in pull request
#1968: DRILL-7570: Fix unstable statistics tests
Posted by GitBox <gi...@apache.org>.
arina-ielchiieva commented on a change in pull request #1968: DRILL-7570: Fix unstable statistics tests
URL: https://github.com/apache/drill/pull/1968#discussion_r375310419
##########
File path: exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestAnalyze.java
##########
@@ -522,46 +601,45 @@ public void testHistogramWithColumnsWithAllNulls() throws Exception {
.go();
} finally {
- resetSessionOption("planner.slice_target");
- resetSessionOption("store.format");
+ client.resetSession(ExecConstants.SLICE_TARGET);
+ client.resetSession(ExecConstants.OUTPUT_FORMAT_OPTION);
}
}
@Test
public void testHistogramWithIntervalPredicate() throws Exception {
try {
- test("ALTER SESSION SET `planner.slice_target` = 1");
- test("ALTER SESSION SET `store.format` = 'parquet'");
- test("create table dfs.tmp.orders2 as select * from cp.`tpch/orders.parquet`");
- test("analyze table dfs.tmp.orders2 compute statistics");
- test("alter session set `planner.statistics.use` = true");
+ client.alterSession(ExecConstants.SLICE_TARGET, 1);
+ client.alterSession(ExecConstants.OUTPUT_FORMAT_OPTION, "parquet");
+ run("create table dfs.tmp.orders2 as select * from cp.`tpch/orders.parquet`");
+ run("analyze table dfs.tmp.orders2 compute statistics");
+ client.alterSession(PlannerSettings.STATISTICS_USE.getOptionName(), true);
String query = "select 1 from dfs.tmp.orders2 o where o.o_orderdate >= date '1996-10-01' and o.o_orderdate < date '1996-10-01' + interval '3' month";
String[] expectedPlan1 = {"Filter\\(condition.*\\).*rowcount = 59?.*,.*", "Scan.*columns=\\[`o_orderdate`\\].*rowcount = 15000.0.*"};
- PlanTestBase.testPlanWithAttributesMatchingPatterns(query, expectedPlan1, new String[]{});
+ queryBuilder()
+ .sql(query)
+ .detailedPlanMatcher()
+ .include(expectedPlan1)
+ .match();
} finally {
- resetSessionOption("planner.slice_target");
- resetSessionOption("store.format");
+ client.resetSession(ExecConstants.SLICE_TARGET);
+ client.resetSession(ExecConstants.OUTPUT_FORMAT_OPTION);
}
}
//Helper function to verify output of ANALYZE statement
private void verifyAnalyzeOutput(String query, String message) throws Exception {
- List<QueryDataBatch>result = testRunAndReturn(QueryType.SQL, query);
- List<List<String>> output = new ArrayList<>();
- assertTrue(result.size() == 1);
- final QueryDataBatch batch = result.get(0);
- final RecordBatchLoader loader = new RecordBatchLoader(getDrillbitContext().getAllocator());
- loader.load(batch.getHeader().getDef(), batch.getData());
- output.add(new ArrayList<String>());
- for (VectorWrapper<?> vw: loader) {
- ValueVector.Accessor accessor = vw.getValueVector().getAccessor();
- Object o = accessor.getObject(0);
- output.get(0).add(o == null ? null: o.toString());
+ DirectRowSet rowSet = queryBuilder().sql(query).rowSet();
+ assertEquals(1, rowSet.rowCount());
+
+ RowSetReader reader = rowSet.reader();
+ assertEquals(2, reader.columnCount());
+ while (reader.next()) {
+ ObjectReader column = reader.column(1);
+ assertEquals(message, column.isNull() ? null: column.getObject().toString());
}
- batch.release();
- loader.clear();
- assertTrue(output.get(0).size() == 2);
- assertEquals(message, output.get(0).get(1));
+
+ rowSet.clear();
Review comment:
finally?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] arina-ielchiieva merged pull request #1968: DRILL-7570: Fix
unstable statistics tests
Posted by GitBox <gi...@apache.org>.
arina-ielchiieva merged pull request #1968: DRILL-7570: Fix unstable statistics tests
URL: https://github.com/apache/drill/pull/1968
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] vvysotskyi commented on a change in pull request #1968:
DRILL-7570: Fix unstable statistics tests
Posted by GitBox <gi...@apache.org>.
vvysotskyi commented on a change in pull request #1968: DRILL-7570: Fix unstable statistics tests
URL: https://github.com/apache/drill/pull/1968#discussion_r375318988
##########
File path: exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestAnalyze.java
##########
@@ -522,46 +601,45 @@ public void testHistogramWithColumnsWithAllNulls() throws Exception {
.go();
} finally {
- resetSessionOption("planner.slice_target");
- resetSessionOption("store.format");
+ client.resetSession(ExecConstants.SLICE_TARGET);
+ client.resetSession(ExecConstants.OUTPUT_FORMAT_OPTION);
}
}
@Test
public void testHistogramWithIntervalPredicate() throws Exception {
try {
- test("ALTER SESSION SET `planner.slice_target` = 1");
- test("ALTER SESSION SET `store.format` = 'parquet'");
- test("create table dfs.tmp.orders2 as select * from cp.`tpch/orders.parquet`");
- test("analyze table dfs.tmp.orders2 compute statistics");
- test("alter session set `planner.statistics.use` = true");
+ client.alterSession(ExecConstants.SLICE_TARGET, 1);
+ client.alterSession(ExecConstants.OUTPUT_FORMAT_OPTION, "parquet");
+ run("create table dfs.tmp.orders2 as select * from cp.`tpch/orders.parquet`");
+ run("analyze table dfs.tmp.orders2 compute statistics");
+ client.alterSession(PlannerSettings.STATISTICS_USE.getOptionName(), true);
String query = "select 1 from dfs.tmp.orders2 o where o.o_orderdate >= date '1996-10-01' and o.o_orderdate < date '1996-10-01' + interval '3' month";
String[] expectedPlan1 = {"Filter\\(condition.*\\).*rowcount = 59?.*,.*", "Scan.*columns=\\[`o_orderdate`\\].*rowcount = 15000.0.*"};
- PlanTestBase.testPlanWithAttributesMatchingPatterns(query, expectedPlan1, new String[]{});
+ queryBuilder()
+ .sql(query)
+ .detailedPlanMatcher()
+ .include(expectedPlan1)
+ .match();
} finally {
- resetSessionOption("planner.slice_target");
- resetSessionOption("store.format");
+ client.resetSession(ExecConstants.SLICE_TARGET);
+ client.resetSession(ExecConstants.OUTPUT_FORMAT_OPTION);
}
}
//Helper function to verify output of ANALYZE statement
private void verifyAnalyzeOutput(String query, String message) throws Exception {
- List<QueryDataBatch>result = testRunAndReturn(QueryType.SQL, query);
- List<List<String>> output = new ArrayList<>();
- assertTrue(result.size() == 1);
- final QueryDataBatch batch = result.get(0);
- final RecordBatchLoader loader = new RecordBatchLoader(getDrillbitContext().getAllocator());
- loader.load(batch.getHeader().getDef(), batch.getData());
- output.add(new ArrayList<String>());
- for (VectorWrapper<?> vw: loader) {
- ValueVector.Accessor accessor = vw.getValueVector().getAccessor();
- Object o = accessor.getObject(0);
- output.get(0).add(o == null ? null: o.toString());
+ DirectRowSet rowSet = queryBuilder().sql(query).rowSet();
+ assertEquals(1, rowSet.rowCount());
+
+ RowSetReader reader = rowSet.reader();
+ assertEquals(2, reader.columnCount());
+ while (reader.next()) {
+ ObjectReader column = reader.column(1);
+ assertEquals(message, column.isNull() ? null: column.getObject().toString());
}
- batch.release();
- loader.clear();
- assertTrue(output.get(0).size() == 2);
- assertEquals(message, output.get(0).get(1));
+
+ rowSet.clear();
Review comment:
Thanks, done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services