[jira] [Resolved] (IMPALA-6187) Scan with conjuncts but no materialized slots crashes Impalad

["Michael Ho (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Ho resolved IMPALA-6187.
--------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.11.0

https://github.com/apache/impala/commit/63f17e9ceaed92a28ea12567a36b746e54fffdb3

IMPALA-6187: Fix missing conjuncts evaluation with empty projection
Previously, scanners will assume that there are no conjuncts associated
with a scan node for queries with no materialized slots (e.g. count(*)).
This is not necessarily the case as one can write queries such as
select count(*) from tpch.lineitem where rand() * 10 < 0; or
select count(*) from tpch.lineitem where rand() > <a partition column>.
In which case, the conjuncts should still be evaluated once per row.

This change fixes the problem in the short-circuit handling logic for
count(*) to evaluate the conjuncts once per row and only commits a row
to the output row batch if the conjuncts evaluate to true.

Testing done: Added the example above to the scanner test

Change-Id: Ib530f1fdcd2c6de699977db163b3f6eb38481517
Reviewed-on: http://gerrit.cloudera.org:8080/8623
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Reviewed-by: Alex Behm <al...@cloudera.com>
Reviewed-by: Dan Hecht <dh...@cloudera.com>
Tested-by: Impala Public Jenkins

> Scan with conjuncts but no materialized slots crashes Impalad
> -------------------------------------------------------------
>
>                 Key: IMPALA-6187
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6187
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.10.0
>            Reporter: Michael Ho
>            Assignee: Michael Ho
>            Priority: Critical
>              Labels: correctness
>             Fix For: Impala 2.11.0
>
>
> The simple query {noformat} select count(*) from tpch_parquet.lineitem where rand() = 10.0 {noformat} will trigger the following DCHECK below in {{HdfsScanner::WriteTemplate()}}:
> {noformat}
> int HdfsScanner::WriteTemplateTuples(TupleRow* row, int num_tuples) {
>   DCHECK_GE(num_tuples, 0);
>   DCHECK_EQ(scan_node_->tuple_idx(), 0);
>   DCHECK_EQ(conjunct_evals_->size(), 0); <<------
>   if (num_tuples == 0 || template_tuple_ == NULL) return num_tuples;
>   Tuple** row_tuple = reinterpret_cast<Tuple**>(row);
>   for (int i = 0; i < num_tuples; ++i) row_tuple[i] = template_tuple_;
>   return num_tuples;
> }
> {noformat}
> The assumption in the scanner is that if there are no materialized slots (e.g. count star), there should be no conjuncts too but that doesn't always seems to be the case.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)