[jira] [Commented] (IMPALA-2376) Scan of array value with 100m elements with reasonable mem limit hits DCHECK.

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457850#comment-16457850 ] 

ASF subversion and git services commented on IMPALA-2376:
---------------------------------------------------------

Commit 93d714c6458a680151c7208b1fe23677f4705a17 in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=93d714c ]

IMPALA-6560: fix regression test for IMPALA-2376

The test is modified to increase the size of collections allocated.
num_nodes and mt_dop query options are set to make execution as
deterministic as possible.

I looped the test overnight to try to flush out flakiness.

Adds support for row_regex lines in CATCH sections so that we can
match a larger part of the error message.

Change-Id: I024cb6b57647902b1735defb885cd095fd99738c
Reviewed-on: http://gerrit.cloudera.org:8080/9681
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Tim Armstrong <ta...@cloudera.com>


> Scan of array value with 100m elements with reasonable mem limit hits DCHECK.
> -----------------------------------------------------------------------------
>
>                 Key: IMPALA-2376
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2376
>             Project: IMPALA
>          Issue Type: Bug
>    Affects Versions: Impala 2.3.0
>            Reporter: Alexander Behm
>            Assignee: Skye Wanderman-Milne
>            Priority: Blocker
>              Labels: crash, nested_types, resource-management
>
> The query below when run without a mem limit needs roughly 2.4g of memory in the scan.
> My expectation is that I get a mem limit exceeded error when running the same query with a mem limit below that 2.4g. However, we hit a DCHECK in the scanner.
> Repro:
> 1. Grab Parquet file from here:
> vd0212.halxg.cloudera.com:/data/1/huge_array_parquet/100m_array.parq
> 2. Copy file to HDFS and use CREATE TABLE LIKE FILE
> 3. The query below runs fine without a mem limit:
> {code}
> select cnt from huge_array_table t, (select count(item) cnt from t.f) v;
> {code}
> 4. Set the mem limit to 1g and run the query again. You will hit this DCHECK:
> {code}
> hdfs-parquet-scanner.cc:1299] Check failed: !parse_status_.ok()
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org