[jira] [Commented] (IMPALA-7648) Add tests for all cases where OOM is expected

["Tim Armstrong (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659883#comment-16659883 ] 

Tim Armstrong commented on IMPALA-7648:
---------------------------------------

IMPALA-7648 part 1: add expected out-of-memory tests
 
 This adds test coverage for some cases where queries are currently
 expected to fail with out-of-memory:
 * memory limit exceeded in exchange node
 * aggregation with large var-len intermediate values
 * top N with large limit
 * hash join with many duplicates on right side
 * analytic with a large window that needs to be buffered in-memory
 
 Note that it's not always totally deterministic where the query hits
 'memory limit exceeded' so we don't include the node ID or name in the
 expected error message.
 
 Testing:
 * ran exhaustive tests
 * looped modified tests locally overnight
 
 Change-Id: Icd1a7eb97837b742a967c260cafb5a7f4f45412e
 Reviewed-on: [http://gerrit.cloudera.org:8080/11564]
 Reviewed-by: Impala Public Jenkins <[impala-public-jenkins@cloudera.com|mailto:impala-public-jenkins@cloudera.com]>
 Tested-by: Impala Public Jenkins <[impala-public-jenkins@cloudera.com|mailto:impala-public-jenkins@cloudera.com]>

> Add tests for all cases where OOM is expected
> ---------------------------------------------
>
>                 Key: IMPALA-7648
>                 URL: https://issues.apache.org/jira/browse/IMPALA-7648
>             Project: IMPALA
>          Issue Type: Test
>          Components: Infrastructure
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>            Priority: Major
>              Labels: resource-management
>
> We should add tests for most or all of the subtasks of IMPALA-4834 that exercise the expected OOM code path. I'll add some bullet points here to track what the coverage is and what's missing.
> * Aggregations with large var-len string expressions
> * Top-N with large window
> * -Kudu scans- : covered by kudu-scan-mem-usage.test
> * -Nested loop join- : covered by single-node-nlj-exhaustive.test
> * Many duplicate keys on build side of hash join
> * -Large number of NULLS on build side of NAAJ- : covered by spilling-naaj-no-deny-reservation.test
> * -HDFS table partitioned insert- : covered insert-mem-limit.test
> * Large analytic window can't be spilled
> * Queries processing large strings (may need multiple tests to cover different places). large_strings.test has some coverage
> * Parquet files with large pages
> * Exchange uses a lot of memory



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org