[jira] [Resolved] (IMPALA-4591) Kudu client error memory should be bounded

["Thomas Tauber-Marshall (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Tauber-Marshall resolved IMPALA-4591.
--------------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.11.0

commit abd9b0e70a3793405af8ad72259e4c7196804a47
Author: Thomas Tauber-Marshall <tm...@cloudera.com>
Date:   Fri Nov 3 14:51:53 2017 -0700

    IMPALA-4591: Bound Kudu client error mem usage
    
    Previously, Kudu client errors could grow in size unbounded,
    potentially causing the process to be killed. This patch sets a
    bound on the mem that can be used for these error messages, with
    the size determined by the flag 'kudu_error_buffer_size'.
    
    If the errors for a Kudu client exceed this size, the query will fail,
    as some errors will be dropped and we won't be able to tell if all of
    the errors can be safely ignored.
    
    Testing:
    - Added a custom cluster test that verifies that a query that exceeds
      the limit fails.
    
    Change-Id: I186ddb3f3b5865e08f17dba57cf6640591d06b14
    Reviewed-on: http://gerrit.cloudera.org:8080/8464
    Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
    Tested-by: Impala Public Jenkins

> Kudu client error memory should be bounded
> ------------------------------------------
>
>                 Key: IMPALA-4591
>                 URL: https://issues.apache.org/jira/browse/IMPALA-4591
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.8.0
>            Reporter: Matthew Jacobs
>            Assignee: Thomas Tauber-Marshall
>              Labels: kudu
>             Fix For: Impala 2.11.0
>
>
> Kudu client memory needs to be tracked by Impala. A short-term solution was added in IMPALA-4478 to TryConsume a reasonably large amount of memory. This helps avoid OOM issues, but the client could still consume unbounded memory for error handling.
> https://issues.apache.org/jira/browse/KUDU-1752 added support in the Kudu client for bounding the error memory size. Impala should set this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)