[jira] [Resolved] (IMPALA-5799) INSERTs into Kudu can crash if a column is dropped concurrently

["Matthew Jacobs (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew Jacobs resolved IMPALA-5799.
------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.10.0

commit dc7792c98026323ad87d49fa6f616ee378cc30cd
Author: Thomas Tauber-Marshall <tm...@cloudera.com>
Date:   Wed Aug 16 12:06:02 2017 -0700

    IMPALA-5799: Kudu DML can crash if schema has changed

    We check that the number/types of columns in a Kudu DML match the
    underlying table during analysis. However, it's possible that the
    schema may be modified between analysis and execution, and if it's
    modified in incompatible ways it can cause Impala to crash.

    Once the KuduTable object has been opened by the KuduTableSink, its
    schema will remain the same, so we can check in Open() that the schema
    is what we're expecting.

    If the schema changes between Open() and when the WriteOp is sent to Kudu,
    Kudu will send back an error and we already handle this gracefully.

    Testing:
    - Added an e2e test that concurrently inserts into a Kudu table while
      dropping and then adding a column. It relies on timing, but running
      in a loop locally it caused Impala to crash every time without this
      change.

    Change-Id: I9fd6bf164310df0041144f75f5ee722665e9f587
    Reviewed-on: http://gerrit.cloudera.org:8080/7688
    Reviewed-by: Matthew Jacobs <mj...@cloudera.com>
    Tested-by: Impala Public Jenkins

> INSERTs into Kudu can crash if a column is dropped concurrently
> ---------------------------------------------------------------
>
>                 Key: IMPALA-5799
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5799
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 2.10.0
>            Reporter: Thomas Tauber-Marshall
>            Assignee: Thomas Tauber-Marshall
>            Priority: Critical
>              Labels: kudu
>             Fix For: Impala 2.10.0
>
>
> When inserting into a Kudu table, we check that the table schema matches the values to insert during planning. Once the KuduTableSink opens the KuduTable object, we're guaranteed that the KuduTable's schema won't change.
> However, there's a window of vulnerability between the checks in the FE and opening the table in the BE where the schema could change. The only change that we allow that is incompatible is dropping a column, which can cause Impala to crash.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)