[jira] [Created] (IMPALA-9825) S3Guard's DynamoDB isntance got deleted during an S3 job run

["Laszlo Gaal (Jira)" <ji...@apache.org>]

Laszlo Gaal created IMPALA-9825:
-----------------------------------

             Summary: S3Guard's DynamoDB isntance got deleted during an S3 job run
                 Key: IMPALA-9825
                 URL: https://issues.apache.org/jira/browse/IMPALA-9825
             Project: IMPALA
          Issue Type: Bug
          Components: Infrastructure
    Affects Versions: Impala 4.0
         Environment: S3 with S3Guard enabled.
            Reporter: Laszlo Gaal


During an S3 test run the DynamoDB instance serving as the data store for S3Guard got deleted. This was explicitly reported by the first S3Guard error message during the run:
{code}
 ERROR at teardown of TestScannersFuzzing.test_fuzz_alltypes[protocol: beeswax | exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none] 
[gw5] linux2 -- Python 2.7.5 /data/jenkins/workspace/impala-cdpd-master-core-s3/repos/Impala/bin/../infra/python/env/bin/python
conftest.py:341: in cleanup
    {'sync_ddl': sync_ddl})
common/impala_test_suite.py:805: in wrapper
    return function(*args, **kwargs)
common/impala_test_suite.py:813: in execute_query_expect_success
    result = cls.__execute_query(impalad_client, query, query_options, user)
common/impala_test_suite.py:903: in __execute_query
    return impalad_client.execute(query, user=user)
common/impala_connection.py:205: in execute
    return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
    handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:363: in __execute_query
    handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:357: in execute_query_async
    handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:520: in __do_rpc
    raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E   ImpalaBeeswaxException: ImpalaBeeswaxException:
E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
E    MESSAGE: ImpalaRuntimeException: Error making 'dropDatabase' RPC to Hive Metastore: 
E   CAUSED BY: MetaException: Got exception: java.io.FileNotFoundException DynamoDB table 'impala-test-uswest2-1' is being deleted in region us-west-2
{code}

This caused subsequent S3 operations to fail, wasting the complete test run.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org