You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Vihang Karajgaonkar (Jira)" <ji...@apache.org> on 2020/06/02 00:11:00 UTC
[jira] [Commented] (IMPALA-9057)
TestEventProcessing.test_insert_events_transactional is flaky
[ https://issues.apache.org/jira/browse/IMPALA-9057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121417#comment-17121417 ]
Vihang Karajgaonkar commented on IMPALA-9057:
---------------------------------------------
I looked into this and I think I found the problem. In case of transactional tables, the events are generated in a transactional context. So for example, in case of an insert into a transactional table, we see a sequence of OPEN_TXN, ALLOCATE_WRITE_ID, ALTER_PARTITION, COMMIT_TXN event on that table. Currently EventsProcessor refreshes the table when we see ALTER_PARTITION however, it is possible that the transaction has not been committed by that time. Hence we will skip the new files which were added by the insert after the reload. The solution is to wait until we see a COMMIT_TXN event before refreshing the tables. I will send out a fix for this soon.
> TestEventProcessing.test_insert_events_transactional is flaky
> -------------------------------------------------------------
>
> Key: IMPALA-9057
> URL: https://issues.apache.org/jira/browse/IMPALA-9057
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Alice Fan
> Assignee: Vihang Karajgaonkar
> Priority: Blocker
> Labels: build-failure, flaky
>
> Assertion failure for custom_cluster.test_event_processing.TestEventProcessing.test_insert_events_transactional
> {code:java}
> Error Message
> assert ['101', 'x', ..., '3', '2019'] == ['101', 'z', '28', '3', '2019'] At index 1 diff: 'x' != 'z' Full diff: - ['101', 'x', '28', '3', '2019'] ? ^ + ['101', 'z', '28', '3', '2019'] ? ^
> Stacktrace
> custom_cluster/test_event_processing.py:49: in test_insert_events_transactional
> self.run_test_insert_events(is_transactional=True)
> custom_cluster/test_event_processing.py:131: in run_test_insert_events
> assert data.split('\t') == ['101', 'z', '28', '3', '2019']
> E assert ['101', 'x', ..., '3', '2019'] == ['101', 'z', '28', '3', '2019']
> E At index 1 diff: 'x' != 'z'
> E Full diff:
> E - ['101', 'x', '28', '3', '2019']
> E ? ^
> E + ['101', 'z', '28', '3', '2019']
> E ? ^
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org