You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2020/02/03 06:44:06 UTC
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15151
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
The test uses a thread pool of 3 slots to run 3 concurrent CreateTable
statements. Each thread creates a client and closes it at the end of the
task(run_create_table_if_not_exists). The first 2 thread tasks start at
the same time. Then after one second, the 3rd thread task starts.
Ideally the 3 tasks should be executed in 3 different threads of the
thread pool. However, if any of the first 2 tasks finish in 1s, the 3rd
task could be executed by reusing the thread. Since the client in the
thread is closed, the task fails.
This patch removes the close operation at the end of the task.
Connection of the thread can be reused and will be closed when the
thread terminates.
Also fix the same problem in test_concurrent_ddls.py.
Tests:
- Run test_concurrent_ddls.py and test_concurrent_kudu_create.py. In
the logs of impalad, observe that connections are being reused and
closed at the end of the tests.
Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
---
M tests/custom_cluster/test_concurrent_ddls.py
M tests/custom_cluster/test_concurrent_kudu_create.py
2 files changed, 4 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/15151/1
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 05 Feb 2020 06:23:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 1: Code-Review+1
(2 comments)
Thanks to Quanlong for providing a fix promptly! I only left 2 minor comments.
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py
File tests/custom_cluster/test_concurrent_kudu_create.py:
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py@a47
PS1, Line 47:
I am not very familiar with the usage of tls.client. Is it true that we do not have to explicitly call close() before we are going to terminate the thread pool after the for-loop at https://gerrit.cloudera.org/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py#65? Thanks!
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py@51
PS1, Line 51: pool = ThreadPool(processes=3)
Just like to check whether or not my understanding is correct. The reason to move ThreadPool() and pool.terminate() out of the for-loop is to reduce the overhead of thread pool creation?
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 17:59:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 05 Feb 2020 01:34:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5495/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 23:38:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5281/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 05 Feb 2020 01:34:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
The test uses a thread pool of 3 slots to run 3 concurrent CreateTable
statements. Each thread creates a client and closes it at the end of the
task(run_create_table_if_not_exists). The first 2 thread tasks start at
the same time. Then after one second, the 3rd thread task starts.
Ideally the 3 tasks should be executed in 3 different threads of the
thread pool. However, if any of the first 2 tasks finish in 1s, the 3rd
task could be executed by reusing the thread. Since the client in the
thread is closed, the task fails.
This patch removes the close operation at the end of the task.
Connection of the thread can be reused and will be closed when the
thread terminates.
Also fix the same problem in test_concurrent_ddls.py.
Tests:
- Run test_concurrent_ddls.py and test_concurrent_kudu_create.py. In
the logs of impalad, observe that connections are being reused and
closed at the end of the tests.
Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Reviewed-on: http://gerrit.cloudera.org:8080/15151
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/custom_cluster/test_concurrent_ddls.py
M tests/custom_cluster/test_concurrent_kudu_create.py
2 files changed, 4 insertions(+), 6 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py
File tests/custom_cluster/test_concurrent_kudu_create.py:
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py@a47
PS1, Line 47:
> I am not very familiar with the usage of tls.client. Is it true that we do
Replied this at line65.
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py@51
PS1, Line 51: pool = ThreadPool(processes=3)
> Just like to check whether or not my understanding is correct. The reason t
Yes, we can reuse the threads and therefore reuse the client (connection) in the thread.
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py@65
PS1, Line 65: pool
> I looked at other usages of client and it seems that we don't need to expli
Yes, when the threads are terminated, their connections are closed so the associated sessions are closed too. I verify these in impalad.INFO. By the way, since this is a custom cluster test, the cluster will be restarted after the test. So we don't need to be afraid of potential session leaks affecting other tests.
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 22:56:07 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/5584/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 07:39:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 23:28:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py
File tests/custom_cluster/test_concurrent_kudu_create.py:
http://gerrit.cloudera.org:8080/#/c/15151/1/tests/custom_cluster/test_concurrent_kudu_create.py@65
PS1, Line 65: pool
I looked at other usages of client and it seems that we don't need to explicitly call a close on it. Is that correct?
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 18:27:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15151 )
Change subject: IMPALA-9289: Fix flakiness in TestConcurrentKuduCreate
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15151
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idefba98ecd65efbd47b1618291330795ef13b910
Gerrit-Change-Number: 15151
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 03 Feb 2020 23:38:38 +0000
Gerrit-HasComments: No