You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2024/09/13 03:55:09 UTC

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21796


Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................

IMPALA-13377: Excercise exec_option in test_recover_partitions.py

Before this patch, test_recover_partitions.py declare exec_option
dimension, but none of the test function exercise it. vector arg is
never used anywhere.

This patch makes each call of execute_query_expect_success in the tests
to pass exec_option from vector.get_value('exec_option').

Testing:
- Run and pass test_recover_partitions.py in exhaustive mode.
- Confirm in coordinator WebUI that there are queries run with either
  sync_ddl true and false.

Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
---
M tests/metadata/test_recover_partitions.py
1 file changed, 91 insertions(+), 70 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21796/1
-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16954/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Sep 2024 15:43:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................

IMPALA-13377: Excercise exec_option in test_recover_partitions.py

Before this patch, test_recover_partitions.py declare exec_option
dimension, but none of the test function exercise it. vector arg is
never used anywhere.

This patch fix the test by setting client configuration with given
vector.get_value('exec_option').

Testing:
- Run and pass test_recover_partitions.py in exhaustive mode.
- Confirm in coordinator log file that there are equal number of queries
  run with either sync_ddl true or false (95 each).

Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Reviewed-on: http://gerrit.cloudera.org:8080/21796
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/common/impala_test_suite.py
M tests/metadata/test_recover_partitions.py
2 files changed, 35 insertions(+), 10 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 7
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/1/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/1/tests/metadata/test_recover_partitions.py@157
PS1, Line 157:     self.execute_query_expect_success(self.client,
> Don't we need OPTIONS here?
Good catch! Done.



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Sep 2024 15:12:50 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 2: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/21796/2//COMMIT_MSG@11
PS2, Line 11: never used anywhere.
nit: Should we want to mention that passing exec-option to query helps better observability (e.g: to print more info) in the logs?



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Sep 2024 22:58:05 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16958/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Sep 2024 16:50:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py@69
PS3, Line 69:     OPTIONS = vector.get_value('exec_option')
I see we just use self.client here and all statements use the same query options. Maybe we can simply use this:

 self.client.set_configuration(vector.get_value('exec_option'))



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2024 02:28:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16941/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 13 Sep 2024 04:26:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/1/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/1/tests/metadata/test_recover_partitions.py@157
PS1, Line 157:     self.execute_query_expect_success(self.client,
Don't we need OPTIONS here?



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Mon, 16 Sep 2024 13:26:03 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Daniel Becker, Sai Hemanth Gantasala, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21796

to look at the new patch set (#2).

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................

IMPALA-13377: Excercise exec_option in test_recover_partitions.py

Before this patch, test_recover_partitions.py declare exec_option
dimension, but none of the test function exercise it. vector arg is
never used anywhere.

This patch makes each call of execute_query_expect_success in the tests
to pass exec_option from vector.get_value('exec_option').

Testing:
- Run and pass test_recover_partitions.py in exhaustive mode.
- Confirm in coordinator WebUI that there are queries run with either
  sync_ddl true and false.

Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
---
M tests/metadata/test_recover_partitions.py
1 file changed, 92 insertions(+), 71 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21796/2
-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 3:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/16957/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Sep 2024 15:51:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py@69
PS3, Line 69:     my_client = self.create_impala_client(protocol=vector.get_value('protocol'))
> I see we just use self.client here and all statements use the same query op
I like that method!

Unfortunately, concurrent test method will share self.client, so I make them create local client in ps4.



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2024 04:27:11 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2024 00:48:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/21796/2//COMMIT_MSG@11
PS2, Line 11: never used anywhere.
> nit: Should we want to mention that passing exec-option to query helps bett
Passing query_option argument is not a requirement for execute_query_expect_success and other similar methods in impala_test_suite.py. It is fully in test writer discretion whether they want to craft one, modify one from test vector, or use vector.get_value('exec_option') as it is.

ps 3 added documentation to remind about existence of vector.get_value('exec_option').



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Sep 2024 15:11:08 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 2: Code-Review+1

Thanks Riza!


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Sep 2024 12:58:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Daniel Becker, Sai Hemanth Gantasala, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21796

to look at the new patch set (#5).

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................

IMPALA-13377: Excercise exec_option in test_recover_partitions.py

Before this patch, test_recover_partitions.py declare exec_option
dimension, but none of the test function exercise it. vector arg is
never used anywhere.

This patch fix the test by setting client configuration with given
vector.get_value('exec_option').

Testing:
- Run and pass test_recover_partitions.py in exhaustive mode.
- Confirm in coordinator log file that there are equal number of queries
  run with either sync_ddl true or false (95 each).

Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
---
M tests/common/impala_test_suite.py
M tests/metadata/test_recover_partitions.py
2 files changed, 35 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21796/5
-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py@69
PS3, Line 69:     self.client.set_configuration(vector.get_value('exec_option'))
> Oh, I thought we use subprocess so it's safe. When running the parallel e2e
Sorry, my bad. I was looking SYNC_DDL=1 and found none.

grep SYNC_DDL=1 logs/cluster/impalad.INFO

But since I run the test using ./bin/run-all-tests.sh, minicluster was restarted and log into ee_test dir, so I should do this instead.

grep SYNC_DDL=1 logs/ee_tests/impalad.INFO

I can confirm that I see some with SYNC_DDL=1.



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2024 14:17:38 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Daniel Becker, Sai Hemanth Gantasala, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21796

to look at the new patch set (#4).

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................

IMPALA-13377: Excercise exec_option in test_recover_partitions.py

Before this patch, test_recover_partitions.py declare exec_option
dimension, but none of the test function exercise it. vector arg is
never used anywhere.

This patch makes each test method execution to create their own impala
client and set the configuration with given
vector.get_value('exec_option').

Testing:
- Run and pass test_recover_partitions.py in exhaustive mode.
- Confirm in coordinator WebUI that there are queries run with either
  sync_ddl true and false.

Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
---
M tests/common/impala_test_suite.py
M tests/metadata/test_recover_partitions.py
2 files changed, 108 insertions(+), 74 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21796/4
-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Daniel Becker, Sai Hemanth Gantasala, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/21796

to look at the new patch set (#3).

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................

IMPALA-13377: Excercise exec_option in test_recover_partitions.py

Before this patch, test_recover_partitions.py declare exec_option
dimension, but none of the test function exercise it. vector arg is
never used anywhere.

This patch makes each call of execute_query_expect_success in the tests
to pass exec_option from vector.get_value('exec_option').

Testing:
- Run and pass test_recover_partitions.py in exhaustive mode.
- Confirm in coordinator WebUI that there are queries run with either
  sync_ddl true and false.

Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
---
M tests/common/impala_test_suite.py
M tests/metadata/test_recover_partitions.py
2 files changed, 112 insertions(+), 74 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/21796/3
-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2024 05:57:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16978/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2024 14:47:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 5: Code-Review+2

(1 comment)

LGTM. Thanks for fixing this!

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py@69
PS3, Line 69:     self.client.set_configuration(vector.get_value('exec_option'))
> Sorry, my bad. I was looking SYNC_DDL=1 and found none.
Ack



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2024 00:47:55 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10961/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Sep 2024 00:48:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/16976/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2024 04:53:09 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13377: Excercise exec option in test recover partitions.py

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21796 )

Change subject: IMPALA-13377: Excercise exec_option in test_recover_partitions.py
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py
File tests/metadata/test_recover_partitions.py:

http://gerrit.cloudera.org:8080/#/c/21796/3/tests/metadata/test_recover_partitions.py@69
PS3, Line 69:     my_client = self.create_impala_client(protocol=vector.get_value('protocol'))
> I like that method!
Oh, I thought we use subprocess so it's safe. When running the parallel e2e tests, we use the "-n numprocesses" parameter of pytest:
https://github.com/apache/impala/blob/58fd45f20c22f8a062f603d6b96f62ee57d85ca9/tests/run-tests.py#L322

If it's using threads, lots of codes should change as well since 'self.client' is used, e.g.
https://github.com/apache/impala/blob/58fd45f20c22f8a062f603d6b96f62ee57d85ca9/tests/common/impala_test_suite.py#L923
https://github.com/apache/impala/blob/58fd45f20c22f8a062f603d6b96f62ee57d85ca9/tests/common/impala_test_suite.py#L1027-L1028

Do you see the query options are set incorrectly when not creating a new client?



-- 
To view, visit http://gerrit.cloudera.org:8080/21796
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4e938dd8667937c996854032a1e13184c62d7b48
Gerrit-Change-Number: 21796
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Sep 2024 08:56:26 +0000
Gerrit-HasComments: Yes