You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2025/07/15 03:49:45 UTC

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23173


Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................

IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Some other tests like tests/query_test/test_cancellation.py might create
tables under the tpch db, which fails the assertion in TestWarmupCatalog
assuming that there are 8 tables under it.

This fixes the test by fetching the table list of tpch db in runtime
instead of hard-coding them.

Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
---
M tests/custom_cluster/test_restart_services.py
1 file changed, 20 insertions(+), 15 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/23173/1
-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23173/1/tests/custom_cluster/test_restart_services.py
File tests/custom_cluster/test_restart_services.py:

http://gerrit.cloudera.org:8080/#/c/23173/1/tests/custom_cluster/test_restart_services.py@1180
PS1, Line 1180:       for i in range(1, 1 + num_tpch_tables):
              :         assert "tpch." in logs[i]
              :       assert "tpcds.customer" in logs[num_tpch_tables + 1]
              :       assert "tpcds.date_dim" in logs[num_tpch_tables + 2]
              :       assert "tpcds.item" in logs[num_tpch_tables + 3]
              :       assert "functional.#" in logs[num_tpch_tables + 4]
              :       assert "functional.alltypes etc #" in logs[num_tpch_tables + 5]
I'm worried if this enumeration will become flaky in the future. What if you simply join logs into single string and assert over that single string instead?



-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 04:04:26 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23173/1/tests/custom_cluster/test_restart_services.py
File tests/custom_cluster/test_restart_services.py:

http://gerrit.cloudera.org:8080/#/c/23173/1/tests/custom_cluster/test_restart_services.py@1180
PS1, Line 1180:       for i in range(1, 1 + num_tpch_tables):
              :         assert "tpch." in logs[i]
              :       assert "tpcds.customer" in logs[num_tpch_tables + 1]
              :       assert "tpcds.date_dim" in logs[num_tpch_tables + 2]
              :       assert "tpcds.item" in logs[num_tpch_tables + 3]
              :       assert "functional.#" in logs[num_tpch_tables + 4]
              :       assert "functional.alltypes etc #" in logs[num_tpch_tables + 5]
> I'm worried if this enumeration will become flaky in the future. What if yo
We need to verify the scheduling order is exactly their order in the config file. Joining logs into a single string might be hard to verify this.



-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 04:15:07 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 1:

> Please test using downstream Ozone jenkins and carry my vote if it pass.

The downstream Ozone build passed. Merging this. Thanks for the quick review!


-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 11:17:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 2: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 11:17:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................

IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Some other tests like tests/query_test/test_cancellation.py might create
tables under the tpch db, which fails the assertion in TestWarmupCatalog
assuming that there are 8 tables under it.

This fixes the test by fetching the table list of tpch db in runtime
instead of hard-coding them.

Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Reviewed-on: http://gerrit.cloudera.org:8080/23173
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/custom_cluster/test_restart_services.py
1 file changed, 20 insertions(+), 15 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/12180/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 11:17:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 17:23:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/19875/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 04:17:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23173 )

Change subject: IMPALA-14221: Avoid hard-coding table list in TestWarmupCatalog
......................................................................


Patch Set 1: Code-Review+2

(1 comment)

Please test using downstream Ozone jenkins and carry my vote if it pass.

http://gerrit.cloudera.org:8080/#/c/23173/1/tests/custom_cluster/test_restart_services.py
File tests/custom_cluster/test_restart_services.py:

http://gerrit.cloudera.org:8080/#/c/23173/1/tests/custom_cluster/test_restart_services.py@1180
PS1, Line 1180:       for i in range(1, 1 + num_tpch_tables):
              :         assert "tpch." in logs[i]
              :       assert "tpcds.customer" in logs[num_tpch_tables + 1]
              :       assert "tpcds.date_dim" in logs[num_tpch_tables + 2]
              :       assert "tpcds.item" in logs[num_tpch_tables + 3]
              :       assert "functional.#" in logs[num_tpch_tables + 4]
              :       assert "functional.alltypes etc #" in logs[num_tpch_tables + 5]
> We need to verify the scheduling order is exactly their order in the config
Ack.



-- 
To view, visit http://gerrit.cloudera.org:8080/23173
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I0aca8ee19146f2e63e7cd82177d9fce0b8c6736a
Gerrit-Change-Number: 23173
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 15 Jul 2025 04:22:12 +0000
Gerrit-HasComments: Yes