You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Michael Smith (Code Review)" <ge...@cloudera.org> on 2023/02/13 22:28:30 UTC

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19497


Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................

IMPALA-11633: Use longer ctas timeout for S3/Ozone

File reads can be slower with S3 and Ozone in minicluster because
they're not using shortcircuiting to directly access the file on disk.
Allow more time for CTAS operations to complete when using a non-HDFS
filesystem.

Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
---
M tests/metadata/test_ddl.py
M tests/metadata/test_load.py
2 files changed, 6 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/19497/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................

IMPALA-11633: Use longer ctas timeout for S3/Ozone

File reads can be slower with S3 and Ozone in minicluster because
they're not using shortcircuiting to directly access the file on disk.
Allow more time for CTAS operations to complete when using a non-HDFS
filesystem.

Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Reviewed-on: http://gerrit.cloudera.org:8080/19497
Reviewed-by: Andrew Sherman <as...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M tests/metadata/test_ddl.py
M tests/metadata/test_load.py
2 files changed, 7 insertions(+), 3 deletions(-)

Approvals:
  Andrew Sherman: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2:

Test run hit an S3 outage, retrying.


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Mar 2023 17:22:02 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has removed a vote on this change.

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has removed a vote on this change.

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Removed -Verified by Impala Public Jenkins <im...@cloudera.com>
-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19497/1/tests/metadata/test_load.py
File tests/metadata/test_load.py:

http://gerrit.cloudera.org:8080/#/c/19497/1/tests/metadata/test_load.py@226
PS1, Line 226: I
flake8: F821 undefined name 'IS_HDFS'



-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Feb 2023 22:29:22 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Mar 2023 22:25:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9093/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 01 Mar 2023 22:47:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9093/


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Mar 2023 03:29:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2: Code-Review+2

LGTM


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 01 Mar 2023 22:43:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12367/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Feb 2023 22:49:01 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Fang-Yu Rao, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19497

to look at the new patch set (#2).

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................

IMPALA-11633: Use longer ctas timeout for S3/Ozone

File reads can be slower with S3 and Ozone in minicluster because
they're not using shortcircuiting to directly access the file on disk.
Allow more time for CTAS operations to complete when using a non-HDFS
filesystem.

Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
---
M tests/metadata/test_ddl.py
M tests/metadata/test_load.py
2 files changed, 7 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/97/19497/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12368/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Feb 2023 22:51:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11633: Use longer ctas timeout for S3/Ozone

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19497 )

Change subject: IMPALA-11633: Use longer ctas timeout for S3/Ozone
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9097/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/19497
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7a49cd44fc0fdc238313b92341467818a8f37fd1
Gerrit-Change-Number: 19497
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Mar 2023 17:22:21 +0000
Gerrit-HasComments: No