You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org> on 2020/05/18 11:18:50 UTC
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15940
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
IMPALA-9753: Fix TRUNCATE of ACID tables on S3
The use of HDFS API was incorrect when creating an empty file
in the new base dir during truncate. Simply calling Create(Path)
does create the file in HDFS, but it is only created on S3 when
the returned stream is closed.
Testing:
- Acid truncate tests are not running on S3 as they need a running
Hive server. Aded a regression test that will run on S3 too. It
would be nice to run all tests on S3, but this is out of the scope
of this change.
Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A testdata/workloads/functional-query/queries/QueryTest/acid-no-hive.test
M tests/query_test/test_acid.py
3 files changed, 20 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15940/1
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
IMPALA-9753: Fix TRUNCATE of ACID tables on S3
The use of HDFS API was incorrect when creating an empty file
in the new base dir during truncate. Simply calling Create(Path)
does create the file in HDFS, but it is only created on S3 when
the returned stream is closed.
Testing:
- Acid truncate tests are not running on S3 as they need a running
Hive server. Aded a regression test that will run on S3 too. It
would be nice to run all tests on S3, but this is out of the scope
of this change.
Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A testdata/workloads/functional-query/queries/QueryTest/acid-no-hive.test
M tests/query_test/test_acid.py
3 files changed, 21 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/15940/3
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 1:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15940/1/tests/query_test/test_acid.py
File tests/query_test/test_acid.py:
http://gerrit.cloudera.org:8080/#/c/15940/1/tests/query_test/test_acid.py@53
PS1, Line 53:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/15940/1/tests/query_test/test_acid.py@53
PS1, Line 53:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/15940/1/tests/query_test/test_acid.py@58
PS1, Line 58:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/15940/1/tests/query_test/test_acid.py@58
PS1, Line 58: """
line has trailing whitespace
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 11:19:34 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 3: Code-Review+2
Carry +2 from Zoltan
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 11:21:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
IMPALA-9753: Fix TRUNCATE of ACID tables on S3
The use of HDFS API was incorrect when creating an empty file
in the new base dir during truncate. Simply calling Create(Path)
does create the file in HDFS, but it is only created on S3 when
the returned stream is closed.
Testing:
- Acid truncate tests are not running on S3 as they need a running
Hive server. Aded a regression test that will run on S3 too. It
would be nice to run all tests on S3, but this is out of the scope
of this change.
Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Reviewed-on: http://gerrit.cloudera.org:8080/15940
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
A testdata/workloads/functional-query/queries/QueryTest/acid-no-hive.test
M tests/query_test/test_acid.py
3 files changed, 21 insertions(+), 1 deletion(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 11:22:13 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6090/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 12:14:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 16:51:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6089/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 1
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 12:11:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9753: Fix TRUNCATE of ACID tables on S3
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15940 )
Change subject: IMPALA-9753: Fix TRUNCATE of ACID tables on S3
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5855/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/15940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I96d315638b669c5c7198a8e47939cb2b236e35bb
Gerrit-Change-Number: 15940
Gerrit-PatchSet: 4
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 11:22:14 +0000
Gerrit-HasComments: No