You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org> on 2021/11/17 21:08:52 UTC

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file 

Fang-Yu Rao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18033


Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
---
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
7 files changed, 292 insertions(+), 6 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18033/1
-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 3:

I updated rat_exclude_files.txt so that sfs_d2.txt and sfs_d4.txt could be added.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:52:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Hello Aman Sinha, kirk@rxd.hu, Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18033

to look at the new patch set (#3).

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
---
M bin/rat_exclude_files.txt
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
8 files changed, 296 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18033/3
-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7:

> Patch Set 7:
> 
> > Patch Set 6:
> > 
> > > Patch Set 6:
> > > 
> > > (1 comment)
> > > 
> > > > Patch Set 6: Verified-1
> > > > 
> > > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/
> > > 
> > > Fang-Yu, could you address the LOCATION related failure as mentioned below?
> > > More details of the error: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5080/
> > 
> > Taking a closer look at the console output of the Jenkins job, I think I should use $DEFAULT_FS to replace each occurrence of "hdfs://localhost:20500" since $DEFAULT_FS is not necessarily "hdfs://localhost:20500" in every test environment.
> 
> In the updated patch set (patch set 7), I replaced each occurrence of "localhost" with $INTERNAL_LISTEN_HOST since this is a string that will be replaced in our test framework according to https://github.com/apache/impala/blob/master/tests/common/impala_test_suite.py#L534L535.

The link above was wrong. I think it should be https://github.com/apache/impala/blob/master/tests/common/impala_test_suite.py#L470 instead. But the patch should still pass according to my current understanding.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 20:29:19 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9962/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 20:30:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 6:

> Patch Set 6:
> 
> (1 comment)
> 
> > Patch Set 6: Verified-1
> > 
> > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/
> 
> Fang-Yu, could you address the LOCATION related failure as mentioned below?
> More details of the error: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5080/

Thanks Aman! I will take a closer look at this failed test and see how to fix the test.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 6
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 06:03:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7: Code-Review+2

Carry forward prior +2


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 05 Jan 2022 03:23:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 5: Code-Review+1

Test coverage and the test output looks good to me.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 5
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Dec 2021 02:24:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
---
M bin/rat_exclude_files.txt
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
8 files changed, 360 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18033/7
-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 6
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 01:40:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7:

> Patch Set 6:
> 
> > Patch Set 6:
> > 
> > (1 comment)
> > 
> > > Patch Set 6: Verified-1
> > > 
> > > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/
> > 
> > Fang-Yu, could you address the LOCATION related failure as mentioned below?
> > More details of the error: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5080/
> 
> Taking a closer look at the console output of the Jenkins job, I think I should use $DEFAULT_FS to replace each occurrence of "hdfs://localhost:20500" since $DEFAULT_FS is not necessarily "hdfs://localhost:20500" in every test environment.

In the updated patch set (patch set 7), I replaced each occurrence of "localhost" with $INTERNAL_LISTEN_HOST since this is a string that will be replaced in our test framework according to https://github.com/apache/impala/blob/master/tests/common/impala_test_suite.py#L534L535.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 20:11:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Reviewed-on: http://gerrit.cloudera.org:8080/18033
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Aman Sinha <am...@cloudera.com>
---
M bin/rat_exclude_files.txt
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
8 files changed, 360 insertions(+), 6 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Aman Sinha: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 9
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 05 Jan 2022 02:49:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 8:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9967/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 8
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 05 Jan 2022 03:55:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py
File tests/query_test/test_sfs.py:

http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@25
PS1, Line 25: class TestSFS(ImpalaTestSuite):
flake8: E302 expected 2 blank lines, found 1


http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@37
PS1, Line 37: =
flake8: E712 comparison to False should be 'if cond is False:' or 'if not cond:'


http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@45
PS1, Line 45: )
flake8: E501 line too long (91 > 90 characters)


http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@50
PS1, Line 50: 
flake8: W292 no newline at end of file



-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:10:16 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9800/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 22:15:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 1:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/9797/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:32:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 4:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/9940/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Thu, 16 Dec 2021 17:29:44 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9941/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 5
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Thu, 16 Dec 2021 19:26:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7:

Hi all, I have fixed sfs.test to not hardcode the host name to "localhost". Hopefully this time the pre-commit tests could pass.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 20:12:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
---
M bin/rat_exclude_files.txt
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
8 files changed, 358 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18033/4
-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
---
M bin/rat_exclude_files.txt
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
8 files changed, 358 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18033/5
-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 5
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 2:

(4 comments)

I have revised the previous patch according to the comments from flake8.

http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py
File tests/query_test/test_sfs.py:

http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@25
PS1, Line 25: 
> flake8: E302 expected 2 blank lines, found 1
Done


http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@37
PS1, Line 37: 
> flake8: E712 comparison to False should be 'if cond is False:' or 'if not c
Done


http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@45
PS1, Line 45: 
> flake8: E501 line too long (91 > 90 characters)
Done


http://gerrit.cloudera.org:8080/#/c/18033/1/tests/query_test/test_sfs.py@50
PS1, Line 50: r
> flake8: W292 no newline at end of file
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 2
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:21:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7714/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 7
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 20:27:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 6:

(1 comment)

> Patch Set 6: Verified-1
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/

Fang-Yu, could you address the LOCATION related failure as mentioned below?
More details of the error: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5080/

http://gerrit.cloudera.org:8080/#/c/18033/6/testdata/workloads/functional-query/queries/QueryTest/sfs.test
File testdata/workloads/functional-query/queries/QueryTest/sfs.test:

http://gerrit.cloudera.org:8080/#/c/18033/6/testdata/workloads/functional-query/queries/QueryTest/sfs.test@4
PS6, Line 4: LOCATION 'sfs+hdfs://localhost:20500/test-warehouse/$DATABASE.db/sfs_d1.parq/#SINGLEFILE#'
The gerrit-verify run failed due to LOCATION pointing to localhost. I believe this should be using the $FILESYSTEM_PREFIX parameterized variable.



-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 6
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 03:13:25 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 6
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Jan 2022 19:20:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 2:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/9798/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 2
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:40:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Hello Aman Sinha, kirk@rxd.hu, Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18033

to look at the new patch set (#2).

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................

IMPALA-10934 (Part 2): Enable table definition over a single file

This patch adds an end-to-end test to validate and characterize HMS'
behavior with respect to external table creation after HIVE-25569 via
which a user is allowed to create an external table associated with a
single file.

Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
---
A testdata/data/sfs_d1.parq
A testdata/data/sfs_d2.txt
A testdata/data/sfs_d3.parq
A testdata/data/sfs_d4.txt
A testdata/workloads/functional-query/queries/QueryTest/sfs.test
M tests/common/file_utils.py
A tests/query_test/test_sfs.py
7 files changed, 294 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18033/2
-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 2
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 1:

Hi all, please review this patch and let me know if you have any suggestion or comment. Thanks!


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:11:01 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 3:

(2 comments)

These are useful tests. I have a couple small comments, but I think we can go forward with this and part 1.

http://gerrit.cloudera.org:8080/#/c/18033/3/testdata/workloads/functional-query/queries/QueryTest/sfs.test
File testdata/workloads/functional-query/queries/QueryTest/sfs.test:

http://gerrit.cloudera.org:8080/#/c/18033/3/testdata/workloads/functional-query/queries/QueryTest/sfs.test@38
PS3, Line 38: ---- QUERY
            : INSERT INTO TABLE test_tbl_01 VALUES ('x', 100);
            : ---- CATCH
            : row_regex: .*Unable to INSERT into target table .+ because .+ is not a supported filesystem.*
            : ====
            : ---- QUERY
            : INSERT INTO TABLE test_tbl_02 VALUES ('x', 100);
            : ---- CATCH
            : row_regex: .*Unable to INSERT into target table .+ because .+ is not a supported filesystem.*
            : ====
            : ---- QUERY
            : LOAD DATA INPATH 'hdfs://localhost:20500/test-warehouse/$DATABASE.db/sfs_d3.parq' INTO TABLE test_tbl_01
            : ---- CATCH
            : Unsupported SFS filesystem operation!
            : ====
            : ---- QUERY
            : LOAD DATA INPATH 'hdfs://localhost:20500/test-warehouse/$DATABASE.db/sfs_d4.txt' INTO TABLE test_tbl_02
            : ---- CATCH
            : Unsupported SFS filesystem operation!
            : ====
            : ---- QUERY
            : LOAD DATA INPATH 'sfs+hdfs://localhost:20500/test-warehouse/$DATABASE.db/sfs_d3.parq/#SINGLEFILE#' INTO TABLE test_tbl_01
            : ---- CATCH
            : row_regex: .*INPATH location .+ must point to one of the supported filesystem URI scheme.*
            : ====
            : ---- QUERY
            : LOAD DATA INPATH 'sfs+hdfs://localhost:20500/test-warehouse/$DATABASE.db/sfs_d4.txt/#SINGLEFILE#' INTO TABLE test_tbl_02
            : ---- CATCH
            : row_regex: .*INPATH location .+ must point to one of the supported filesystem URI scheme.*
Let's add a case for TRUNCATE TABLE. I'm guessing it should fail and not delete the file?

https://impala.apache.org/docs/build/html/topics/impala_truncate_table.html

Let's also call COMPUTE STATS just to make sure it works.


http://gerrit.cloudera.org:8080/#/c/18033/3/testdata/workloads/functional-query/queries/QueryTest/sfs.test@157
PS3, Line 157: ---- QUERY
             : DROP TABLE test_tbl_03;
             : DROP TABLE test_tbl_04
It would be nice to have a test case that verifies that the files are still there. Maybe create another table that references the same location and select to show it is still there.



-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 15 Dec 2021 21:32:08 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 5: Code-Review+2

Thanks for adding the extra test cases, this looks good.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 5
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Dec 2021 18:48:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/9961/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 6
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Jan 2022 19:41:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10934 (Part 2): Enable table definition over a single file

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18033 )

Change subject: IMPALA-10934 (Part 2): Enable table definition over a single file
......................................................................


Patch Set 6:

> Patch Set 6:
> 
> (1 comment)
> 
> > Patch Set 6: Verified-1
> > 
> > Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7712/
> 
> Fang-Yu, could you address the LOCATION related failure as mentioned below?
> More details of the error: https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5080/

Taking a closer look at the console output of the Jenkins job, I think I should use $DEFAULT_FS to replace each occurrence of "hdfs://localhost:20500" since $DEFAULT_FS is not necessarily "hdfs://localhost:20500" in every test environment.


-- 
To view, visit http://gerrit.cloudera.org:8080/18033
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia4f57f07a9f543c660b102ebf307a6cf590a6784
Gerrit-Change-Number: 18033
Gerrit-PatchSet: 6
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 04 Jan 2022 17:30:08 +0000
Gerrit-HasComments: No