You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2019/04/26 17:32:51 UTC

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog 

Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13131


Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................

IMPALA-8463: fix skip.header.line.count on local catalog

This adds missing logic based on the existing implementation
for the traditional catalog tables.

Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.

Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
---
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
2 files changed, 24 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/13131/1
-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13131/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java:

http://gerrit.cloudera.org:8080/#/c/13131/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@524
PS1, Line 524:     org.apache.hadoop.hive.metastore.api.Table msTbl = getMetaStoreTable();
Can we just move the code from HdfsTable.java into FeFsTable.Utils and call it from here? Or, now that we are Java8-capable, move it into FeFsTable as a default interface method?



-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 19:31:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Todd Lipcon, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13131

to look at the new patch set (#3).

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................

IMPALA-8463: fix skip.header.line.count on local catalog

This adds missing logic based on the existing implementation
for the traditional catalog tables.

Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.

Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
5 files changed, 63 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/13131/3
-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 7: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 30 Apr 2019 05:51:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 29 Apr 2019 23:46:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 6:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4108/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 29 Apr 2019 23:47:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13131/2/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
File fe/src/main/java/org/apache/impala/catalog/FeFsTable.java:

http://gerrit.cloudera.org:8080/#/c/13131/2/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@501
PS2, Line 501:       Preconditions.checkState(tblProperties.containsKey(TBL_PROP_SKIP_HEADER_LINE_COUNT));
line too long (91 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 20:39:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 7: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 30 Apr 2019 11:29:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 29 Apr 2019 23:47:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4112/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 30 Apr 2019 05:51:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Todd Lipcon, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13131

to look at the new patch set (#2).

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................

IMPALA-8463: fix skip.header.line.count on local catalog

This adds missing logic based on the existing implementation
for the traditional catalog tables.

Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.

Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
5 files changed, 62 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/13131/2
-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 6: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4108/


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Tue, 30 Apr 2019 01:25:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2940/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 21:33:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13131/4/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:

http://gerrit.cloudera.org:8080/#/c/13131/4/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1122
PS4, Line 1122:   public static int parseSkipHeaderLineCount(Map<String, String> tblProperties,
can this copy of the function be removed now?



-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 21:03:41 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Todd Lipcon, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13131

to look at the new patch set (#4).

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................

IMPALA-8463: fix skip.header.line.count on local catalog

This moves the logic to FeFsTable and calls it from
LocalFsTable.

Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.

Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
5 files changed, 63 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/13131/4
-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2942/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 22:58:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................

IMPALA-8463: fix skip.header.line.count on local catalog

This moves the logic to FeFsTable and calls it from
LocalFsTable.

Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.

Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Reviewed-on: http://gerrit.cloudera.org:8080/13131
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
5 files changed, 57 insertions(+), 51 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13131/4/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:

http://gerrit.cloudera.org:8080/#/c/13131/4/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1122
PS4, Line 1122:       org.apache.hadoop.hive.metastore.api.Table msTbl) throws Exception {
> can this copy of the function be removed now?
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 22:06:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2937/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 18:22:01 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13131/2/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
File fe/src/main/java/org/apache/impala/catalog/FeFsTable.java:

http://gerrit.cloudera.org:8080/#/c/13131/2/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@501
PS2, Line 501: 
> line too long (91 > 90)
Done


http://gerrit.cloudera.org:8080/#/c/13131/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java:

http://gerrit.cloudera.org:8080/#/c/13131/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@524
PS1, Line 524:     org.apache.hadoop.hive.metastore.api.Table msTbl = getMetaStoreTable();
> Can we just move the code from HdfsTable.java into FeFsTable.Utils and call
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 20:43:50 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 5: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 29 Apr 2019 21:38:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13131 )

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2939/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 26 Apr 2019 21:26:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8463: fix skip.header.line.count on local catalog

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Todd Lipcon, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13131

to look at the new patch set (#5).

Change subject: IMPALA-8463: fix skip.header.line.count on local catalog
......................................................................

IMPALA-8463: fix skip.header.line.count on local catalog

This moves the logic to FeFsTable and calls it from
LocalFsTable.

Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.

Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
5 files changed, 57 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/13131/5
-- 
To view, visit http://gerrit.cloudera.org:8080/13131
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Gerrit-Change-Number: 13131
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>