You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2018/07/18 01:22:07 UTC

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY 

Hello Tianyi Wang, Vuk Ercegovac,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/10974

to review the following change.


Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

A regression test is added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
8 files changed, 175 insertions(+), 77 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/10974/1
-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 4:

Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/35/ 

Running initial code review checks. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 24 Jul 2018 18:46:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10974/5/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
File fe/src/main/java/org/apache/impala/catalog/FeFsTable.java:

http://gerrit.cloudera.org:8080/#/c/10974/5/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@470
PS5, Line 470: / No explicit partition was specified. Need to ensure that write access is available
             :         // to all partitions.
> Why isn't hasWriteAccessToBaseDir needed here?
good catch -- guess we need to be able to write to all existing partitions as well as to create new ones in this case.



-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Jul 2018 01:23:36 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10974/5/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
File fe/src/main/java/org/apache/impala/catalog/FeFsTable.java:

http://gerrit.cloudera.org:8080/#/c/10974/5/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@470
PS5, Line 470: / No explicit partition was specified. Need to ensure that write access is available
             :         // to all partitions.
> good catch -- guess we need to be able to write to all existing partitions 
Fixed and added tests



-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 01 Aug 2018 22:15:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Tianyi Wang (Code Review)" <ge...@cloudera.org>.
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 9: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 9
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 Aug 2018 23:35:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2946/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 9
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 Aug 2018 20:01:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Tianyi Wang (Code Review)" <ge...@cloudera.org>.
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10974/5/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
File fe/src/main/java/org/apache/impala/catalog/FeFsTable.java:

http://gerrit.cloudera.org:8080/#/c/10974/5/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@470
PS5, Line 470: / No explicit partition was specified. Need to ensure that write access is available
             :         // to all partitions.
Why isn't hasWriteAccessToBaseDir needed here?



-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Mon, 30 Jul 2018 23:50:50 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/142/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 01 Aug 2018 23:30:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2831/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Jul 2018 04:59:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Vuk Ercegovac, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/10974

to look at the new patch set (#4).

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

A regression test is added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
7 files changed, 177 insertions(+), 77 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/10974/4
-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

New regression tests are added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Reviewed-on: http://gerrit.cloudera.org:8080/10974
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Tianyi Wang <tw...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
8 files changed, 221 insertions(+), 61 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Tianyi Wang: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 10
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 9: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 9
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 Aug 2018 23:29:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 8: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2942/


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 8
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 Aug 2018 17:30:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/10974

to look at the new patch set (#6).

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

New regression tests are added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
7 files changed, 219 insertions(+), 77 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/10974/6
-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/10974

to look at the new patch set (#5).

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

A regression test is added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
7 files changed, 175 insertions(+), 75 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/10974/5
-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/10974

to look at the new patch set (#7).

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

New regression tests are added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
7 files changed, 219 insertions(+), 59 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/10974/7
-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Tianyi Wang (Code Review)" <ge...@cloudera.org>.
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Aug 2018 01:32:02 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 9:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/225/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 9
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 Aug 2018 20:46:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 8:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2942/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 8
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 07 Aug 2018 14:34:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/63/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 21:03:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/156/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 02 Aug 2018 04:04:20 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Tianyi Wang, Impala Public Jenkins, Vuk Ercegovac, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/10974

to look at the new patch set (#9).

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................

IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY

This changes the permissions-checking of INSERT so that, if a partition
is specified, we only verify writability of the specific explicit
partition. This allows insertion into a table even if it contains one or
more read-only partitions. This matches the existing behavior of LOAD
DATA.

New regression tests are added which failed prior to the fix.

Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
---
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/metadata/test_hdfs_permissions.py
M tests/query_test/test_insert_behaviour.py
8 files changed, 221 insertions(+), 61 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/10974/9
-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 9
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 5:

Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/63/ 

Running initial code review checks. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Jul 2018 20:07:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 6:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/10974/6/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
File fe/src/main/java/org/apache/impala/catalog/FeFsTable.java:

http://gerrit.cloudera.org:8080/#/c/10974/6/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@435
PS6, Line 435:      * If 'partitionKeyValues' is null, the user should have write access to all partitions
line too long (91 > 90)


http://gerrit.cloudera.org:8080/#/c/10974/6/fe/src/main/java/org/apache/impala/catalog/FeFsTable.java@470
PS6, Line 470:         // No explicit partition was specified. Need to ensure that write access is available
line too long (93 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 01 Aug 2018 22:34:36 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/35/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Tue, 24 Jul 2018 19:33:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ ONLY

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10974 )

Change subject: IMPALA-7311. Allow INSERT on writable partitions even if some other partition is READ_ONLY
......................................................................


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2831/


-- 
To view, visit http://gerrit.cloudera.org:8080/10974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1dd81100ae73fcabdbfaf679c20cea7dc102cd13
Gerrit-Change-Number: 10974
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Jul 2018 06:35:11 +0000
Gerrit-HasComments: No