You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Fredy Wijaya (Code Review)" <ge...@cloudera.org> on 2018/11/08 19:00:46 UTC
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11910
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Before this patch, enabling object ownership and running INVALIDATE
METADATA for the role and user catalog objects with the same principal
name could cause INVALIDATE METADATA to hang because a principal name
without a type is not guaranteed to be unique. This causes the catalog
delta logic to assume principal catalog objects with the same name but
with different types are considered the same causing a catalog version
inconsistency. This patch makes the principal object key to be unique by
appending the principal type information.
Testing:
- Added E2E authorization test
- Ran all FE tests
- Ran all E2E authorization tests
Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
---
M be/src/catalog/catalog-util.cc
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/Principal.java
M tests/authorization/test_authorization.py
M tests/authorization/test_grant_revoke.py
5 files changed, 72 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/11910/5
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Nov 2018 03:37:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 6:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc
File be/src/catalog/catalog-util.cc:
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc@206
PS5, Line 206: // The format is <principal name>.<principal type>
> is this what Catalog.java is producing? If so, its ok that principal name b
Yeah that's how the catalog determines the object_type. So the first part of the key is used to determine the object_type.
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java@568
PS5, Line 568: return "PRINCIPAL:" + principalName + "." +
> why is getUniqueName not used here? if it can't be, this looks ripe for fac
Yup, this is definitely a repetition. With the existing design, it can take a bit of refactoring to do that. But I totally agree that the whole method here needs to be refactored to use getUniqueName(). I can do the follow up JIRA to clean it up if that's okay with you.
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java
File fe/src/main/java/org/apache/impala/catalog/Principal.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java@163
PS5, Line 163: getName().toLowerCase()
> I see this is done just for the role case now, why? I understood the issue
It's because user name is supposed to be case sensitive. In other words, `foo` and `FOO` users should be considered two different objects. I'll update the test.
http://gerrit.cloudera.org:8080/#/c/11910/5/tests/authorization/test_grant_revoke.py
File tests/authorization/test_grant_revoke.py:
http://gerrit.cloudera.org:8080/#/c/11910/5/tests/authorization/test_grant_revoke.py@351
PS5, Line 351: role_name, grp.getgrnam(getuser()).gr_name))
> nit: missing "to"
Done
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 20:41:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1332/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 21:17:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 6:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc
File be/src/catalog/catalog-util.cc:
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc@206
PS5, Line 206: // The format is <principal name>.<principal type>
> Yeah that's how the catalog determines the object_type. So the first part o
ok, add a pointer to the java code to make it easier to track down how this is created.
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java@568
PS5, Line 568: return "PRINCIPAL:" + principalName + "." +
> Yup, this is definitely a repetition. With the existing design, it can take
that's fine. pls add a todo/jira.
http://gerrit.cloudera.org:8080/#/c/11910/6/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/11910/6/fe/src/main/java/org/apache/impala/catalog/Catalog.java@564
PS6, Line 564: String principalName =
simpler?
String principalName = catalogObject.getPrincipal().getPrincipal_name();
if (catalogObject.getPrincipal().getPrincipal_type() == TPrincipalType.ROLE) {
principalName = principalName.toLowerCase();
}
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java
File fe/src/main/java/org/apache/impala/catalog/Principal.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java@163
PS5, Line 163: getName().toLowerCase()
> It's because user name is supposed to be case sensitive. In other words, `f
makes sense.. that sounds like further fallout from that case-sensitivity issue. I'm ok with making these two fixes here, but just wanted to clarify that this issue (case sensitivity) is not the cause of this particular hang.
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 21:32:38 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3447/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 23:47:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 22:45:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1334/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 23:01:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Before this patch, enabling object ownership and running INVALIDATE
METADATA for the role and user catalog objects with the same principal
name could cause INVALIDATE METADATA to hang because a principal name
without a type is not guaranteed to be unique. This causes the catalog
delta logic to assume principal catalog objects with the same name but
with different types are considered the same causing a catalog version
inconsistency. This patch makes the principal object key to be unique by
appending the principal type information.
Testing:
- Added E2E authorization test
- Ran all FE tests
- Ran all E2E authorization tests
Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Reviewed-on: http://gerrit.cloudera.org:8080/11910
Reviewed-by: Vuk Ercegovac <ve...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/catalog/catalog-util.cc
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/Principal.java
M tests/authorization/test_authorization.py
M tests/authorization/test_grant_revoke.py
5 files changed, 88 insertions(+), 11 deletions(-)
Approvals:
Vuk Ercegovac: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 8
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Vuk Ercegovac (Code Review)" <ge...@cloudera.org>.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 5:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc
File be/src/catalog/catalog-util.cc:
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc@206
PS5, Line 206: // The format is <principal name>.<principal type>
is this what Catalog.java is producing? If so, its ok that principal name be prefixed with a "PRINCIPAL:" string?
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java@568
PS5, Line 568: return "PRINCIPAL:" + principalName + "." +
why is getUniqueName not used here? if it can't be, this looks ripe for factoring.
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java
File fe/src/main/java/org/apache/impala/catalog/Principal.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java@163
PS5, Line 163: getName().toLowerCase()
I see this is done just for the role case now, why? I understood the issue here to be that a lack of a principal type caused unintentional duplication.
http://gerrit.cloudera.org:8080/#/c/11910/5/tests/authorization/test_grant_revoke.py
File tests/authorization/test_grant_revoke.py:
http://gerrit.cloudera.org:8080/#/c/11910/5/tests/authorization/test_grant_revoke.py@351
PS5, Line 351: # Verify that running invalidate metadata won't hang due having the same name
nit: missing "to"
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 20:01:39 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Before this patch, enabling object ownership and running INVALIDATE
METADATA for the role and user catalog objects with the same principal
name could cause INVALIDATE METADATA to hang because a principal name
without a type is not guaranteed to be unique. This causes the catalog
delta logic to assume principal catalog objects with the same name but
with different types are considered the same causing a catalog version
inconsistency. This patch makes the principal object key to be unique by
appending the principal type information.
Testing:
- Added E2E authorization test
- Ran all FE tests
- Ran all E2E authorization tests
Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
---
M be/src/catalog/catalog-util.cc
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/Principal.java
M tests/authorization/test_authorization.py
M tests/authorization/test_grant_revoke.py
5 files changed, 85 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/11910/6
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 6
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 7:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc
File be/src/catalog/catalog-util.cc:
http://gerrit.cloudera.org:8080/#/c/11910/5/be/src/catalog/catalog-util.cc@206
PS5, Line 206: break;
> ok, add a pointer to the java code to make it easier to track down how this
Done
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Catalog.java@568
PS5, Line 568: }
> that's fine. pls add a todo/jira.
Done
http://gerrit.cloudera.org:8080/#/c/11910/6/fe/src/main/java/org/apache/impala/catalog/Catalog.java
File fe/src/main/java/org/apache/impala/catalog/Catalog.java:
http://gerrit.cloudera.org:8080/#/c/11910/6/fe/src/main/java/org/apache/impala/catalog/Catalog.java@564
PS6, Line 564: // to have the same name for both role and user.
> simpler?
Done
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java
File fe/src/main/java/org/apache/impala/catalog/Principal.java:
http://gerrit.cloudera.org:8080/#/c/11910/5/fe/src/main/java/org/apache/impala/catalog/Principal.java@163
PS5, Line 163: getName().toLowerCase()
> makes sense.. that sounds like further fallout from that case-sensitivity i
Yup, case sensitivity isn't the issue. I just thought it would also be nice to add a test case here.
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 22:19:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Before this patch, enabling object ownership and running INVALIDATE
METADATA for the role and user catalog objects with the same principal
name could cause INVALIDATE METADATA to hang because a principal name
without a type is not guaranteed to be unique. This causes the catalog
delta logic to assume principal catalog objects with the same name but
with different types are considered the same causing a catalog version
inconsistency. This patch makes the principal object key to be unique by
appending the principal type information.
Testing:
- Added E2E authorization test
- Ran all FE tests
- Ran all E2E authorization tests
Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
---
M be/src/catalog/catalog-util.cc
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/Principal.java
M tests/authorization/test_authorization.py
M tests/authorization/test_grant_revoke.py
5 files changed, 88 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/11910/7
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 7
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
[Impala-ASF-CR] IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11910 )
Change subject: IMPALA-7835: Role and user catalog objects with the same name can cause INVALIDATE METADATA to hang
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1331/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I516cf72e69e142a1349950cfca91f035c1ed445f
Gerrit-Change-Number: 11910
Gerrit-PatchSet: 5
Gerrit-Owner: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Vuk Ercegovac <ve...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Nov 2018 19:36:34 +0000
Gerrit-HasComments: No