You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/05/07 05:39:26 UTC
[GitHub] [hive] simhadri-g opened a new pull request #1007: Hive 23361
simhadri-g opened a new pull request #1007:
URL: https://github.com/apache/hive/pull/1007
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r427038003
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -78,6 +83,9 @@ private void addACLsToBag(
for (Map.Entry<String, Map<HiveResourceACLs.Privilege, HiveResourceACLs.AccessResult>> principalAcls
: principalAclsMap.entrySet()) {
String principal = principalAcls.getKey();
+ int[] columnPrivilegeBits = new int[] {0, 0, 0, 0, 0, 0, 0, 0, 0};
Review comment:
The integer array is later changed to string of privilege bits that are updated in metastore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r427038003
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -78,6 +83,9 @@ private void addACLsToBag(
for (Map.Entry<String, Map<HiveResourceACLs.Privilege, HiveResourceACLs.AccessResult>> principalAcls
: principalAclsMap.entrySet()) {
String principal = principalAcls.getKey();
+ int[] columnPrivilegeBits = new int[] {0, 0, 0, 0, 0, 0, 0, 0, 0};
Review comment:
Yes, i will change this to booleans.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r431341213
##########
File path: metastore/scripts/upgrade/hive/hive-schema-4.0.0.hive.sql
##########
@@ -1700,7 +1701,18 @@ SELECT DISTINCT
P.`TBL_COL_PRIV`,
IF (P.`GRANT_OPTION` == 0, 'NO', 'YES')
FROM
- `sys`.`TBL_COL_PRIVS` P JOIN `sys`.`TBLS` T ON (P.`TBL_ID` = T.`TBL_ID`)
+ (SELECT
+ Q.`GRANTOR`,
+ Q.`GRANT_OPTION`,
+ Q.`PRINCIPAL_NAME`,
+ Q.`PRINCIPAL_TYPE`,
+ Q.`AUTHORIZER`,
+ Q.`COLUMN_NAME`,
Review comment:
Not all the columns just the ones here, these columns are needed for the view.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nareshpr commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
nareshpr commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r426342566
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFStringToPrivilege.java
##########
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.Text;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * UDFSplitMapPrivs.
+ *
+ */
+
+@Description(name = "split_map_privs", value = "_FUNC_(str, regex) - Splits binary str and maps to privilege type "
+ + "regex", extended = "Example:\n" + " > SELECT _FUNC_('0 1 1 0 1 1 0 0 0', ' ') FROM src LIMIT 1;\n"
+ + " [\"UPDATE\", \"CREATE\", \"ALTER\", \"INDEX\"]") class PrivilegeMap {
+ private Map<Integer, String> privilegeMap = new HashMap<Integer, String>();
+
+ Map<Integer, String> getPrivilegeMap() {
Review comment:
Can we use enum instead of map?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g closed pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g closed pull request #1007:
URL: https://github.com/apache/hive/pull/1007
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r431341379
##########
File path: metastore/scripts/upgrade/hive/hive-schema-4.0.0.hive.sql
##########
@@ -1700,7 +1701,18 @@ SELECT DISTINCT
P.`TBL_COL_PRIV`,
IF (P.`GRANT_OPTION` == 0, 'NO', 'YES')
FROM
- `sys`.`TBL_COL_PRIVS` P JOIN `sys`.`TBLS` T ON (P.`TBL_ID` = T.`TBL_ID`)
+ (SELECT
+ Q.`GRANTOR`,
+ Q.`GRANT_OPTION`,
+ Q.`PRINCIPAL_NAME`,
+ Q.`PRINCIPAL_TYPE`,
+ Q.`AUTHORIZER`,
+ Q.`COLUMN_NAME`,
+ `TBL_COL_PRIV_TMP`.`TBL_COL_PRIV`,
+ Q.`TBL_ID`
+ FROM `sys`.`TBL_COL_PRIVS` AS Q
+ LATERAL VIEW explode(split_map_privs(Q.`TBL_COL_PRIV`)) `TBL_COL_PRIV_TMP` AS `TBL_COL_PRIV`) P
Review comment:
Derby, postgres and mysql
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] aasha commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
aasha commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r422932739
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -95,16 +103,28 @@ private void addACLsToBag(
(int) (System.currentTimeMillis() / 1000), GRANTOR, PrincipalType.USER, false), authorizer));
break;
case COLUMN:
- privBag.addToPrivileges(
- new HiveObjectPrivilege(new HiveObjectRef(HiveObjectType.COLUMN, dbName, tblName, null, columnName),
- principal, principalType, new PrivilegeGrantInfo(acl.getKey().toString(),
- (int) (System.currentTimeMillis() / 1000), GRANTOR, PrincipalType.USER, false), authorizer));
+
+ int privilegeBit = acl.getKey().ordinal();
+ columnPrivilegeBits[privilegeBit] = columnPrivilegeBits[privilegeBit] == 0 ? 1 : 0;
Review comment:
This extra conditional check is not needed. Can set it to 1 directly
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r431342709
##########
File path: metastore/scripts/upgrade/hive/upgrade-3.1.0-to-4.0.0.hive.sql
##########
@@ -497,6 +497,201 @@ CREATE OR REPLACE VIEW `VERSION` AS SELECT 1 AS `VER_ID`, '4.0.0' AS `SCHEMA_VER
USE INFORMATION_SCHEMA;
+
+CREATE OR REPLACE VIEW `COLUMNS`
+(
+ `TABLE_CATALOG`,
+ `TABLE_SCHEMA`,
+ `TABLE_NAME`,
+ `COLUMN_NAME`,
+ `ORDINAL_POSITION`,
+ `COLUMN_DEFAULT`,
+ `IS_NULLABLE`,
+ `DATA_TYPE`,
+ `CHARACTER_MAXIMUM_LENGTH`,
+ `CHARACTER_OCTET_LENGTH`,
+ `NUMERIC_PRECISION`,
+ `NUMERIC_PRECISION_RADIX`,
+ `NUMERIC_SCALE`,
+ `DATETIME_PRECISION`,
+ `INTERVAL_TYPE`,
+ `INTERVAL_PRECISION`,
+ `CHARACTER_SET_CATALOG`,
+ `CHARACTER_SET_SCHEMA`,
+ `CHARACTER_SET_NAME`,
+ `COLLATION_CATALOG`,
+ `COLLATION_SCHEMA`,
+ `COLLATION_NAME`,
+ `UDT_CATALOG`,
+ `UDT_SCHEMA`,
+ `UDT_NAME`,
+ `SCOPE_CATALOG`,
+ `SCOPE_SCHEMA`,
+ `SCOPE_NAME`,
+ `MAXIMUM_CARDINALITY`,
+ `DTD_IDENTIFIER`,
+ `IS_SELF_REFERENCING`,
+ `IS_IDENTITY`,
+ `IDENTITY_GENERATION`,
+ `IDENTITY_START`,
+ `IDENTITY_INCREMENT`,
+ `IDENTITY_MAXIMUM`,
+ `IDENTITY_MINIMUM`,
+ `IDENTITY_CYCLE`,
+ `IS_GENERATED`,
+ `GENERATION_EXPRESSION`,
+ `IS_SYSTEM_TIME_PERIOD_START`,
+ `IS_SYSTEM_TIME_PERIOD_END`,
+ `SYSTEM_TIME_PERIOD_TIMESTAMP_GENERATION`,
+ `IS_UPDATABLE`,
+ `DECLARED_DATA_TYPE`,
+ `DECLARED_NUMERIC_PRECISION`,
+ `DECLARED_NUMERIC_SCALE`
+) AS
+SELECT DISTINCT
+ 'default',
+ D.NAME,
+ T.TBL_NAME,
+ C.COLUMN_NAME,
+ C.INTEGER_IDX,
+ cast (null as string),
+ 'YES',
+ C.TYPE_NAME as TYPE_NAME,
+ CASE WHEN lower(C.TYPE_NAME) like 'varchar%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^VARCHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ WHEN lower(C.TYPE_NAME) like 'char%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^CHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) like 'varchar%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^VARCHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ WHEN lower(C.TYPE_NAME) like 'char%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^CHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 19
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 5
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 3
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 23
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 53
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN regexp_extract(upper(C.TYPE_NAME), '^DECIMAL\\s*\\((\\d+)',1)
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN regexp_extract(upper(C.TYPE_NAME), '^NUMERIC\\s*\\((\\d+)',1)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 2
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 2
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN 10
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN 10
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) like 'decimal%' THEN regexp_extract(upper(C.TYPE_NAME), '^DECIMAL\\s*\\((\\d+),(\\d+)',2)
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN regexp_extract(upper(C.TYPE_NAME), '^NUMERIC\\s*\\((\\d+),(\\d+)',2)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'date' THEN 0
+ WHEN lower(C.TYPE_NAME) = 'timestamp' THEN 9
+ ELSE null END,
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ C.CD_ID,
+ 'NO',
+ 'NO',
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ 'NEVER',
+ cast (null as string),
+ 'NO',
+ 'NO',
+ cast (null as string),
+ 'YES',
+ C.TYPE_NAME as DECLARED_DATA_TYPE,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 19
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 5
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 3
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 23
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 53
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN regexp_extract(upper(C.TYPE_NAME), '^DECIMAL\\s*\\((\\d+)',1)
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN regexp_extract(upper(C.TYPE_NAME), '^NUMERIC\\s*\\((\\d+)',1)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 2
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 2
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN 10
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN 10
+ ELSE null END
+FROM
+ `sys`.`COLUMNS_V2` C JOIN `sys`.`SDS` S ON (C.`CD_ID` = S.`CD_ID`)
+ JOIN `sys`.`TBLS` T ON (S.`SD_ID` = T.`SD_ID`)
+ JOIN `sys`.`DBS` D ON (T.`DB_ID` = D.`DB_ID`)
+ LEFT JOIN `sys`.`TBL_COL_PRIVS` P ON (T.`TBL_ID` = P.`TBL_ID`)
+WHERE
+ NOT restrict_information_schema() OR P.`TBL_ID` IS NOT NULL
+ AND C.`COLUMN_NAME` = P.`COLUMN_NAME`
+ AND (P.`PRINCIPAL_NAME`=current_user() AND P.`PRINCIPAL_TYPE`='USER'
+ OR ((array_contains(current_groups(), P.`PRINCIPAL_NAME`) OR P.`PRINCIPAL_NAME` = 'public') AND P.`PRINCIPAL_TYPE`='GROUP'))
+ AND array_contains(split_map_privs(P.`TBL_COL_PRIV`),"SELECT") AND P.`AUTHORIZER`=current_authorizer();
+
+
+CREATE OR REPLACE VIEW `COLUMN_PRIVILEGES`
+(
+ `GRANTOR`,
+ `GRANTEE`,
+ `TABLE_CATALOG`,
+ `TABLE_SCHEMA`,
+ `TABLE_NAME`,
+ `COLUMN_NAME`,
+ `PRIVILEGE_TYPE`,
+ `IS_GRANTABLE`
+) AS
+SELECT DISTINCT
+ P.`GRANTOR`,
+ P.`PRINCIPAL_NAME`,
+ 'default',
+ D.`NAME`,
+ T.`TBL_NAME`,
+ P.`COLUMN_NAME`,
+ P.`TBL_COL_PRIV`,
+ IF (P.`GRANT_OPTION` == 0, 'NO', 'YES')
+FROM
+ (SELECT
+ Q.`GRANTOR`,
+ Q.`GRANT_OPTION`,
+ Q.`PRINCIPAL_NAME`,
+ Q.`PRINCIPAL_TYPE`,
+ Q.`AUTHORIZER`,
+ Q.`COLUMN_NAME`,
+ `TBL_COL_PRIV_TMP`.`TBL_COL_PRIV`,
+ Q.`TBL_ID`
+ FROM `sys`.`TBL_COL_PRIVS` AS Q
+ LATERAL VIEW explode(split_map_privs(Q.`TBL_COL_PRIV`)) `TBL_COL_PRIV_TMP` AS `TBL_COL_PRIV`) P
+ JOIN `sys`.`TBLS` T ON (P.`TBL_ID` = T.`TBL_ID`)
+ JOIN `sys`.`DBS` D ON (T.`DB_ID` = D.`DB_ID`)
+ JOIN `sys`.`SDS` S ON (S.`SD_ID` = T.`SD_ID`)
+ LEFT JOIN `sys`.`TBL_PRIVS` P2 ON (P.`TBL_ID` = P2.`TBL_ID`)
+WHERE
+ NOT restrict_information_schema() OR P2.`TBL_ID` IS NOT NULL
+ AND P.`PRINCIPAL_NAME` = P2.`PRINCIPAL_NAME` AND P.`PRINCIPAL_TYPE` = P2.`PRINCIPAL_TYPE`
+ AND (P2.`PRINCIPAL_NAME`=current_user() AND P2.`PRINCIPAL_TYPE`='USER'
Review comment:
I set up hive locally with different meta stores, modified the upgrade files with these changes and tested them using schematool.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] aasha commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
aasha commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r422903458
##########
File path: metastore/scripts/upgrade/hive/hive-schema-4.0.0.hive.sql
##########
@@ -1672,6 +1672,7 @@ FROM
JOIN `sys`.`TBLS` T ON (S.`SD_ID` = T.`SD_ID`)
JOIN `sys`.`DBS` D ON (T.`DB_ID` = D.`DB_ID`)
LEFT JOIN `sys`.`TBL_COL_PRIVS` P ON (T.`TBL_ID` = P.`TBL_ID`)
Review comment:
This needs to be removed. 2 left join
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r427037871
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -95,16 +103,27 @@ private void addACLsToBag(
(int) (System.currentTimeMillis() / 1000), GRANTOR, PrincipalType.USER, false), authorizer));
break;
case COLUMN:
- privBag.addToPrivileges(
- new HiveObjectPrivilege(new HiveObjectRef(HiveObjectType.COLUMN, dbName, tblName, null, columnName),
- principal, principalType, new PrivilegeGrantInfo(acl.getKey().toString(),
- (int) (System.currentTimeMillis() / 1000), GRANTOR, PrincipalType.USER, false), authorizer));
+
+ int privilegeBit = acl.getKey().ordinal();
+ columnPrivilegeBits[privilegeBit] = 1;
+ columnUpdateFlag = true;
+
break;
default:
throw new RuntimeException("Get unknown object type " + objectType);
}
}
}
+ if (columnUpdateFlag) {
+ String columnPrivilegeBitsString =
Review comment:
InformationSchemaWithPrivilegeTestBase.java used by TestInformationSchemaWithPrivilegeZookeeperSSL.java and TestInformationSchemaWithPrivilegeZookeeperPlain.java , will initially add the acls to mock a table. At this time, the privilege synchroniser is called to update the privileges. If the privilege string was wrong, the test would have failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] maheshk114 commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
maheshk114 commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r427021921
##########
File path: metastore/scripts/upgrade/hive/hive-schema-4.0.0.hive.sql
##########
@@ -1700,7 +1701,18 @@ SELECT DISTINCT
P.`TBL_COL_PRIV`,
IF (P.`GRANT_OPTION` == 0, 'NO', 'YES')
FROM
- `sys`.`TBL_COL_PRIVS` P JOIN `sys`.`TBLS` T ON (P.`TBL_ID` = T.`TBL_ID`)
+ (SELECT
+ Q.`GRANTOR`,
+ Q.`GRANT_OPTION`,
+ Q.`PRINCIPAL_NAME`,
+ Q.`PRINCIPAL_TYPE`,
+ Q.`AUTHORIZER`,
+ Q.`COLUMN_NAME`,
Review comment:
do we need to project all the columns ?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -78,6 +83,9 @@ private void addACLsToBag(
for (Map.Entry<String, Map<HiveResourceACLs.Privilege, HiveResourceACLs.AccessResult>> principalAcls
: principalAclsMap.entrySet()) {
String principal = principalAcls.getKey();
+ int[] columnPrivilegeBits = new int[] {0, 0, 0, 0, 0, 0, 0, 0, 0};
Review comment:
can this be changed to array of booleans or char ?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -43,8 +46,9 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+
/**
- * PrivilegeSynchronizer defines a thread to synchronize privileges from
+ * PrivilegeSynchronizer defines a thread to synchronize privileges from.
Review comment:
full stop is required ?
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/PrivilegeSynchronizer.java
##########
@@ -95,16 +103,27 @@ private void addACLsToBag(
(int) (System.currentTimeMillis() / 1000), GRANTOR, PrincipalType.USER, false), authorizer));
break;
case COLUMN:
- privBag.addToPrivileges(
- new HiveObjectPrivilege(new HiveObjectRef(HiveObjectType.COLUMN, dbName, tblName, null, columnName),
- principal, principalType, new PrivilegeGrantInfo(acl.getKey().toString(),
- (int) (System.currentTimeMillis() / 1000), GRANTOR, PrincipalType.USER, false), authorizer));
+
+ int privilegeBit = acl.getKey().ordinal();
+ columnPrivilegeBits[privilegeBit] = 1;
+ columnUpdateFlag = true;
+
break;
default:
throw new RuntimeException("Get unknown object type " + objectType);
}
}
}
+ if (columnUpdateFlag) {
+ String columnPrivilegeBitsString =
Review comment:
is there any test case already present to test this ?
##########
File path: metastore/scripts/upgrade/hive/upgrade-3.1.0-to-4.0.0.hive.sql
##########
@@ -497,6 +497,201 @@ CREATE OR REPLACE VIEW `VERSION` AS SELECT 1 AS `VER_ID`, '4.0.0' AS `SCHEMA_VER
USE INFORMATION_SCHEMA;
+
+CREATE OR REPLACE VIEW `COLUMNS`
+(
+ `TABLE_CATALOG`,
+ `TABLE_SCHEMA`,
+ `TABLE_NAME`,
+ `COLUMN_NAME`,
+ `ORDINAL_POSITION`,
+ `COLUMN_DEFAULT`,
+ `IS_NULLABLE`,
+ `DATA_TYPE`,
+ `CHARACTER_MAXIMUM_LENGTH`,
+ `CHARACTER_OCTET_LENGTH`,
+ `NUMERIC_PRECISION`,
+ `NUMERIC_PRECISION_RADIX`,
+ `NUMERIC_SCALE`,
+ `DATETIME_PRECISION`,
+ `INTERVAL_TYPE`,
+ `INTERVAL_PRECISION`,
+ `CHARACTER_SET_CATALOG`,
+ `CHARACTER_SET_SCHEMA`,
+ `CHARACTER_SET_NAME`,
+ `COLLATION_CATALOG`,
+ `COLLATION_SCHEMA`,
+ `COLLATION_NAME`,
+ `UDT_CATALOG`,
+ `UDT_SCHEMA`,
+ `UDT_NAME`,
+ `SCOPE_CATALOG`,
+ `SCOPE_SCHEMA`,
+ `SCOPE_NAME`,
+ `MAXIMUM_CARDINALITY`,
+ `DTD_IDENTIFIER`,
+ `IS_SELF_REFERENCING`,
+ `IS_IDENTITY`,
+ `IDENTITY_GENERATION`,
+ `IDENTITY_START`,
+ `IDENTITY_INCREMENT`,
+ `IDENTITY_MAXIMUM`,
+ `IDENTITY_MINIMUM`,
+ `IDENTITY_CYCLE`,
+ `IS_GENERATED`,
+ `GENERATION_EXPRESSION`,
+ `IS_SYSTEM_TIME_PERIOD_START`,
+ `IS_SYSTEM_TIME_PERIOD_END`,
+ `SYSTEM_TIME_PERIOD_TIMESTAMP_GENERATION`,
+ `IS_UPDATABLE`,
+ `DECLARED_DATA_TYPE`,
+ `DECLARED_NUMERIC_PRECISION`,
+ `DECLARED_NUMERIC_SCALE`
+) AS
+SELECT DISTINCT
+ 'default',
+ D.NAME,
+ T.TBL_NAME,
+ C.COLUMN_NAME,
+ C.INTEGER_IDX,
+ cast (null as string),
+ 'YES',
+ C.TYPE_NAME as TYPE_NAME,
+ CASE WHEN lower(C.TYPE_NAME) like 'varchar%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^VARCHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ WHEN lower(C.TYPE_NAME) like 'char%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^CHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) like 'varchar%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^VARCHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ WHEN lower(C.TYPE_NAME) like 'char%' THEN cast(regexp_extract(upper(C.TYPE_NAME), '^CHAR\\s*\\((\\d+)\\s*\\)$', 1) as int)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 19
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 5
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 3
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 23
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 53
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN regexp_extract(upper(C.TYPE_NAME), '^DECIMAL\\s*\\((\\d+)',1)
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN regexp_extract(upper(C.TYPE_NAME), '^NUMERIC\\s*\\((\\d+)',1)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 2
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 2
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN 10
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN 10
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) like 'decimal%' THEN regexp_extract(upper(C.TYPE_NAME), '^DECIMAL\\s*\\((\\d+),(\\d+)',2)
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN regexp_extract(upper(C.TYPE_NAME), '^NUMERIC\\s*\\((\\d+),(\\d+)',2)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'date' THEN 0
+ WHEN lower(C.TYPE_NAME) = 'timestamp' THEN 9
+ ELSE null END,
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ C.CD_ID,
+ 'NO',
+ 'NO',
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ cast (null as string),
+ 'NEVER',
+ cast (null as string),
+ 'NO',
+ 'NO',
+ cast (null as string),
+ 'YES',
+ C.TYPE_NAME as DECLARED_DATA_TYPE,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 19
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 5
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 3
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 23
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 53
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN regexp_extract(upper(C.TYPE_NAME), '^DECIMAL\\s*\\((\\d+)',1)
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN regexp_extract(upper(C.TYPE_NAME), '^NUMERIC\\s*\\((\\d+)',1)
+ ELSE null END,
+ CASE WHEN lower(C.TYPE_NAME) = 'bigint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'int' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'smallint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'tinyint' THEN 10
+ WHEN lower(C.TYPE_NAME) = 'float' THEN 2
+ WHEN lower(C.TYPE_NAME) = 'double' THEN 2
+ WHEN lower(C.TYPE_NAME) like 'decimal%' THEN 10
+ WHEN lower(C.TYPE_NAME) like 'numeric%' THEN 10
+ ELSE null END
+FROM
+ `sys`.`COLUMNS_V2` C JOIN `sys`.`SDS` S ON (C.`CD_ID` = S.`CD_ID`)
+ JOIN `sys`.`TBLS` T ON (S.`SD_ID` = T.`SD_ID`)
+ JOIN `sys`.`DBS` D ON (T.`DB_ID` = D.`DB_ID`)
+ LEFT JOIN `sys`.`TBL_COL_PRIVS` P ON (T.`TBL_ID` = P.`TBL_ID`)
+WHERE
+ NOT restrict_information_schema() OR P.`TBL_ID` IS NOT NULL
+ AND C.`COLUMN_NAME` = P.`COLUMN_NAME`
+ AND (P.`PRINCIPAL_NAME`=current_user() AND P.`PRINCIPAL_TYPE`='USER'
+ OR ((array_contains(current_groups(), P.`PRINCIPAL_NAME`) OR P.`PRINCIPAL_NAME` = 'public') AND P.`PRINCIPAL_TYPE`='GROUP'))
+ AND array_contains(split_map_privs(P.`TBL_COL_PRIV`),"SELECT") AND P.`AUTHORIZER`=current_authorizer();
+
+
+CREATE OR REPLACE VIEW `COLUMN_PRIVILEGES`
+(
+ `GRANTOR`,
+ `GRANTEE`,
+ `TABLE_CATALOG`,
+ `TABLE_SCHEMA`,
+ `TABLE_NAME`,
+ `COLUMN_NAME`,
+ `PRIVILEGE_TYPE`,
+ `IS_GRANTABLE`
+) AS
+SELECT DISTINCT
+ P.`GRANTOR`,
+ P.`PRINCIPAL_NAME`,
+ 'default',
+ D.`NAME`,
+ T.`TBL_NAME`,
+ P.`COLUMN_NAME`,
+ P.`TBL_COL_PRIV`,
+ IF (P.`GRANT_OPTION` == 0, 'NO', 'YES')
+FROM
+ (SELECT
+ Q.`GRANTOR`,
+ Q.`GRANT_OPTION`,
+ Q.`PRINCIPAL_NAME`,
+ Q.`PRINCIPAL_TYPE`,
+ Q.`AUTHORIZER`,
+ Q.`COLUMN_NAME`,
+ `TBL_COL_PRIV_TMP`.`TBL_COL_PRIV`,
+ Q.`TBL_ID`
+ FROM `sys`.`TBL_COL_PRIVS` AS Q
+ LATERAL VIEW explode(split_map_privs(Q.`TBL_COL_PRIV`)) `TBL_COL_PRIV_TMP` AS `TBL_COL_PRIV`) P
+ JOIN `sys`.`TBLS` T ON (P.`TBL_ID` = T.`TBL_ID`)
+ JOIN `sys`.`DBS` D ON (T.`DB_ID` = D.`DB_ID`)
+ JOIN `sys`.`SDS` S ON (S.`SD_ID` = T.`SD_ID`)
+ LEFT JOIN `sys`.`TBL_PRIVS` P2 ON (P.`TBL_ID` = P2.`TBL_ID`)
+WHERE
+ NOT restrict_information_schema() OR P2.`TBL_ID` IS NOT NULL
+ AND P.`PRINCIPAL_NAME` = P2.`PRINCIPAL_NAME` AND P.`PRINCIPAL_TYPE` = P2.`PRINCIPAL_TYPE`
+ AND (P2.`PRINCIPAL_NAME`=current_user() AND P2.`PRINCIPAL_TYPE`='USER'
Review comment:
have you done a upgrade test from 3.1 to 4.0 ?
##########
File path: metastore/scripts/upgrade/hive/hive-schema-4.0.0.hive.sql
##########
@@ -1700,7 +1701,18 @@ SELECT DISTINCT
P.`TBL_COL_PRIV`,
IF (P.`GRANT_OPTION` == 0, 'NO', 'YES')
FROM
- `sys`.`TBL_COL_PRIVS` P JOIN `sys`.`TBLS` T ON (P.`TBL_ID` = T.`TBL_ID`)
+ (SELECT
+ Q.`GRANTOR`,
+ Q.`GRANT_OPTION`,
+ Q.`PRINCIPAL_NAME`,
+ Q.`PRINCIPAL_TYPE`,
+ Q.`AUTHORIZER`,
+ Q.`COLUMN_NAME`,
+ `TBL_COL_PRIV_TMP`.`TBL_COL_PRIV`,
+ Q.`TBL_ID`
+ FROM `sys`.`TBL_COL_PRIVS` AS Q
+ LATERAL VIEW explode(split_map_privs(Q.`TBL_COL_PRIV`)) `TBL_COL_PRIV_TMP` AS `TBL_COL_PRIV`) P
Review comment:
on what all dbms its tested ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] simhadri-g commented on a change in pull request #1007: Hive 23361
Posted by GitBox <gi...@apache.org>.
simhadri-g commented on a change in pull request #1007:
URL: https://github.com/apache/hive/pull/1007#discussion_r427094821
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFStringToPrivilege.java
##########
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory;
+import org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.Text;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * UDFSplitMapPrivs.
+ *
+ */
+
+@Description(name = "split_map_privs", value = "_FUNC_(str, regex) - Splits binary str and maps to privilege type "
+ + "regex", extended = "Example:\n" + " > SELECT _FUNC_('0 1 1 0 1 1 0 0 0', ' ') FROM src LIMIT 1;\n"
+ + " [\"UPDATE\", \"CREATE\", \"ALTER\", \"INDEX\"]") class PrivilegeMap {
+ private Map<Integer, String> privilegeMap = new HashMap<Integer, String>();
+
+ Map<Integer, String> getPrivilegeMap() {
Review comment:
yes, will update the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org